Also: Your relatives and friends. And their friends and friends' friends. Your neighbors. Ask around. And whenever someone moves, they usually want to get rid of things.
For specific music Discogs.org is a very good marketplace. It's not free, it's a second hand marketplace, people don't give away things, but prices aren't worse than retail except for actually rare collector's stuff. But I've used it a lot to fill in the gaps in some artists' discography. Also older CDs are usually mastered with a bigger dynamic range, so they sound better on music hardware with a functioning volume knob. ;) So I much prefer buying original CD releases versus re-releases of an album on CDs anyway.
