The bargain basement part came from sound quality and being able to buy single songs. Apple's iTunes does have $.69 and $.79 songs along with bargain album prices of $6.99 (on the front page with links to cheap albums under the category headings near the bottom of the front page).
I just went on the iTunes store and picked random albums from the front page, a few to be extra thorough, and you know what I found? Most albums are $9.99 or $12.99 with single tracks for $1.29. Maybe it's because I'm browsing "chart toppers" but after clicking through a couple of albums, I honestly don't see any songs for $0.99.
Where can you actually buy music online that is any more expensive than iTunes, if it's such a bargain?