They do let you hide purchases so that they won't show up on in the music list on iOS and iTunes. They won't let you delete a purchase, which makes sense.

It's confusing because they're using purchase history as a basis for your entire library, which might not sync up with your mindset for your music list. I've definitely hidden past purchases that I know I'll never need easy access to.