There is also a good chance that this is just not the right kind of content for this community.
How would you feel if Referly much more clearly said "hey, if you decide to buy through my links that is totally up to you - but if you do I am going to earn 6-8% commission. This doesn't change the price of your products, but I hope it will serve as a nice tip to me for organizing all this info in one place for you. If not, no worries"
1) If the author of a book writes a ~10 page article (free) or posts a chapter, and includes a link to "buy my book", I have zero problem. In fact, I'm glad he made it easier for me to buy the book from Amazon, saving me a step.
2) If someone sets up a site (like Box O' Truth, an ammunition/firearms performance test), then includes links to purchase tested products (and potentially discloses that Federal gave him a bunch of Flitecontrol #00, and Benelli loaned him an M1 for testing), I have no problem buying through those links.
3) An author who has a following (say, a known expert in cycling) who throws some links to products he likes and reviews, especially if it's a product he owns already.
4) Someone who is a domain expert and reviews products he doesn't really like, slightly disingenuously. Although in some cases it makes sense for say a skiing expert to review a beginner product that he personally doesn't use; that's closer to 3. A Canon shooter who includes a Nikon body for comparison (without actually reviewing it) is more like a 4 or 4.5.
...
6) Random lists of products, like "top 10 new items for the new year" from publications like Stuff or Wired or whatever, largely selected through vendor promotion and who buys display ads.
...
9) Ubuntu stripping out referrals from some packages and replacing them with their own (some audio package?)
10) Malware which secretly appends an affiliate tag to everything a browser goes to, without a choice.
Antipathy starts around 5, and jihad starts around 8.
I think Referly in this case is around a 3, and in the worst case is around a 5 or 6.
You might be able to save some credibility by also including non-affiliate links so users are given an easy-to-act-on choice, but I cringe for even possibly helping.
This is also why Klout is such a bunch of crap.
A simple and easy first step to making your site seem less spammy would be to register a whole bunch of different domains composed of relevant keywords. These can be used to mitigate being banned from link aggregator sites and to delay users noticing the pattern. Markov chain generated text surrounding the links would do wonders on the less savvy users, too. I'm sure there are plenty more of such tips if you hang out in the right forums.
What you fail to realize is that credibility has been destroyed by each and every detail of the situation. The earnest blog that has the occasional spammy referral post can be mostly forgiven (although the users that upvote it shouldn't be), as it's clear that the writer has a life purpose besides referral links. I see no such redeeming qualities here.
"Should know better" refers to the fact that you present yourself as part of the tech community, while simultaneously disrespecting the general consensus on spam. Although maybe it's just that the shark that was jumped has died of old age.
Oh, I just remembered it's Sunday - I've got to go do a bit of sanitation hacking.