People associate it with the most intrusive forms of online advertising including spam, toolbars, pop-ups/unders, misleading ads, annoying banners, and much scummier stuff.
It also doesn't help that is a strong association and overlap with the "get rich quick" folks.
There are definitely affiliates out there that add value, and affiliate programs in general can be a very profitable channel when managed properly. But there are MANY bad actors out there, and just as many eager newbies ready to take what they say as gospel based on promises of quick cash.
Possibly the largest affiliate network would be Amazon Associates. You sign up for an AA account, and you share your affiliate link for products with other people. If you do it casually (e.g. just recommending to friends and family on FB), then you can get some extra pocket change. If you do it professionally (e.g. actually finding a niche, doing market research, building a website w/ a capture page, etc), then you really need to work on promoting the products, which can be a full-time job in itself.
Of course, Amazon doesn't rely on its affiliate program to be big, but that's also what's great about being an Amazon affiliate--you don't really need to "sell" since everyone knows Amazon.
I can't think of any legit affiliate websites off the top of my head, but just imagine a site about dog training (a very popular affiliate niche). It has some free content on how to effectively train your dog (which could be valid or invalid), and in the content are some affiliate links. If the content is actually valid, and it works, and it's actually good content, I don't see how putting affiliate links on the article is spammy. I always appreciate it when a blog post links to a product it mentions because it saves me from having to Google it (but I will anyway to get a best price).
Of course, you can have complete spammy people, who have fake websites with false content (or crappily outsourced content) and comment random people's blogs for link bait, and unfortunately that's a practice that some "mega affiliates" promote in their "how i became a rockstar affiliate" products, but it's not fair to judge affiliate marketing as an industry based on a few bad eggs, no matter how numerous they may be.
It doesn't just come from the affiliate side. It can also come from the retailers and networks.
1) Not sure why all the hate towards us. We can take a company from zero to millions overnight. I get that there are some shady stuff floating around but that's not 100% of the industry.
2) Facebook + Teespring is almost a perfect combination.
3) Aff. Marketeers have created some issues in FB using Custom Audiences (T-shirts that said stuff like "This was made specifically for Joe"). Those sold like hot-cakes but FB went down hard on them.
4) Everyone wants a piece of the action, that's why you see so much tutorials and stuff around it. It's also a great way to learn Aff. Marketing if you are into that.