You're the affiliate sites customer. You're the vendors customer. You're the VISA/Mastercard's customer. You're the gateway/merchant account providers customer.
People need to learn to accept that you simply aren't Amazon's customer unless you buy direct from them, in very much the same way you aren't eBay's customer.
Second: Using their store makes you their customer. If I go into a store and buy a product, giving them my credit card number to purchase merchandise they advertised, that store can't claim that I'm not really their customer, that they're just a distribution platform for the manufacturer, and that they're not responsible for the counterfeit they just sold me.
Second: The courts strongly disagree with you. You are drawing a thin line between marketplace and vendor. As far as the courts are concerned, they are not the same thing, and you absolutely do not have the exact same protections you would in purchasing directly.