A lot of the fee goes to the customer's bank that funded the transaction, which is what enables Credit Card rewards programs. If you limit that fee (like the E.U. has), Visa et al. don't go out of business, but then you won't see any similar level rewards programs like we have here in the US (e.g. 2% back on everything, or 3-5% on specific categories, etc).