Retailers often include their half of CC fees in the price you pay at the till, regardless of whether a cc is used. This is on top of any fees you may pay.
Since it's the retailer deciding to leave their CC fees in the retail price, I'm getting back my money on my purchase.
It's the retailer who's keeping the extra few percent on their prices, not visa, or you, when someone is paying cash.
The questions of ethics is interesting. While I appreciate the morality checks from strangers at any time, if the retailers are pocketing the extra %, instead of providing a cash/debit price, should disdain towards 'ill-gotten gains' be focused on each 'mildly unethical' retailer you are dealing with?
That feels like "not solving the problem"
Actually it feels like the prisoners dilemma: if everyone chooses what's best for themselves the we're worse off overall...
What might be interesteing is a till discount at each retailer for not using a credit card.
I preferred to use debit for a long time too. As fraud increases, I'd rather the bank be hit with fraud on a CC instead of my bank account. It's not the best insurance, but banks are far more motivated to get their own money back quickly over their customers'.
When you're paying with cash - you're giving a little bit more profit to the retailer.
This is also an issue for the under privileged who do not have access to meaningful credit to get cash back.
Not sure if that might change the perspective.