Plenty of places (in the U.S. at least) charge a credit card fee, or require a minimum transaction. Also, as the grandparent mentioned, the cost of processing credit cards gets inevitably passed on to the customer. There was an article a while back about a local business owner who had fought the necessity to process credit cards for years (http://www.oregonlive.com/portland/index.ssf/2013/07/northwe...). Businesses that don't process credit cards presumably don't have that cost to pass on, but could still accept bitcoin as a non-cash method of payment.
Very few places charge an extra fee for using a credit card.
Honestly, I trust my credit card company more than most merchants I do business with. I'd rather have them be able to claw back the payment if something goes wrong. I think that I'm way more likely to be screwed by the other party in the transaction than I am by the trusted third party.