Processing card transactions takes longer and reduces the serve rate of drinks. This is IMO probably a larger factor than the processing cost - cash has a processing cost too (staff time, security, fraud, bank charges) and it seems pretty close to the card processing cost.
When I was in France I would sometimes pay in a bakery with my debit card for a couple of euros.
Actually, I believe handling cash costs them money too and card processing fees might be less than what cash costs them.
I've heard several small shop owners say they prefer debit cards to cash since they claim it reduces the risk of robbery. The more people pay by card, the less cash there is in the till and thus the less reason to try to rob a place.
IE, your choice of supermarket is more elastic than your choice of pub.
Bitcoin seems to be the closest thing we have to disrupting this, but the chicken and egg problem isn't going anywhere.