Not really. All the people not using a credit card are subsidizing your purchases so the impact on prices is negligible or non-existent.

VISA merchant rules generally guarantee that the plastic price will match the cash price. It's true that the merchant raises prices to compensate, but paying cash doesn't avoid that charge -- everyone pays the CC tax regardless of payment method. So you might as well get some of it back.