It's not corruption. It's what I'll call the interest-group problem.

Pennies are an issue a few people care deeply about and most people don't. It's electorally thrifty to accomodate those few, and so electeds do. It's an easy win, particularly in a partisan environment that punishes consensus building as betrayal of one's base.

Put another way, keeping the penny won't piss anyone off enough to get one primaried. Killing the penny might.