One of the bosses I had at a big bank told me the one thing she observed about successful people was that they never got hung up on mistakes. If they made a mistake, they just moved on as if it didn't happen, but would apologize if you asked sincerely one on one.

I think that's a pretty good middle ground. Apologizing too readily might also signal you lack self-respect or confidence which may invite even more vitriol.