There are at least some examples of startups choosing to apologize, and try to make things better in the future.

I think the real advice is “don’t say sorry unless you actually mean it.” People see right through that, because your actions and follow through won’t match your words. But people seem to appreciate sincere apologies that come with substantive corrective action.