It's about incentives. The heartbleed bug was present in the code for almost two full years before someone who discovered it exercised responsible disclosure. It's possible that others have noticed it before this, but it's highly likely that the only people looking were security researchers, power users (like Google, the ones who first reported it to the authors), or actors looking to exploit it for their own agenda.

As it stands, average people (or average developers) have little incentive to go trawling through the existing body of open source code, mostly because they probably have better things to do with their time. In the commercial world, bug bounties attempt to skew the incentives to encourage the 'more eyes' part of the axiom 'given enough eyeballs, all bugs are shallow'.