If Linus's Law and so on really did rely on people encountering the bugs by chance in everyday use of the software, it's no wonder (in hindsight) this doesn't help with a lot of the security bugs we face, many of which would never be triggered randomly in normal use, but rather require constructing elaborate attack scenarios. Even those that might occur by chance are not likely to be repeatable, and so not likely to get reported or analyzed.
Maybe this points to a change in our prototypical concept of a "bug". When ESR first wrote "The Cathedral and the Bazaar", I would have associated "bug" with something like "the TIFFs produced by program A can't be read by program B", or "program C seems to crash if you have a non-ASCII filename", or "program D drops its network connections if more than 65536 packets are received". Today, I would associate it with something like "an attacker who sends a certificate containing an extension with invalid ASN.1 encoding that follows one or more syntactically valid extensions that are marked Critical and that are unknown to the user-agent can get remote code execution" or "an attacker who sends an XML payload that is parsed correctly by library A and incorrectly by library B due to discrepant handling of Unicode escaping can request operations that should be forbidden".
Well, a literal hot-off-the-presses example would be a bug in handling multibyte characters in the regular expression library in Flash Player, which was exploitable:
http://googleprojectzero.blogspot.com/2015/02/exploitingscve...
In other words, these bugs are often complicated artifacts that require research to find and malice to use -- not annoyances and breakages that are frustrating end users every day.
Given the universal quantifier, one would assume that ESR meant security bugs here as well.
Also, it is stated as a law, but ESR did not back it up with much statistical evidence to support it. I would be surprised if it holds in reality for all bugs.
Extra questions:
- Does releasing early and often lead to more (non-shallow) eyeballs? Not really: see Heartbleed or the OpenSSL RNG bug in Debian.
- Do you need more eyeballs, or just the right eyeballs? (Given the right eyeballs, all bugs are shallow)
- Isn't the effect of moving to a safer language larger?
The existence of those bugs doesn't disprove the claim. You'd need to show that software that is not released early and often doesn't have (even) fewer eyeballs.
Generally speaking, 'eyeballs' aren't actively looking for problems. They're passively using software and noticing when something seems out of place. Many security bugs will never seem out of place, and are only discoverable when someone is intentionally probing for security bugs.
Mozilla has passed the 1 million bug mark.
Some possibly more useful statistics:
For the "Core" product (read: Gecko), there are as of right now 52k open bugs and 230k closed bugs.
For the "Firefox" product (desktop, not Firefox android), there are 21k open bugs and 128k closed bugs.
Note that these "bugs" include feature requests, so unless you think the number of requested features will decline over time....
I know of at least one study that contradicts this claim:
"We found that with shorter release cycles, users do not experience significantly more post-release bugs and bugs are fixed faster, yet users experience these bugs earlier during software execution (the program crashes earlier)."
http://www.catb.org/esr/writings/homesteading/cathedral-baza...