That's true. And yet, the linux kernel consistently has bugs like these in it. If you want exploitable vulns in literal media codecs go have fun taking a look at the history of ffmpeg.

I love open source. In so many ways it is uniquely responsible for the development of our technology landscape. It is observably not a meaningfully different path to secure code than closed source development.

I think that’s true of all software, people are fallible open source or not. I’d love to see average time to discovery and reporting in closed versus open source though. I’ve always heard it’s better in open source, which intuitively makes sense, and by the nature of closed source I think gathering the data will be challenging but valuable to see a tight comparison.

Lots of people have attempted this sort of analysis. You can find attempts at this in ICSE of FSE or whatever. But frankly there is no way to make effective science out of this. All of the data are always messy and make huge compromises to get anything even close to resembling an apples-to-apples comparison. I don't believe that anybody who claims it is meaningfully better in open source has any actual data really backing that up.

If you want my opinion, there is a huge gap between the tiny portion of open source projects that get any real professional scrutiny and the rest of the open source ecosystem. For something like the linux kernel, there are a lot of professionals who are deliberately focusing their novel tools at it and reporting issues. This is clearly better than nothing - though I'm not certain it is so much better than nothing to call it a big win. And this is the result of a large number of different teams all looking at this one codebase.

But pretty much immediately below "the linux kernel" in visibility, everybody stops caring. Even hugely deployed security-critical open source projects that manage media decoding and network stacks get absolutely zero professional analysis. All these projects get is the useless "drive-by CVE-report" garbage where somebody throws an off the shelf system at the repo and reports everything it spits out, no matter how useless the report.

Seems straightforward to compare open-source vs closed-source bug fix performance. The tricky part is adjusting for size. "Many eyes make all bugs shallow" but I wonder how many eyes are on the typical random java library that's been built into Spring Boot forever and no-one even thinks about anymore (there are probably 30+ of these). How often does any company using Spring Boot, even the very security conscious ones, look at the source of these dependencies, even when they update? I've heard Google code reviews literally all of their dependencies, but who else could afford to do that?

(In fact, this impossibility is a big part of why I'm so bullish on redbean - being really small means really fast and really really secure. It is a joy to deal with so few moving parts in a server!)

> Seems straightforward to compare open-source vs closed-source bug fix performance.

You can publish a best-paper in ICSE if you could pull this off. There are so many things that make this challenging. For starters, we don't even have the ground truth for what bugs exist. Even just looking at bugs we've already skewed our process dramatically based on the various different development processes of different projects.

You are right to question how many eyes are on typical random libraries. The answer is zero. Even huge libraries have extremely few eyes on them. When it comes to "many eyes" it is actually basically just the linux kernel and a very small number of other projects that get this sort of attention. The large majority of all open source projects, even those used by millions of projects, get zero meaningful attention beyond "hey I threw my tool at everything on github and spammed owners with nearly useless reports."

Good insight about the long tail of open source projects that don't have the same level of activity or interest from the developer community. I hadn't considered how sharply that drop off is, even for some what still widely used projects simply because the amount of people with the know how, and interest, to look for vulnerabilities is a lot smaller than the available project surface area.

I'm not even sure that "long tail" is the right phrase for it. I'd say "virtually all." The number of open source projects that get meaningful external scrutiny from security researchers is in the tens. Tens.

There is some automation out there. It is largely worthless. Some stuff is real like "hey, you've got a private key committed over here" but pretty quickly you run into high false positive rate garbage when looking at automated systems.

>Good insight about the long tail of open source projects that don't have the same level of activity or interest from the developer community

I don't think "long tail" is a good way to put it. Both OpenSSL and Log4j had millions of deployments and had pretty major bugs. I'd argue it's Linux then everything else.

It's definitely a lot better in memory safe languages (and especially in those applications that don't depend on C libraries under the hood). You can still have security bugs due to logic errors, but you won't ever get remote code execution or ability to read arbitrary memory. And in general bugs are much more likely to cause a crash rather than give the attacker access.

I suspect once C has been supplanted all the way down the stack it might actually be feasible to eliminate these kind of vulnerabilities entirely for apps where security is of utmost importance.

It is true that memory-safe languages are a massive massive massive boon! I believe that the entire industry needs to be making plans to find a way to shift all applications that operate on untrusted data away from C and C++. But this is completely orthogonal to the purported security benefits of making your source available.

IMO it’s not entirely orthogonal. One of the main benefits (from a security perspective) of open sourcing your app is to allow it to be audited more thoroughly. But even with that kind of auditing it’s hard to make thing secure in non memory safe languages. If you have both, then we might expect open sourcing to more reliably lead to an actually secure app.

I don't understand. Is the idea that memory errors are too hard to find but that once we've eliminated them at the language level that now auditors can review OSS projects effectively to verify that they are free of vulns? log4j should be a very clear example of how that'll fall over.

C should have been kept as a more type safe than a raw macro assembler.

The problem is that many think they need to write their application using a macro assembler to this day.

Actually, macro assemblers are better than C regarding security, because they don't do optimizations that wipe out code sections.

The difference in "who you have to trust" is reason enough.

If your concern is about deliberately inserted exploits by the WhatsApp developers, that's got virtually nothing to do with the topic at hand.

If this particular exploit had been deliberately inserted by WhatsApp developers, what would you expect to be different?

They are the only party who have access to the code. They don't even need to pretend about unintentional bugs. The world had a lesson with PRISM, if we learned anything... that is a different matter.

They didn't learned anything. The mantra is "i would not happen to me".

>Also when the protocol is open, it enables multiple implementations

Nice theory, but most of the time it's "just use zlib, bro, it's battle tested".