Being open source doesn't actually save you from exploitable vulns related to integer arithmetic.

I enables independent, non-involved, non-interested parties to check it. Also when the protocol is open, it enables multiple implementations; keeping a known-by-few trojan style bug in all of them is specially difficult.

That's true. And yet, the linux kernel consistently has bugs like these in it. If you want exploitable vulns in literal media codecs go have fun taking a look at the history of ffmpeg.

I love open source. In so many ways it is uniquely responsible for the development of our technology landscape. It is observably not a meaningfully different path to secure code than closed source development.

I think that’s true of all software, people are fallible open source or not. I’d love to see average time to discovery and reporting in closed versus open source though. I’ve always heard it’s better in open source, which intuitively makes sense, and by the nature of closed source I think gathering the data will be challenging but valuable to see a tight comparison.

Lots of people have attempted this sort of analysis. You can find attempts at this in ICSE of FSE or whatever. But frankly there is no way to make effective science out of this. All of the data are always messy and make huge compromises to get anything even close to resembling an apples-to-apples comparison. I don't believe that anybody who claims it is meaningfully better in open source has any actual data really backing that up.

If you want my opinion, there is a huge gap between the tiny portion of open source projects that get any real professional scrutiny and the rest of the open source ecosystem. For something like the linux kernel, there are a lot of professionals who are deliberately focusing their novel tools at it and reporting issues. This is clearly better than nothing - though I'm not certain it is so much better than nothing to call it a big win. And this is the result of a large number of different teams all looking at this one codebase.

But pretty much immediately below "the linux kernel" in visibility, everybody stops caring. Even hugely deployed security-critical open source projects that manage media decoding and network stacks get absolutely zero professional analysis. All these projects get is the useless "drive-by CVE-report" garbage where somebody throws an off the shelf system at the repo and reports everything it spits out, no matter how useless the report.

Seems straightforward to compare open-source vs closed-source bug fix performance. The tricky part is adjusting for size. "Many eyes make all bugs shallow" but I wonder how many eyes are on the typical random java library that's been built into Spring Boot forever and no-one even thinks about anymore (there are probably 30+ of these). How often does any company using Spring Boot, even the very security conscious ones, look at the source of these dependencies, even when they update? I've heard Google code reviews literally all of their dependencies, but who else could afford to do that?

(In fact, this impossibility is a big part of why I'm so bullish on redbean - being really small means really fast and really really secure. It is a joy to deal with so few moving parts in a server!)

> Seems straightforward to compare open-source vs closed-source bug fix performance.

You can publish a best-paper in ICSE if you could pull this off. There are so many things that make this challenging. For starters, we don't even have the ground truth for what bugs exist. Even just looking at bugs we've already skewed our process dramatically based on the various different development processes of different projects.

You are right to question how many eyes are on typical random libraries. The answer is zero. Even huge libraries have extremely few eyes on them. When it comes to "many eyes" it is actually basically just the linux kernel and a very small number of other projects that get this sort of attention. The large majority of all open source projects, even those used by millions of projects, get zero meaningful attention beyond "hey I threw my tool at everything on github and spammed owners with nearly useless reports."

Good insight about the long tail of open source projects that don't have the same level of activity or interest from the developer community. I hadn't considered how sharply that drop off is, even for some what still widely used projects simply because the amount of people with the know how, and interest, to look for vulnerabilities is a lot smaller than the available project surface area.

I'm not even sure that "long tail" is the right phrase for it. I'd say "virtually all." The number of open source projects that get meaningful external scrutiny from security researchers is in the tens. Tens.

There is some automation out there. It is largely worthless. Some stuff is real like "hey, you've got a private key committed over here" but pretty quickly you run into high false positive rate garbage when looking at automated systems.

>Good insight about the long tail of open source projects that don't have the same level of activity or interest from the developer community

I don't think "long tail" is a good way to put it. Both OpenSSL and Log4j had millions of deployments and had pretty major bugs. I'd argue it's Linux then everything else.

It's definitely a lot better in memory safe languages (and especially in those applications that don't depend on C libraries under the hood). You can still have security bugs due to logic errors, but you won't ever get remote code execution or ability to read arbitrary memory. And in general bugs are much more likely to cause a crash rather than give the attacker access.

I suspect once C has been supplanted all the way down the stack it might actually be feasible to eliminate these kind of vulnerabilities entirely for apps where security is of utmost importance.

It is true that memory-safe languages are a massive massive massive boon! I believe that the entire industry needs to be making plans to find a way to shift all applications that operate on untrusted data away from C and C++. But this is completely orthogonal to the purported security benefits of making your source available.

IMO it’s not entirely orthogonal. One of the main benefits (from a security perspective) of open sourcing your app is to allow it to be audited more thoroughly. But even with that kind of auditing it’s hard to make thing secure in non memory safe languages. If you have both, then we might expect open sourcing to more reliably lead to an actually secure app.

I don't understand. Is the idea that memory errors are too hard to find but that once we've eliminated them at the language level that now auditors can review OSS projects effectively to verify that they are free of vulns? log4j should be a very clear example of how that'll fall over.

C should have been kept as a more type safe than a raw macro assembler.

The problem is that many think they need to write their application using a macro assembler to this day.

Actually, macro assemblers are better than C regarding security, because they don't do optimizations that wipe out code sections.

The difference in "who you have to trust" is reason enough.

If your concern is about deliberately inserted exploits by the WhatsApp developers, that's got virtually nothing to do with the topic at hand.

If this particular exploit had been deliberately inserted by WhatsApp developers, what would you expect to be different?

They are the only party who have access to the code. They don't even need to pretend about unintentional bugs. The world had a lesson with PRISM, if we learned anything... that is a different matter.

They didn't learned anything. The mantra is "i would not happen to me".

>Also when the protocol is open, it enables multiple implementations

Nice theory, but most of the time it's "just use zlib, bro, it's battle tested".

It saves you from obviously planted ones that can be found by code scanners.

Is there any evidence that this overflow was easily found with straightforward static analysis?

Seems you might be missing a key point - see, without transparent, open access to the source code, there is nothing easily found. At a certain point, if a murderer keeps "losing" the murder weapon, you might consider the evidence you find to be that of criminal obstruction. There is evidence that everything is more easily found when it's not hidden, obfuscated, or obstructed.

Sure. It is easier to throw an off the shelf analysis at source than worrying about binary decompilation with ghidra or whatever (well, for binaries - for bytecode it is almost exactly the same when given bytecode or source). But is this a meaningful difference? Real researchers, both academic and non-academic, do inspect open source code and report vulns they find. But this isn't actually actionable information from the perspective of a user who wants to make a risk assessment about their software choices. "Hey, you can run ${STATIC_TOOL} on this app" does not actually convert to "app is free from vulns." It just doesn't.

I love static analysis for vuln detection. I did my PhD on it. It remains my day job. It helps us find vulns. It doesn't actually convert us from unsafe software to safe software.

Even the App Store version of Signal is allegedly not the same as what's in the open source project. So unless you compile and install the applications yourself, there's no way of knowing anything.

That is why we must support initiatives like f-droid. They put a special focus on reproducibility.

We already know Signal team doesn't like F-Droid. They've got 100 totally outdated reasons why they won't put it there.

So Apple has their Xcode Build service, why not add a badge to verify that an app was built from a linked public Github/Gitlab Repo

if you can't trust Meta, why could you trust apple?

Apple has been building their brand on privacy and trust for at least a couple of years now. Can you be sure they're not sending everything to the NSA? Of course not. But they also make their money by directly charging users for services unlike the ad-based companies. There have also been many attempts by various governments to publicly force Apple to insert backdoors or prevent them from fixing security vulnerabilities which have failed.

> But they also make their money by directly charging users for services unlike the ad-based companies.

this does not make them more trustworthy

> There have also been many attempts by various governments to publicly force Apple to insert backdoors or prevent them from fixing security vulnerabilities which have failed.

Except in china, I suppose.

Apple's privacy is a marketing farce. They run data centers in China that provide full access to the government. Their anti-ad campaign was simply a push to gain dominance in the space themselves. They make a big fuss about end-to-end encryption but don't even bother to end to end encrypt your photos and backups!

I actually worked at Apple a few years ago in security. I was wondering why we didn't E2EE photos. The reason seemed to be - from what other engineers told me - is that it was at the behest of law enforcement. Lot easier to cooperate with LE and comply with NSLs when you can simply hand over the data they need.

Until Apple end-to-end encrypts these two things, it's all for naught. It doesn't fucking matter if your HomeKit data is E2EE if someone can take a look at your nudes without any cryptographic barrier.

Take that for what you will. Having worked at both companies during my career in a security capacity, I see no reason to trust one over the other wrt cloud services.

N.B. There are people at Apple that are very passionate about security and privacy. I was privileged to work with these people during my career. They really try to - and do - make a difference. My post is not an attack on them, but on the wider vision of the company, which is somewhat hypocritical.

I really need you to understand the difference between their marketing claims and reality. Apple is really not the champion for privacy they claim to be beyond the extent that they can try and hurt Google in their marketing.

Why would I think there is any truth in something apple's marketing department is saying?

Apple has a multi-billion dollar ads business and is going all-in to expand it.

That'd be cool.

If you're running iOS then I always assume the random number generator is backdoored by the NSA anyway. That's got to be the single juiciest target going; frankly if the NSA haven't backdoored that then what are they even spending tax dollars on?

That's interesting. Do you have any links for more info?

It's not a realistic danger and just fear mongering. I'm not sure why people on HN feel the need go after Signal so hard. I do think criticism is important (and Signal definitely deserves plenty) but these types of criticisms are off base and not specific to Signal, nor are they that relevant (kinda how people post on Signal's tweets about Iran complaining about lack of usernames. Not the time nor place).

It isn't meaningfully different from saying that Google/Apple can pretend to put the real App in the App Store but replace it with one that has a backdoor. This is entirely possible. But also the risk of this is extremely high and people do decompile apps like Signal, WhatsApp, and Telegram (albeit this can only go so far). These are all high profile and highly scrutinized apps. It is just fear mongering.

Before any backdooring purposes there is probably some marketing/analytics reasons, keys, OTF updates etc...

even if you compile yourself you can't be sure. [Reflections on Trusting Trust ](https://www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_Ref...)

Has that attack ever been observed in the wild?

While I don't know if the current incarnations of Nix/Guix will succeed, I think we are slowly making progress towards reproducible builds everywhere.

No one knows for sure, though compromised compilers are not far fetched - there has been an implicit trust on compiler toolchains. Reproducible builds are a few years out from full general adoption.

Assembly code can be read to see if it matches.

> Has that attack ever been observed in the wild?

Yes: https://www.quora.com/What-is-a-coders-worst-nightmare/answe...

Also, I remember in the 90's, people talking about a virus that infect pascal source code files. Memory is spotty about it.

> While I don't know if the current incarnations of Nix/Guix will succeed, I think we are slowly making progress towards reproducible builds everywhere.

Fortunately, the answer is also positive here.

Not with Guix and Mes.

Reproducible builds make an attack like this as likely as "the whole world is a big conspiracy".

There was an interesting case where a bunch of Android messenger things got a WebRTC based remote code execution[1]. Signal got dinged to the extent that an attacker could trigger it with no action on the user's part.

The root problem here is that users want lots of features. Each added feature, particularly super complex ones like video, takes away from security. There is not point in spending a lot of time on your own code if you are going to end up invoking a whole lot of code that you can't control.

[1] https://googleprojectzero.blogspot.com/2020/08/exploiting-an...

> The root problem here is that users want lots of features

Do devs have to implement these features in shitty memory-unsafe languages?