News for hackers. Stuff that matters.
Re:Messenger systems compared by security, privacy, compatib... by lrvick on Tuesday October 16, 2018 @12:41PM (#18230811)
Please make comments on individual cells for improvements to be seen/added more easily. This is obviously big research undertaking that got thrown together last weekend :)

> Another example: "open server" and "on-premise" says nothing about whether or not you really want to run one of those instances. It just says that hypothetically one could.

I know a number of people that run matrix.org servers for personal use and companies. The entire French government runs on riot.im/matrix.org.

When security and really privacy matters, you don't want a third party being able to push updates to your clients/servers at any time without warning.

> None of them implement even close to the privacy features Signal has implemented. But on this diagram it is clearly better because there is more green and less red.

What features specifically? Happy to add more columns if signal really has anything unique to offer here.

The things Signal gets red marks on are pretty fair though imo, and things others do better.

> Use WhatsApp to talk to normal people.

I think you will find many options above WhatsApp on the list in terms of security and privacy that have clients that are every bit as simple to use.

Other than their (very) effective marketing advantage, -why- would you encourage people towards these respective walled gardens instead of more open alternatives listed?

Re:Messenger systems compared by security, privacy, compatib... by lvh on Tuesday October 16, 2018 @02:29PM (#18231891)
In order to comment on individual cells, we appear to first have to have an argument about how audits work. You say WhatsApp can only "claim" certain features as a consequence of it being closed source, but that's because of a misunderstanding about how audits work.

In a backchannel, as a consequence of this HN article, someone (names withheld to protect the guilty, they can identify themselves if they'd like) started looking at Dust and figured out the key store password is a hardcoded, short, ASCII string and the messages are encrypted with unauthenticated AES-CBC. They did not need the source code to do that.

JP Aumasson phrased it more eloquently than I could: https://research.kudelskisecurity.com/2018/10/02/open-source...

Re:Messenger systems compared by security, privacy, compatib... by lrvick on Tuesday October 16, 2018 @03:50PM (#18232601)
Even if they release all their code at this moment to a dozen third party code auditing firms, they can then undermine all the verified security with the next closed binary they release.

I will never use Signal with their current direction and don't recommend anyone use it, but they get credit where it is due. They actually offer source code for their walled garden and allow that their basic crypto can -generally- be verified except for extreme cases like my other comment.

Allo, Whatsapp and other closed systems that give you no reason to trust them other than faith in the people advocating them and that their engineers got it 100% right. Their claims can't be verified so they can only be marked as just that, claims.

Sure, plenty of obvious flaws can be spotted without source code access, but many are hard to find even if you -do- have source code to the point they would probably have never been found were they not open to allow the right set of eyes to eventually read the right section of code (Heartbleed etc).

I found random number generation flaws in Terraform I would of -never- found without source code access. It is for this reason I trust Terraform and Hashicorp quite a bit. They normally get it right, but are not afraid to have other people audit and point out flaws because they don't have this arrogant idea their engineers will get everything right 100% of the time.

Security is -hard- and anyone that thinks they can get it right with a SPOF closed source approach is more interested in marketshare than security.

Trust, but verify. Likewise if you are not even allowed to verify, you should instantly distrust.

Re:Messenger systems compared by security, privacy, compatib... by lvh on Tuesday October 16, 2018 @03:54PM (#18232651)
> Allo, whatsapp and other closed systems that give you no resaon to trust them other than faith in the people shilling them. Their claims can't be verified so they can only be marked as just that, claims.

I have repeatedly refuted that point and you have repeatedly ignored it.

> Sure, plenty of obvious flaws can be spotted without source code access, but many are hard to find even if you -do- have source code to the point they would probably have never been found were they not open to allow the right set of eyes to eventually read the right section of code (Heartbleed etc).

You are implying that no source code prevents you from finding obvious flaws, and I have given you two counterexamples in a messaging service that came up in this thread that I had never heard of before. That was casual peeking, not even a serious audit.

> I found random number generation flaws in Terraform I would of -never- found without source code access.

Do you professionally audit software? The fact that you can't find a bug without source does not mean that no-one can. Project Zero finds bugs in Windows and Edge every other day that are a lot more complicated than figuring out what RNG Terraform uses and how Terraform uses it.

> Security is -hard- and anyone that thinks they can get it right with a SPOF closed source approach is more interested in marketshare than security.

Since you've impugned my motives in two separate places in that post (referring to me as a "shill"), I am no longer interested in discussing this with you. I'm sure people can make up their minds from the thread.

Re:Messenger systems compared by security, privacy, compatib... by lrvick on Tuesday October 16, 2018 @04:12PM (#18232828)
I changed the word "shill" in an edit right after I posted as that was unfair/unhelpful and realized you might think it was directed at you. It was not. I welcome this type of debate personally.

> Do you professionally audit software?

I do as a matter of fact. I'll be honest it is normally much easier in closed products as I know what to look for. It is generally much harder to find flaws in popular open source systems as someone else has usally long beaten me to the low hanging fruit in critical codepaths.

I have been burned and seen others burned so many times by closed software and teams that ship security regressions that I now personally use only open tools I can audit at some level, or can audit the auditing and reproducible build process. I have seen "security" companies cut corners too many times in order to feature farm to trust anything that won't let me see the source code.

I have even professionally audited multiple systems on the spreadsheet myself, some of which I am aware of vulnerabilities for currently under embargo.

So far you have cited examples which I not only didn't ignore but responded to in the form of counter examples.

This is however turning into a open source vs closed source debate with subjective evidence, which in the end always boils down to if you have blind faith in a small group of people or not.

In the spreadsheet I tried to only include things that could be fairly objective, but when we get into concepts of trust of binaries and their authors, it gets muddy to be sure, and we will end up choosing paths based on our own threat profiles and experience.

This is why I tried to include -everything- in the spreadsheet I am aware of, so people with threat profiles different than mine can make informed choices.

Re:Messenger systems compared by security, privacy, compatib... by tptacek on Tuesday October 16, 2018 @05:29PM (#18233461)
You professionally audit secure messaging applications? For what kinds of vulnerabilities? I have turned down invitations to audit some of these applications because I didn't feel qualified to render an assessment (I've been doing professional software security assessment, of closed/open source applications, since 1996). Did you take money for those assessments? Which ones did you do? I'd like to take a closer look at some of them sometime.
Re:Messenger systems compared by security, privacy, compatib... by lrvick on Tuesday October 16, 2018 @07:03PM (#18234044)
I audit tools myself, my employers, or partners consider and if I find obvious flaws then I file bugs for those products and generally we either don't use them or limit their use.

I feel qualified only to certify that a tool is obviously insecure, but I don't think -anyone- will ever be qualified to solo certify something as totally secure. (but sadly that is how clean audits are generally read). I have in the past hired 3-4 audit firms and each would find flaws the others did not, and that mine did not, but fail to spot flaws that mine did. No one has the full picture.

The only relevant one not under embargo atm was my recent casual 2-3 hour audit of Lifesize which I found right away had a number of alarming issues the company would not address. Some of these may of been found to be non issues under further inspection, but there was more than enough to assert security was not a major focus of the platform.

This cursory look was all it took for me to feel confident in ending any consideration of that product to protect the interests of the entity considering it.

I reported all these to the company and there was no response for over 90 days and made my findings public after due warning.

They are available here: https://gist.github.com/lrvick/6e600d8484cfb415d1e2b06e8b345...

(reminder: this was only 2-3 hours of work and by all means grain of salt)

Re:Messenger systems compared by security, privacy, compatib... by tptacek on Tuesday October 16, 2018 @10:25PM (#18235567)
I think you probably understand where I am coming from at this point. When I think about "audits" for secure messengers, I think about professional, specialized, contracted assessments with dedicated teams of experts (I'm a little biased, since that used to be my line of work). And my two objections to this discussion are:

1. I think LVH is right to point out that "audit" doesn't have much meaning in the document you've produced; it gloms together assessments of wildly differing depth and quality and condenses them all to a single pass/fail.

2. Secure messaging security is hard, much harder than secure transports (which you alluded to in the other subthread when you mentioned TLS). There are in fact not that many people in the world who can do a proper cryptographic assessment of a messenger at this point (not because it's prohibitively difficult; it should be well within the reach of everyone with a graduate degree in cryptography who enjoys coding --- rather, just because it's a specialized skill set that not many people have an opportunity to get good at). Like I said: I wouldn't say I'm qualified to perform such an assessment (LVH, different story). And all that "just" gets you the cryptography! If your messenger gets popular, the framework it's built on becomes one of the most important targets on the Internet!

So I get itchy when people say things that amount to "I've audited things, I have an authoritative opinion about this stuff". Probably not? Maybe, but, like, if we were going to place a bet...?

You can turn that logic around on me. But I'm not the one making an extraordinary claim. You are. So: what really bugs me is the idea that this is a problem domain you can reduce to a Wikipedia-style comparison chart. That kind of chart will get people hurt.

Re:Messenger systems compared by security, privacy, compatib... by lrvick on Wednesday October 17, 2018 @03:40AM (#18236982)
I don't know that I made any extrodinary claims.

You did ask me if I audited things before and I answered honestly. I frequently report vulnerabilities in a range of open and closed systems and read a lot about those others find. I also don't consider myself an expert and generally distrust people that claim they are in this space because it is, as you say, really hard.

I initially started a spreadsheet to document the very high level objective traits of messengers I find useful and others found it interesting/useful and helped extend it.

It is far too much cognitive overhead to do deep dive evaluation of 75+ messengers, but what this list allows one to do is quickly eliminate services from consideration that lack features vital to address their use cases, platform targets, or threat profile: for instance the ability to self host, or end to end encryption.

From there once you have your short list, you can then make more effective use of time reading code, doing audits, reviewing audits of others.

If a list like this simply makes someone aware of new up and coming projects in the space, or old ones that have been quietly evolving in capability... then I feel justified in having shared it instead of having kept it as a private document of my personal notes.

I for one learned about a number of new projects when putting this sheet together, and interesting new approaches on solving these problems.

The thing that is pretty subjective about the list is how I sorted items based on my own research and threat profile.

I won't sit here and say Matrix or briar or any other items near the top are perfect, or lacks any security flaws. I personally place my bets on Matrix at this point based on my deep dive evaluation of it and similarly featured alternatives, but that is subject to change! Others are free to share your view that a well funded but closed source messenger like whatsapp is the general best bet.

To sober up on this topic a bit: Matrix has had glaring security flaws in the past, but also so have the options you personally recommend like Signal and Whatsapp.

At the end of the day all one can do is collect data, look over all the options, deep dive into the relative merits and claims of each to the extent one can, look at the research provided by others, and make a judgement call.

This list should be considered a starting point for research and a way to discover lesser known projects as it was for me.

It should -not- be seen as an end all "use this it is most secure" recommendation by any means. Anyone that looks at any one high level list of binary data points and makes an exclusive choice on that alone is doomed to shoot themselves in the foot with or without my help.

Hopefully that addresses your concerns about my intent here.

Re:Messenger systems compared by security, privacy, compatib... by pvg on Wednesday October 17, 2018 @12:38PM (#18240797)
I don't know that I made any extrodinary claims.

You published a spreadsheet listing messaging apps 'ordered by security' in which Signal, Whatsapp and iMessage are shown as way less secure than IRC. It also still says, and you've repeatedly argued here, that things like whether, say, WhatsApp uses E2E encryption is essentially unknowable.

Re:Messenger systems compared by security, privacy, compatib... by lrvick on Thursday October 18, 2018 @01:09PM (#18250104)
IRC is lacking in features but setup correctly with modern OTR I stand by it having easy to reason about security advantages Whatsapp and iMessage do not. (Usability is another story)

Notably if there was a blackbox audit of Whatsapp or iMessage 6 months ago you have no path to easily check if a blatant backdoor was introduced in the build you installed last night. You also can't know if there were obvious flaws in the code the whole time that would be very hard spot in blackbox testing if you didn't know what to look for. Maybe the app build from last night leaks its key intentionally via very subtle steganography in metadata?

Compare to a binary I installed via F-Droid that I can confirm was built reproducibly from a given git head I can go see the code review for.

To use a simple analogy: I can see the exact ingredients of what went into -my- meal instead of what went into the meal of the food inspector.

This allows much deeper release accountability and -is- a major security feature iMessage and Whatsapp lack worth flagging.

Verifying security with source code is hard enough. Without source code it is substantially harder and I for one have no interest in using or recommending security tools that fail to be 100% transparent about how they work.

Without source code all we have are claims that can never be thoroughly verified.

Re:Messenger systems compared by security, privacy, compatib... by tptacek on Thursday October 18, 2018 @02:15PM (#18250704)
You keep making this argument and ignoring experts who tell you it is bogus. The whole thread is there for everyone to read; you can pretend you haven't read the rebuttals, but you can't pretend for everyone else. The idea that "without source code we can't thoroughly verify things" is false, and at odds with basically all of modern software security.
Re:Messenger systems compared by security, privacy, compatib... by lrvick on Friday October 19, 2018 @04:25AM (#18255056)
Oh, I read them. I simply firmly disagree with them and my personal experience of 15+ years finding and fixing security issues flys in the face of your statements. We clearly test software for bugs very differently.

I made specific arguments and use cases to justify my position and you have simply told me I am wrong without directly addressing them.

Once again, I find the term "expert" overrated. I for one admit I am not an expert on security, a field that is already hard enough on auditors like myself without withheld source code.

I have also worked with a half dozen or so security auditing firms all of which stated source code access would make their job much easier.

It didn't take me hours of blackbox testing for me to find CVE-2018-9057. It took me 20 minutes of reading code on Github half asleep at 4am because I was curious about an unrelated bug.

I remain convinced blackbox testing would of very probably never found that vuln, and even if it did, not in as short of a time period. I trust Terraform over closed alternatives because it was patched within a couple hours of me mentioning it on IRC by a peer who submitted the bug report and patch in one shot. I could verify the source code fix easily and compile the new binary myself to take advantage of the patch before Hashicorp even merged it.

I can also easily verify there are no regressions in future releases.

Tell me how you go about solving for this or other subtle cases like stego exfiltration more easily -without- source code. Also how you or your team could of patched the issue yourself without source code.

If I really solved this the hard way then I will by all means move my security engineering career to focus more on blackbox testing as you seem to be advocating for.

Re:Messenger systems compared by security, privacy, compatib... by tptacek on Friday October 19, 2018 @11:26AM (#18257351)
It sounds like you're arguing that you'd require source code to see whether something was using math/rand vs. crypto/rand, something that is literally evident in the control flow graph of a program. I do not doubt that source code makes it easier to review code when you're half-asleep at 4 in the morning.

For your particular example: go download a copy of radare and pull up any go build artifact and see for yourself how hard it is understand what's going on.

I don't know about your security engineering career, but if you intend to get serious about vulnerability research, yes, you should learn more about how researchers test shrink-wrap software. I spent years at a single stodgy midwest financial client doing IDA assessments to find vulnerabilities in everything from Windows management tools to storage appliances. It wasn't my IDA license; I was augmenting their in-house security team, which had 4 other licenses. This was in 2005.

Re:Messenger systems compared by security, privacy, compatib... by lrvick on Friday October 19, 2018 @12:51PM (#18258025)
The primary vulnerability here was that the author used the current time as the sole random seed for generating passwords. From there the output of a PRNG -looks- random, but is in fact based on something that happens every day and thus not really random.

If you can understand a PRNG algorithim and how it was seeded without source code using nothing but radare faster than I could read the code... then you really do have some superhuman skill, and most of my arguments fall flat.

Subtle cryprography flaws like this could be introduced intentionally as well by a bad actor, or pressure from a state actor. They are very hard to see without source code in my experience.

You just kind of made my point for me, in that seeing the output in something like radare -is- often much harder to understand what is going on than just looking at the source code.

Don't get me wrong, I have a deep respect for people that are very good at finding bugs this way. When you -don't have source code then finding bugs via methods like this is the only thing you have on the table, and it is impressive.

What I am taking issue with is you trying to in effect claim that some people like yourself are so good at blackbox testing that you could find all potential bugs faster with those tactics than you could reading the relevant source code.

Consider though that not all researchers work this way. Many bugs have been found by myself and other researchers I know by simply reading source code, so your argument that a vendor releasing source code gives it no security advantage is just not true.

While I am no fan of Signal, the fact they make their source code public makes it much easier to audit and trust its e2e cryptography implementation than say Whatsapp. Even the two tools you favor are wildly unequal in transparency and auditability.

Perhaps the majority of my background working with FOSS software has made me undervalue blackbox testing and you have made a good argument for it. It would make me more well rounded and I intend to pursue it.

I think if there is anything you can take from my side of this discussion it is seeing the value of providing source code to the right eyeballs that know how to quickly spot certian classes of issues.

That source code in the hands of the right person is a faster way to find some bugs than one could in a binary reverse engineering environment.

Re:Messenger systems compared by security, privacy, compatib... by tptacek on Friday October 19, 2018 @01:33PM (#18258381)
Oh for God's sake, dude. Write a Go program with a function that seeds math/rand from time.Time, compile it, and then load it into radare. "aa", then find the symbol for your function, then switch to disasm view and look at the function.

(The Terraform function you found this problem in is literally called "generatePassword", in case you planned on writing 6 paragraphs on how hard it is to find the right symbol to start analysis at.)

This is such a silly, marginal bug, it's bizarre to see you kibbitzing on the "right" way to fix it (the bug is that they're using math/rand to generate a password instead of crypto/rand, not that they're seeding math/rand from time). But whatever! Either way: it's clear as day from literally just the call graph of the program what is happening.

Your example of a bug that is hard to spot from disassembly is a terrible, silly example, that I've trivially refuted in just a couple of Unix shell commands.

I don't think you understand the arguing you're trying to have. I get it: you have a hard time looking for bugs in software. That's fine: auditing messengers is supposed to be hard. You don't have to be up to the task; others are.

Re:Messenger systems compared by security, privacy, compatib... by pvg on Thursday October 18, 2018 @03:57PM (#18251605)
IRC is lacking in features but setup correctly with modern OTR I stand by it having easy to reason about security advantages Whatsapp and iMessage do not. (Usability is another story)

The fact that you can replace OTR with OTP in this sort of statement and it becomes even truer should tell you what a lousy argument for the practical security of anything it is.

Re:Messenger systems compared by security, privacy, compatib... by smbullet on Thursday October 18, 2018 @12:27AM (#18245901)
The apps listed in the spreadsheet are clearly sorted by number of features supported (with weights). I don't think OP is necessarily claiming IRC is more secure than Signal.
Re:Messenger systems compared by security, privacy, compatib... by pvg on Thursday October 18, 2018 @12:53AM (#18246012)
They are sorted largely by security, according to the author.

"It is currently roughly sorted by security. Usability is subjective and people can work down the list to the most secure tool they feel they can comfortably use."

https://news.ycombinator.com/item?id=18233721

Re:Messenger systems compared by security, privacy, compatib... by smbullet on Thursday October 18, 2018 @08:53AM (#18247871)
Oops, I guess I missed that comment. Thanks!
Re:Messenger systems compared by security, privacy, compatib... by lrvick on Thursday October 18, 2018 @12:55PM (#18249959)
You are correct. I generalized this as "roughly sorted by security". Some in the list, like Tox, have notable design flaws, but this list is binary and does not account for implementation quality.

This is mostly for discovery of options to consider diving into.

Re:Messenger systems compared by security, privacy, compatib... by tptacek on Thursday October 18, 2018 @02:17PM (#18250713)
He's not correct (as he has since acknowledged!), and you did, and still do, suggest that placement on the chart implies greater or lesser security. You literally included instructions on how to read and use the chart that make that point. "Usability is subjective and people can work down the list to the most secure tool they feel they can comfortably use."
Re:Messenger systems compared by security, privacy, compatib... by lvh on Tuesday October 16, 2018 @02:30PM (#18231902)
The care Signal puts in that actually makes it a better, more secure messenger is primarily about metadata management. Consider how long it took for them to implement profiles, how much time they took to explain how their contact discovery works, et cetera. These are not trivial matters: they got subpoenaed and had nothing to respond with, because they're tried extraordinarily hard not to.
Re:Messenger systems compared by security, privacy, compatib... by lrvick on Tuesday October 16, 2018 @03:34PM (#18232457)
These are problems other tools have solved, without having to resort to a walled garden network or having a SPOF.

Sure, maybe Signal has done some useful technicality -legal- protections for now for US citizens, but what happens when a state actor threatens to kill the family of a Signal employee if they don't ship a very subtle compromise in how their binaries source random numbers, or if they don't sell the metadata of who is talking to who.

Signal is not anonymous so that metadata alone could have real value. It is at the end of the day using phone numbers as identifiers. Sure they do SGX remote attestation but that has been demonstrated broken multiple times and won't stand up to a motivated physical attacker. Even if it -is- solid now, I would not underestimate how far a state actor will go. (As demonstrated by NSA wiretaps on google datacenters). Can they compel the right Intel employee to CA certify a manipulated enclave? Can they just get handed the key?

Also why would people outside the US trust the legal protections afforded to a US company to protect US citizens?

Their refusal to federate their network just creates a Lavabit sized target... and I have yet to hear any technical reasons for doing so particularly when, again, other projects have demonstrated end to end encryption and decentralization are not ast mutually exclusive as Moxie claims.

The idea that only Signal can do this right, and only if they keep it centralized on their servers, with them being the only people that can sign the binaries... is pure hubris imo.

There are a lot of alternatives we all should be carefully considering for the next -standard- for ubiquitious secure messaging.

Re:Messenger systems compared by security, privacy, compatib... by lvh on Tuesday October 16, 2018 @03:45PM (#18232546)
You asked "what valuable thing has Signal done" and offered to track them, and I responded with two examples. "What if someone threatens a Signal employee" is a moved goalpost. Who else has solved private contact discovery? Conversely: who else has solved Mossad as a threat model? I have repeatedly pointed out the Signal subpoena elsewhere, which is responsive to a number of your comments.

If your threat model includes Mossad, you're going to get Mossaded upon. You say "thinking only Signal can do this is hubris", but in the same breath suggest we're just carefully consideration of a standard away from being safe against Mossad threatening someone's kids. That's hubris.

Re:Messenger systems compared by security, privacy, compatib... by lrvick on Tuesday October 16, 2018 @04:29PM (#18232954)
I will look closer into private contact discovery across messengers, sorry for rushing over that point.

I did try to respond to the sopoena comment in that that is only of limited value. A blackhat will proably have an easier time getting to servers than a lawyer with Signals setup, and I do credit them with providing a substantially better assurances than say Whatsapp... but still not good enough for my particular threat profile.

Personally I don't like using permanant non anonymous identifiers like phone numbers so I would not use the feature as they have implemented it, but that doesn't mean it does not have some value for some use cases worh exploring.

That is at least something that can be looked at objectivly in the scope of the spreadsheet.

Re:Messenger systems compared by security, privacy, compatib... by pvg on Tuesday October 16, 2018 @03:48PM (#18232572)
but what happens when a state actor threatens to kill the family [...]

No messenger system protects you against that. You seem to be going through the full sequence of well-known poor ways to evaluate the security of something like an instant messenger, starting with the feature matrix, going through 'it can't be secure if it's not open source/self-hosted/federated' and reaching the Mossad. Which is a worthwhile and educational exercise but it's still not a good way to evaluate the security of something like an instant messenger.

Re:Messenger systems compared by security, privacy, compatib... by codedokode on Tuesday October 16, 2018 @05:55PM (#18233636)
> No messenger system protects you against that.

If the system doesn't have central server then it would become more difficult. You cannot subpoena a Tox network.

Any messenger that requires and stores a phone number (read your real-world identity and physical location) is neither anonymous nor private.

Also, a centralized messenger with a single server means that all traffic between all the users around the world goes through one data-center in one country that can do what? Observe the traffic and detect who is using this messenger, at least their IP address and country.

Of course, for most users this doesn't change anything because they don't do anything illegal. But if they don't do anything illegal then they don't really need protection and can use VK or Telegram.

Re:Messenger systems compared by security, privacy, compatib... by lvh on Tuesday October 16, 2018 @06:34PM (#18233890)
> Also, a centralized messenger with a single server means that all traffic between all the users around the world goes through one data-center in one country that can do what? Observe the traffic and detect who is using this messenger, at least their IP address and country.

I mean, yes, a single server would have that property. But what system are we discussing that has a single server in a single data center?

Furthermore: if you actually care about metadata hiding it's a lot more complicated than "we have more than one person operating the servers".

> Of course, for most users this doesn't change anything because they don't do anything illegal. But if they don't do anything illegal then they don't really need protection and can use VK or Telegram.

Privacy is not just for people who do illegal things.

Re:Messenger systems compared by security, privacy, compatib... by tialaramex on Tuesday October 16, 2018 @07:44PM (#18234270)
So that whole "it's all flowing though a single DC" thing for Signal - that has a name in this sphere. It's called "Don't Stand Out".

When five people call the known mob boss you follow all of them. Maybe one is just a friend from high school. Another is the mob accountant, another an enforcer, you're getting leads. But what if it's five thousand people - now you can't follow them all, it's overwhelming.

Knowing five people in your country use Signal puts them all on the watchlist. If it's five million that's pointless. Without "Don't stand out" the encryption used just makes you a target.

Barry, who uses Barry's very own private self-hosted server for a popular federated system, Stands Out. Message from Japan to Barry's system? That's for Barry. How do we know? Well Barry's the only one on that server, easy. No cryptography can fix this.

Don't Stand Out

Re:Messenger systems compared by security, privacy, compatib... by lvh on Tuesday October 16, 2018 @08:54PM (#18234642)
You know what makes you not stand out? WhatsApp.
Re:Messenger systems compared by security, privacy, compatib... by tialaramex on Tuesday October 16, 2018 @09:02PM (#18234684)
Indeed, and since it uses the Signal protocol I'd suggest it's the best option for a wide variety of practical circumstances.
Re:Messenger systems compared by security, privacy, compatib... by pvg on Tuesday October 16, 2018 @07:17PM (#18234129)
If the system doesn't have central server then it would become more difficult. You cannot subpoena a Tox network.

The threat that was brought up was an actor with state-level resources, coercive capability and lack of scruples - the specific example being the threat of murdering someone's family, not subpoenas.

Re:Messenger systems compared by security, privacy, compatib... by lrvick on Tuesday October 16, 2018 @04:34PM (#18232999)
I actually think it is.

Journalists covering sensitive topics in sensitive areas -must- care about these questions.

If you are using something anonymous, fully end to end encrypted with open source reproducible verified builds on decentralized servers, you can greatly limit the risk of having a central third party that can be compelled to act against your interests.

Maybe in the US we don't think we need those sorts of protections, but we should consider the worst cases when designing security systems, and ensure no one single compromised person has the ability to backdoor thousands or millions of people.

Not everyone cares about this sort of thing though, and there are 70+ other options listed with various tradeoffs.

Re:Messenger systems compared by security, privacy, compatib... by tptacek on Tuesday October 16, 2018 @05:11PM (#18233307)
None of this makes any sense. This reads like an argument from a parallel universe where the only sane option for end-users is Linux on the Desktop.

In fact: a huge portion of commercial vulnerability research and assessment work is done on binaries of closed-source products, and it has never been easier to do that kind of work (we're coming up on a decade into the era of IR lifters). Meanwhile, the types of vulnerabilities we're discussing here --- cryptographic exfiltration --- are difficult to identify in source, and source inspection has a terrible track record of finding them.

There's no expertise evident in the strident arguments you've made all over this thread (network security work is not the same as shrink-wrap software security work, which is in turn not the same as cryptographic security work --- these are all well-defined specialties) and it concerns me that people might be taking these arguments seriously. No. This isn't how things actually work; it's just how advocates on message boards want them to.

Re:Messenger systems compared by security, privacy, compatib... by lrvick on Tuesday October 16, 2018 @06:07PM (#18233721)
You can of course do network analysis and say "this looks TLS encrypted with X algo" then dive into those packets and then say "this looks end to end encrypted with X algo".

You can also look into a binary application and study jumps, where the private key gets loaded into memory etc etc.

I am with you on this, there is tons of low hanging fruit you can do on a binary application, and I do a lot of this sort of thing myself.

Making sure an application at a high level seems to do what it says on the tin is a great phase one audit of something as a baseline, and should be done even on open source projects.

Trouble with closed source tools is you either have to do this over and over again on every new binary released, -or- you can look at the diffs from one release to next and see if any of those critical codepaths have changed.

You also -much- more easily evaluate bad entropy in key generation if you have the source code. People think they are clever all the time using silly approaches along the lines of random.seed(time.now()+"somestring") and it is much less time consuming to spot those types of flaws if you have the code.

This is why I argue you need both whitebox and blackbox testing when it comes to mission critical security tools, and a clear public auditable log of changes made to a codebase so review can continually happen, instead of only at specific checkpoints.

Again, some people may not care about that level of rigor and just want to share memes with their friends instead of journalists communicating in the feild. I tried to be pretty comprehensive with the list, and welcome recommendations for new objective criteria to include.

It is currently roughly sorted by security. Usability is subjective and people can work down the list to the most secure tool they feel they can comfortably use.

Re:Messenger systems compared by security, privacy, compatib... by tptacek on Tuesday October 16, 2018 @06:17PM (#18233776)
No, I don't think you're following me. You don't tcpdump and look at the packets (that would be silly; even horribly encrypted messages will look fine in a pcap). You don't even "study jumps, where private keys get loaded into memory". You take the binary and lift it to an IR and audit that.

People who do a lot of binary reversing vuln research were doing a version of this long before the we had modern tooling; you don't look at each load and store, but rather the control flow graph. People were auditing C programs, by manual inspection, in IDA Pro before there even was a market for secure messaging applications.

And that's just manual static analysis! Professional software assessors do a lot more than that; for instance, Trail has been talking about and even open-sourcing symbolic and concolic evaluation tools for years, and, obviously, people have been iterating and refining fuzzers and fault injectors for at least a decade before that.

This isn't "low hanging fruit" analysis where we find out, like, the messenger application is logging secret keys to a globally accessible log file, or storing task-switcher screenshots with message content. This is verification of program logic --- and, I'd argue even more importantly, of cryptographic protocol security.

"Bad entropy in key generation" is a funny example. Straightforward binary analysis will tell you what the "entropy source" is. If it's not the platform secure random number generator, you already have a vulnerability.

The problem with your chart is that there is no real rigor to it. It's just a list of bullets. It's obviously catnip to nerds. We love things like this: break a bunch of things into a grid, with columns for all sorts of make-me-look-smart check-marks! But, for instance, there's no way to tell in this chart whether a messenger "audit" was done by JP Aumasson for Kudelski, or some random Python programmer who tcpdumps the application and looks for local-XSS.

You can't even neatly break these audits down by "did they do crypto or not". The Electron messengers got bit a few months ago because they'd been audited by non-Electron specialists --- that's right, your app needed to be audited by someone who actually understood Electron internals, or it almost might as well not have been audited.

Given that: what's the point of this spreadsheet? It makes superficial details look important and super-important details look like they don't exist. You didn't even capture forward secrecy.

People who pay a lot of attention to cryptographic software security took a hard bite on this problem 5 years ago, with the EFF Secure Messaging Scorecard. EFF almost did a second version of it (I saw a draft). They gave up, because their experts convinced them it was a bad idea. I see LVH here trying to tell you the same thing, in detail, and I see you shrugging him off and at one point even accusing him of commenting in bad faith. Why should people trust your analysis?

Re:Messenger systems compared by security, privacy, compatib... by lrvick on Tuesday October 16, 2018 @06:40PM (#18233922)
Obviously I was just attempting to make a simple and easy to reason about example here, but yes of course there are a wide array of more practical approaches and tooling today depending on the type of binary and platform we are talking about.

I also fully agree you need to get people that specialize in the gotchas of a given framework to be able to spot the vuln. In fact that kind of bolsters my point, that you need the -right- eyeballs on something to spot the vuln, and this is much easier if something is open source vs only shown to a few select people without the right background knowledge to see a flaw.

The point I was trying to make is that you need both continual whitebox and checkpointed blackbox testing and if a tool is not open source then half of that story is gone unless you are Google sized and really can pay dedicated red teams to continually do both for every release... and even they miss major things sometimes in chromium/android that third parties have to point out!

Being open source client and server does -not- imply something has a sane implementation (and to your point things that rank well on paper like Tox have notable crypto design flaws).

This is more so you can quickly evaluate a few tool that contain the highest level criteria you care about for your threat profile, then you can drill down those options with less objective research into the team, funding, published research, personal code audting etc etc to make a more informed choice.

For me as an AOSP user, something without F-Droid reproducible build support is a non starter. Anything that is closed source I can't personally audit the critical codepaths of and have knowledge a lot of the security researchers I communicate with can do the same... is also a non starter. Lastly I want strong knowledge I can control the metadata end to end when it counts, and being able to self-host or being decentralized entirely is also an important consideration.

Many people have reviewd this list and described to me they have learned about a lot of options they didn't even know existed, or didn't know where closed or open source. That is the point. Discoverability.

It would be unreasonable and even irresponsible for someone to fully choose a tool based -only- on this list imo. If that was not clear elsewhere then hopefully it is clear now.

Re:Messenger systems compared by security, privacy, compatib... by tptacek on Tuesday October 16, 2018 @06:45PM (#18233949)
Have you used any of these tools? Have you done a binary reversing engagement? Have you found a vulnerability in closed-source software? Can you tell us more about the tools you used to do this work? What kinds of vulnerabilities did you find? You have very strong opinions about what's possible with open- vs. closed- source applications. I think it's reasonable to ask where the frontiers of your experience are.

It's clear to me from all your comments that you are somebody who wants to sysadmin their phone. That's fine; phone sysadmin is a legitimate lifestyle choice. But it has never been clear to me what any of the phone sysadmins are talking about when they demand that the rest of us, including billions of users who will never interact with a shell prompt, should care about their proclivities. The argument always seems to devolve to "well, if we can't sysadmin this application onto our phones, you can't be sure they're secure". But that's, respectfully, total horseshit, and only a few of the many reasons that it's horseshit appear among the many informed criticisms of your spreadsheet on this thread.

You might want to put your warning about how unreasonable and irresponsible this tool is on the actual spreadsheet, instead of just advocating that it be enshrined in Wikipedia.

Re:Messenger systems compared by security, privacy, compatib... by mwcampbell on Thursday October 18, 2018 @10:25AM (#18248549)
On the topic of phone sysadmin, I agree it's unreasonable to expect a large proportion of people to sysadmin their phones directly. But what if we advocated for a world where we tech-savvy folk volunteer to be sysadmins for our family and close friends? They would trust our motives more than they can trust the motives of platform and device vendors.
Re:Messenger systems compared by security, privacy, compatib... by pvg on Tuesday October 16, 2018 @05:22PM (#18233392)
My point is not that this isn't of interest or important (especially to people who routinely handle sensitive information) but that your methodology is poor and it's poor in ways that have been reasonably comprehensively examined. As I said, it's still a worthwhile exercise and there's no law of physics that says you have to agree with existing consensus-y thought but your effort would be more serious if you're familiar with the arguments. You, on the other hand, just reinvented a threat model that has its very own funny paper.
Re:Messenger systems compared by security, privacy, compatib... by throwawaymath on Tuesday October 16, 2018 @06:57PM (#18234002)
> Not everyone cares about this sort of thing though, and there are 70+ other options listed with various tradeoffs.

This is precisely pvg's point. The problem with your methodology is a systemic one that emerges in every crowdsourced threat modeling exercise. You've enumerated every possible security attribute and security feature of every software in a specific category, then tossed them all into a matrix of boolean values. But that does not result in a threat model users can competently assess, for several reasons:

1. You're treating all features as equal - if not in intention, then at least in presentation. Even if you don't intend it, the sea of green at the top is a loud proclamation of safety; likewise the sea of red at the bottom is a siren of insecurity.

2. You're not allowing any nuance in assessment feature or attributes. Boolean flags cannot capture all the nuance inherent in cryptographic security. Which specific party was responsible for an assessment? What are their credentials? What did they find?

3. You're including features which most users don't and shouldn't care about just because some minority might. Moreover you're not being opinionated enough, which is something that comes with expertise - for many of the "features" you listed, the minority that cares probably shouldn't if they only care because of a vague notion of security.

4. You're leaving out important features which should absolutely be considered for security. Where is forward secrecy? Where is authenticated encryption? Where is consideration of specific algorithms or primitives? Where is nonce misuse resistance?

5. Most importantly: you do not have any explicitly called out methodology that allows someone to audit what you've done. If you begin by pre-supposing that a given feature is worthy of inclusion because it's an important security metric, your conclusion is just going to end up magnifying that bias. Therefore it's paramount that you call out methodologies explicitly and early.

We see this time and time again in security. People try to first principles the security of an entire category of software by being exhaustive about every type of threat and security feature they can think of. But they invariably leave out important threats/features, underestimate the importance of some and overestimate the importance of others. Exercises which attempt to give users the world almost always end up "empowering" them to boil the ocean. People looking at this spreadsheet are approximately all unqualified to make an informed decision based on a critical assessment of all those features, which means they're likely to just go with the most green option (or worse, proselytize the most green option as the most secure one to their friends and coworkers).

Re:Messenger systems compared by security, privacy, compatib... by tripzilch on Friday October 19, 2018 @06:11AM (#18255543)
> Also why would people outside the US trust the legal protections afforded to a US company to protect US citizens?

Oh, hello there. The short answer is, of course we don't.

It's a very obvious attack surface that Signal and Whatsapp could avoid if they wanted to. For Whatsapp, being tied to Facebook, I kind of get why they don't (it's mainly that I don't expect them to be better).

But for Signal there aren't any good reasons. I've read various threads (on github and HN?) with Moxie being asked questions about this and I've not heard reasons that satisfied me. On occasion he was even evasive. Now, when the reasons given aren't good enough to explain why to take such a major security affecting decision when there is an obvious better alternative, and Signal seems to be very meticulous about doing the right thing in almost every other area of the protocol and systems around it, then there MUST be another motivation behind the decision that is not stated openly. Maybe it's just something benign left unsaid, who knows?

But even then, the only reasons I can imagine for choosing becoming this huge target, are reasons that are just good for Signal/Whispersystems but meaningless risk to its users.