> Complete compromise could have been avoided if developers were prohibited from using ForwardAgent yes or not using -A in their SSH commands. The flaws with agent forwarding are well documented.
I use agent forwarding daily and had no idea it contained well known security holes. If that's the case, why is the feature available by default?
Ed: looks like I need to edit my sshcontrol-file
https://www.gnupg.org/documentation/manuals/gnupg/Agent-Conf...
For people in the same boat, it can be done trivially using the YubiKey Manager CLI: https://developers.yubico.com/yubikey-manager/
Look for -c here: https://man.openbsd.org/ssh-add
Use separate keyboard-interactive 2FA (I recommend google-authenticator) for production ssh access.
Use a key system which requires confirmation or a PIN to authenticate (such as a Yubikey). Use a persisting ssh connection with Ansible (ControlPersist) to avoid unnecessary multiple authentications.
Allow connections only from whitelisted IPs, or Uuse port knocking to open temporary holes in your firewall, or require connections to production infrastructure to go through a VPN.
Access production infrastructure from hardware dedicated for that purpose, never do anything else on it.
I wish there was a way in ssh to tag connections and only allow agent forwarding to keys with the same tag. That would prevent agent forwarding production keys from a dev host.
another option would be for a SSH client to present a full-screen "$HOST is trying to use your your SSH PRIVATE keys. Press enter, then type "~A" to allow." prompt.
Here is a much better explanation (from [0]):
> ProxyJump was added in OpenSSH 7.3 but is nothing more than a shorthand for using ProxyCommand, as in: "ProxyCommand ssh proxy-host -W %h:%p"
so the same thing that top poster was talking about.
[0] https://superuser.com/questions/1253960/replace-proxyjump-in...
Agent forwarding forwards the agent socket to the proxy server. Thus any ssh connection originating from the proxy server can reuse the agent, and with that has the same access to the agent as the originating host.
ProxyJump routes the ssh connection through the proxy host. The crypto takes place between originating host and target host, not between proxy host and target host. ssh connections originating from the proxy host can not access keys from the originating host.
But maybe my understanding of ProxyJump is incorrect?
ProxyJump proxies your ssh connection, so connecting from A to B via proxy X the connections go A->X and X->B.
You can use AgentForwarding with ProxyJump, in which case agent connections go B->X->A.
I cannot see how ProxyJump would somehow be an alternative to AgentForwarding. You can use both independently.
No, it rather works like this:
A -> B via X establishes A->X and then, through that connection tunnels a new ssh-connection from A->B.
A->X, then X->B would require forwarding the Agent from A to X, so that the connection from X->B can authenticate using that agent. Proxying the connection does not require X to ever authenticate to B, the authentication happens straight from A->B (1). Thus, no agent (forwarding) needed. You can also chain ProxyJumps: A->X->Y->B tunnels A->B in A->Y which is then tunneled through A->X. In that regard, ProxyJump and ProxyCommand can replace AgentForwarding in most use cases. There are some uses where AgentForwarding is the only solution, though.
(1) Added benefit: X never sees the actual traffic in unencrypted form and all port forwards A<->B work
I was thinking that the threat is that a compromised B gives access to your keys via agent forwarding. Presumably if you make keys available on B, you need them there. There's nothing ProxyCommand does to help there.
But you're talking about using ProxyCommand as an alternative for connecting A->X and then X->B, so keys are not available on X. That's of course an improvement.
So if you SSH -A to a compromised Jenkins server, and you've got all your production keys loaded in your agent, the hacker can now authenticate to all those production machines as well.
So don't ever SSH -A into a machine unless you KNOW its secure. The way I think about it is unless I trust the machine enough to leave my private keys on that machine, then I'm not going to SSH -A into it.
That's why. It's useful, but you have to be mindful of the security risks involved in using it.
Which of course doesn't mean that the hacker should have just send an email to the matrix team.
We should have more bounties. Let users donate and put wallets on servers. Attacker will be able to take these funds. It's a reasonable measure of an infrastructure security.
Therefore, the wallets should be stored GPG encrypted in some published location. After the hacker has successfully penetrated and retrieved the file, they need to publish a "how I did it" document along with the hash of the GPG encrypted wallet.
Once devs have confirmed the vulnerabilities exist, they respond with the passphrase to decrypt the wallet.
You still need some trust that private keys to given wallet are on the server, but apart from that, when you know there's $10,000 dollars on the server for anybody who can access it, it says something about how secure this machine is.
Plus you get instant notification when the server is compromised. Not every hacker is kind enough to let you know.
Just please don't take the other valuables and ... oh yeah, please don't mess with any of my family members and maybe please let's try to keep it at no more than one hundred people trying at the same time b/c otherwise things might get out of hand.
Some of them go beyond just instruction booklets but promise access to their chat systems via invitation (upon purchase of the pdf) and offer some kind of limited coaching. It is essentially the recruiting mechanism to bring in lower ranking soldiers starting out as mules, handlers, or basically move up from re-selling goods.
A couple of these guides point out how much Telegram sucks etc, and that they now have moved to p2p based systems. One praised Matrix heavily for it's good security feature.
The tech-savvy-ness of many vendors has picked up considerably since I first started watching. There is a strong push to re-think and refactor both tools and their processes (yes yes - this happens constantly otherwise they get caught, but never as fast or aggressive than these past months).
It's likely that this is just a (s)kiddy enjoying the attention. Though quite a lot of players have more than just an "academic" desire to ensure these (their) systems can withstand an attack by LE. When I browsed the matrix issues on github I couldn't help but immediately recall the strange emphasis on "we have switched to matrix". It's far fetched but I'd say somebody may have a strong interest in seeing these issues resolved (->or has gotten genuinely fed up and wanted to do something, as opposed to this being just a skid that only did it for the attention)
for a good analysis on how some of these tutorials and the philosophy behind them see: "Discovering credit card fraud methods in online tutorials"
https://www.researchgate.net/publication/303418684_Discoveri...
hey anybody else remember the days of T-Philez?
While I didn't loose access to the encrypted messages, since I used the 'Encrypted Messages Recovery' function of Riot.im, I guess a lot of people have. Maybe allow to store more information on the client side?
I do however have a keys backup dating back some time, that will hopefully restore some of my encrypted messages. But basically, I understand that every encrypted message was at risk of being lost, so it's not that big of a deal.
People have different threat models. When chatting with my family, it's more important that we have a permanent history of our messages rather than the worry of them getting leaked. But if you're a whistleblower you have a different set of requirements.
[1]: https://github.com/uhoreg/matrix-doc/blob/e2e_backup/proposa...
Infrastructure with ssh access without hole punching for currently active authorized connections only? Decrypted signing keys accessible over the network? CI servers and developers having root access?
Though the "we had to revoke all the keys so you lost access to your encrypted messages unless you backed them up" takes the cake.
This is just how it works. It's been well documented and mobile clients got updates that backs up the keys automatically. It's also effectively the same as WhatsApp and some other IMs (they just don't even save your encrypted messages). Either way - backup, or lose your history.
Enforcing in the clients to properly back up by default, or otherwise properly educating the user of what happens if they don't back-up would be as important as getting the code right. There is little difference to the user whether they lost data because they didn't understand they really had to do backups, or they got their keys compromised and messages deleted by a malicious 3rd party.
I do agree with all of GP's other points though.
If this is a design constraint, then the security model needs to accommodate that the user keys are the pot of gold, which means that there needs to be a service provided by a dedicated server which is inaccessible in the course of normal operation via any means other than a well defined braindead simple protocol <keyid>:<command>:<message> providing the message manipulations/key store functions from only other authorized production hosts that need to be able to access this functionality.
The server running the service should have a security policy that would prevent one from running any software that is not supposed to be already present on a server ( use SELinux enforcement policy ) to minimize the attack surface; have its own SSH keys not generally accessible during the normal operation, be accessible only from specific IP addresses, etc etc etc. If it is on AWS, it should probably be in a separate account.
Deleting the keys isn't something the matrix.org folks explicitly had to do because of the compromise; it's simply how the riot.im client reacts when you terminate it's session.
Imagine if this was facebook. Or whatsapp. Or signal and this was the result. They would be crucified ( justifiably ). But for some reason we are giving Matrix a pass.
But, your comparison with other messaging apps aren't really a fair comparison (other than "they are messaging apps"). The reason why they don't have these issues is because they don't provide features that Matrix does -- and those features make it harder for Matrix to implement something as simply others they might. For example, Signal stores all your messages locally and doesn't provide a way for new devices to get your history -- Matrix doesn't store messages locally long-term and all your devices have access to your history. In addition, there is no "log out" with Signal unless you unlink your device.
The reason why Matrix doesn't have e2e by default yet is because they want to ensure issues like this don't happen to every user.
If users' keys are linked to the session key then the system has to be designed in a way that the centralized session key store is protected like a pot of gold. That's a design constraint and dictates operational constraints.
> Matrix doesn't store messages locally long-term and all your devices have access to your history. In addition, there is no "log out" with Signal unless you unlink your device.
If one designs this kind of a system, one accepts the security constraints this system has. That's a basic competence or in this case a lack of it.
I would also like to point out that e2e is still not enabled by default because of issues like this. If you enable it you should know to enable key backups.
Riot has supported automatic key backups for the past few months, and if you'd used that you wouldn't have had a problem (yes it should've existed earlier but there are a lot of things for the underfunded Matrix team to deal with). And the reason it's not default is because making such a system opt-out would also make people start screaming about how Matrix is insecure because "it stores your keys on the server".
I think in many respects, the people working on Matrix are going to get criticised like this no matter what they do. I note you haven't actually suggested a specific proposal for how to fix this -- you're just going on about design cinstraints and how Matrix is therefore a joke system. To me that seems to be more snark than useful advice.
Encrypt the bloody backup keys with a key derived from a passphrase selected by a user.
> I think in many respects, the people working on Matrix are going to get criticised like this no matter what they do. I note you haven't actually suggested a specific proposal for how to fix this -- you're just going on about design cinstraints and how Matrix is therefore a joke system. To me that seems to be more snark than useful advice.
The snark would be to say "Use Matrix. Who cares about the system not being built to deal with the design constraints"
No one should defend Matrix after this. It was not a mess up. It was an Equifax level fuckup that was totally preventable.
Actually the system they have is better than that. You generate a random Curve25519 private key and the public part is stored. This allows your client to upload backups of session keys without needing to constantly ask the user for their recovery password.
You can then set a password which will be used to encrypt the private key and upload it to the homeserver (but you can just save the private key yourself).
So, not only do they have a system like you proposed, it's better than your proposal.
> It was an Equifax level fuckup that was totally preventable.
I agree with you that their opsec was awful on several levels, but you're not arguing about that -- you're arguing that their protocol doesnt fit their design constraints (by which you mean that they clear keys on forced logout without prompting to enable backups if you don't have them enabled yet -- as I mentioned there is an open bug about that but that's basically a UI bug).
All of that said, it's ridiculous that they don't have all their internal services on an internal network which you need a VPN to access.
Not the implementation of the encryption code.
Which we'll continue to use from people like www.modular.im, and whoever else springs up. As well as self-hosted servers.
Don't trust them? Host it yourself, and it's easier every day.
That's what drew me to the platform, and what will keep those serious about security, and decentralization/federation.
Some might argue that this is harmful to matrix as a product and as a brand. But as long as there was no actual harm done and they react appropriately by taking infrastructure security seriously, it could play out well in the end for them. This whole ordeal could end up actually increase trust in the project, if they take swift steps to ensure that something like this does not happen again.