Using remote attestation in the browser to attest the server rather than the client is refreshing.

Using passkeys to encrypt data does limit browser/hardware combinations, though. My Firefox+Bitwarden setup doesn't work with this, unfortunately. Firefox on Android also seems to be broken, but Chrome on Android works well at least.

https://developer.nvidia.com/blog/confidential-computing-on-...

This is similar to the weasely language Google is now using with the Magic Cue feature ever since Android 16 QPR 1. When it launched, it was local only -- now it's local and in the cloud "with attestation". I don't like this trend and I don't think I'll be using such products

https://news.ycombinator.com/item?id=44601023

Think PCs in 5y to 10y that can run SoTA multi-modal LLMs (cf Mac Pro) will cost as much as cars do, and I reckon folks will buy it.

Lots of ifs there, though. I do trust Moxie in terms of execution though. Doesn’t seem like the type of person to take half measures.

This is the key question.

What makes it so strange is such an execution environment would have clear applications outside of AI usage.

Sure, for e.g. E2E email, the expectation is that all the computation occurs on the client, and the server is a dumb store of opaque encrypted stuff.

In a traditional E2E chat app, on the other hand, you've still got a backend service acting as a dumb pipe, that shouldn't have the keys to decrypt traffic flowing through it; but you've also got multiple clients — not just your own that share your keybag, but the clients of other users you're communicating with. "E2E" in the context of a chat app, means "messages are encrypted within your client; messages can then only be decrypted within the destination client(s) [i.e. the client(s) of the user(s) in the message thread with you.]"

"E2E AI chat" would be E2E chat, with an LLM. The LLM is the other user in the chat thread with you; and this other user has its own distinct set of devices that it must interact through (because those devices are within the security boundary of its inference infrastructure.) So messages must decrypt on the LLM's side for it to read and reply to, just as they must decrypt on another human user's side for them to read and reply to. The LLM isn't the backend here; the chat servers acting as a "pipe" are the backend, while the LLM is on the same level of the network diagram as the user is.

Let's consider the trivial version of an "E2E AI chat" design, where you physically control and possess the inference infrastructure. The LLM infra is e.g. your home workstation with some beefy GPUs in it. In this version, you can just run Signal on the same workstation, and connect it to the locally-running inference model as an MCP server. Then all your other devices gain the ability to "E2E AI chat" with the agent that resides in your workstation.

The design question, being addressed by Moxie here, is what happens in the non-trivial case, when you aren't in physical possession of any inference infrastructure.

Which is obviously the applicable case to solve for most people, 100% of the time, since most people don't own and won't ever own fancy GPU workstations.

But, perhaps more interesting for us tech-heads that do consider buying such hardware, and would like to solve problems by designing architectures that make use of it... the same design question still pertains, at least somewhat, even when you do "own" the infra; just as long as you aren't in 100% continuous physical possession of it.

You would still want attestation (and whatever else is required here) even for an agent installed on your home workstation, so long as you're planning to ever communicate with it through your little chat gateway when you're not at home. (Which, I mean... why else would you bother with setting up an "E2E AI chat" in the first place, if not to be able to do that?)

Consider: your local flavor of state spooks could wait for you to leave your house; slip in and install a rootkit that directly reads from the inference backend's memory; and then disappear into the night before you get home. And, no matter how highly you presume your abilities to detect that your home has been intruded into / your computer has been modified / etc once you have physical access to those things again... you'd still want to be able to detect a compromise of your machine even before you get home, so that you'll know to avoid speaking to your agent (and thereby the nearby wiretap van) until then.

Similarly with AI. The AI is one of the ends of the conversation.

The difference here is that the web server receiving a request for Confer receives an encrypted blob that only gets decrypted when running in memory in the TEE where the data will be used, which IS an end in the system.

It's like, come on you know exactly what you're doing, it's unambiguous how people will interpret this, so just stop it. Cue everyone arguing over the minutiae while hardly anyone points out how troubling it is that these people/entities have no concerns with being so misleading/dishonest...

Edit: I'm a little weary to find there is convenient import but not export functionalities. I manually copied the conversation into a markdown file <https://gist.github.com/Gravifer/1051580562150ce7751146be0c9...>

Also fwiw I think tees and remote attestation are a pretty pragmatic solution here that meaningfully improves on the current state of the art for llm inference and I'm happy to see it.

The entire point of E2EE is that both "ends" need to be fully under your control.

From Wikipedia: "End-to-end encryption (E2EE) is a method of implementing a secure communication system where only the sender and intended recipient can read the messages."

Both ends do not need to be under your control for E2EE.

If the software can modify data legitimately, it can be tampered with.

> This application requires passkey with PRF extension support for secure encryption key storage. Your browser or device doesn't support these advanced features.

> Please use Chrome 116+, Firefox 139+, or Edge 141+ on a device with platform authentication (Face ID, Touch ID, Windows Hello, etc.).

(Running Chrome 143)

So... does this just not support desktops without overpriced webcams, or am I missing something?

I see references to vLLM in the GitHub but not which actual model (Llama, Mistral, etc.) or if they have a custom fine tune, or you give your own huggingface link?

Now, of course, it is in question as to whether my little graphics card can reasonably compare to a bigger cloud thing (and for me presently a very genuine question) but that really should be the gold standard here.

Like when someone sends me a message, I made something that categorises it for urgency. If I'd use cloud it means they get a copy of all those messages. But locally there's no issue and complexity wise it's pretty low for an LLM.

Things like research jobs I do do in cloud, but they don't really contain any personal content, they just research using sources they already have access to anyway. Same with programming, there's nothing really sensitive in there.

Inevitably, the TEE hardware vendor must be trusted. I don't think this is a bad assumption in today's world, but this is still a fairly new domain and longer term it becomes increasingly likely TEE compromises like design flaws, microcode bugs, key compromises, etc. are discovered (if they haven't already been!) Then we'd need to consider how Confer would handle these and what sort of "break glass" protocols are in place.

This also requires a non-trivial amount of client side coordination and guards against any supply chain attacks. Setting aside the details of how this is done, even with a transparency log, the client must trust something about “who is allowed to publish acceptable releases”. If the client trusts “anything in the log,” an attacker could publish their own signed artifacts, So the client must effectively trust a specific publisher identity/key, plus the log’s append-only/auditable property to prevent silent targeted swaps.

The net result is a need to trust Confer's identity and published releases, at least in the short term as 3rd party auditors could flag any issues in reproducible builds. As I see it, the game theory would suggest Confer remains honest, Moxie's reputation plays are fairly large role in this.

Perhaps manual, user-controlled updates is not part of the design

If the source code is available^1 then surely someone has modified it to remove the phone number requirement, not to mention other improvements

1. https://github.com/signalapp/Signal-Server

It seems like Signal may be another example of "read-only" open source, where there is no expectation anyone will actually try to _use_ the source code. Instead, there is an expectation that everyone will use binaries distributed by a third party and allow remote code installation and RCE of software on their computers _at the third party's discretion_. In other words, all users will cede control to a third party

NB. This comment is not referring to the "Signal protocol". It pertains to _control_ over the software that implements it

It actually is the least bad option available, and decentralization is always worth it even if development is slower and more complex as a consequence.

I think Matrix has a lot of smart people involved, and I can't believe I'm saying this, but dear god, are you hiring for a TPM, by chance?

Every, single, time I have to go "Home" -> "Expand" -> "Favorites" to do what used to take one click, I lose a bit more faith in Element/Matrix.

Though, y'all did finally fix the notification sound that made me want to kill myself every single time it violated my ear drums. Only took.... 4 years? for that?

>broken SGX metadata protections

Citation needed. Also, SGX is just there to try to verify what the server is doing, including that the server isn't collecting metadata. The real talking is done by the responses to warrants https://signal.org/bigbrother/ where they've been able to hand over only two timestamps of when the user created their account and when they were last seen. If that's not good enough for you, you're better off using Tor-p2p messengers that don't have servers collecting your metadata at all, such as Cwtch or Quiet.

>weak supply chain integrity

You can download the app as an .apk from their website if you don't trust Google Play Store.

>a mandate everyone supply their phone numbers

That's how you combat spam. It sucks but there are very few options outside the corner of Zooko's triangle that has your username look like "4sci35xrhp2d45gbm3qpta7ogfedonuw2mucmc36jxemucd7fmgzj3ad".

>and agree to Apple or Google terms of service to use it?

Yeah that's what happens when you create a phone app for the masses.

"You have to agree to Apple's terms to use it"? What's Signal meant to do, jailbreak your phone before installing itself on it?

> so-called creator of some encryption protocol

All evidence points to him being one of the protocol’s designers, along with Trevor Perrin.

I’ve met both of them. The first time I met Moxie and talked about axolotl (as it was called back then) was in 2014. Moxie and Trevor strike me as having more integrity and conviction than most. There is no doubt in my mind that they are real and genuine.

Interestingly enough, some of the work Trevor did related to Signal’s cryptography was later used by Jason Donenfeld in the design of WireGuard.

> It screams honeypot like nothing else.

As you can see there is plenty of evidence suggesting otherwise.

This criticism has absolutely zero substance and honestly just reads like paranoid rambling. The Signal protocol has been independently formally analyzed [1] and has no known security issues.

[1] https://eprint.iacr.org/2016/1013

The specific part is that Signal forces Google and Apple on its users, and forces the specific kind of push notifications, too.

(You really have a lot of free time to write unsubstantiated dismissals to all my posts.)

Are you against using an Android (or LineageOS) emulator to do so?

> (You really have a lot of free time to write unsubstantiated dismissals to all my posts.)

You pop up in a lot of threads I'm interested in and seem to say a lot of incorrect things, that's all.

1. It's annoying and inconvenient.

2. It's the result of an artificial restriction by Moxie, for which I can't see any good reason, making me suspicious. In my opinion, this is basically an attack on true mobile freedom.

3. I do not believe in a good app isolation of Waydroid, so I would prefer to use as rare as possible. I also do not trust Android too much. And I will have to run two Signal apps simultaneously.

[0] https://github.com/boxdot/gurk-rs

Even if you do all that, it is not an official option, let alone a recommended one. The recommendation is to accept the google or apple terms of service.

Moxie even went as far as to say he would actively do anything in his power to discourage or stop the use of third party clients.

Potentially targeted users are set up to fail.

I wish apple & google provided a way to verify that an app was actually compiled from some specific git SHA. Right now applications can claim they're opensource, and claim that you can read the source code yourself. But there's no way to check that the authors haven't added any extra nasties into the code before building and submitting the APK / ios application bundle.

It would be pretty easy to do. Just have a build process at apple / google which you can point to a git repo, and let them build the application. Or - even easier - just have a way to see the application's signature in the app store. Then opensource app developers could compile their APK / ios app using github actions. And 3rd parties could check the SHA matches the app binaries in the store.

Is that a typo or are you really implying half the human population use Signal?

Edit: I misread, you are counting almost every messaging app user.

>Citation needed.

https://sgx.fail

https://en.wikipedia.org/wiki/Software_Guard_Extensions#List...

> Yeah that's what happens when you create a phone app for the masses.

No, that's what happens when you actively forbid alternative clients and servers, prevent (secure) alternative methods of delivery for your app and force people to rely on the American megacorps known for helping governmental spying on users, https://news.ycombinator.com/item?id=38555810

edit @ -4 points: please go ahead and explain why does Signal need your phone number and reject third party clients.

Same goes for Whatsapp, but the marketing is different there.

Or is your problem that your peer might run the app on an insecure device? How would you exclude decade old Android devices with unpatched holes? I don't want to argue nirvana fallacy here but what is the solution you'd like to propose?

Fortunately, the parties that you have to trust when you use signal haven't been malicious in any way, but that doesn't mean that they can't.

Kind of because Whatsapp adopted Signal's E2EE... And not even that long ago!

Time flies I guess, I feel old now

Also while we would expect heavy promotion for a trapped app from some agency it's also a very reasonable situation for a protocol/app that actually was secure.

You can of course never be sure but the fact that it's heavily promoted/used by people on both the whistleblowers, large corporations and multiple different National Officials at the same time is probably the best trustworthyness signal we can ever get for something like this.

(if all of these can trust it somewhaat it has to be a ridiculously deep conspiracy to not have leaked at least to some national security agency and forbidden to use(

To be fair, that is largely because WhatsApp partnered with Open Whisper to bring the Signal protocol into Whatsapp. So effectively, you're saying "Signal-the-app is hardly more private than another app that shares Signal-the-protocol".

In practical terms, the only way for Signal to be significantly more private than WhatsApp is if WhatsApp were deliberately breaking privacy through some alternative channel (e.g. exfiltrating messages through a separate connection to Meta).

"Confer - Truly private AI. Your space to think."

"Your Data Remains Yours, Never trained on. Never sold. Never shared. Nobody can access it but you."

"Continue With Google"

Make of that what you will.

Usually in a context where a cypherpunk deploys E2EE it means only the intended parties have access to plaintexts. And when it's you having chat with a server it's like cloud backups, the data must be encrypted by the time it leaves your device, and decrypted only once it has reached your device again. For remote computing, that would require LLM handles ciphertexts only, basically, fully homomorphic encryption (FHE). If it's that, then sure, shut up and take my money, but AFAIK the science of FHE isn't nearly there yet.

So the only alternative I can see here is SGX where client verifies what the server is doing with the data. That probably works against surveillance capitalism, hostile takeover etc., but it is also US NOBUS backdoor. Intel is a PRISM partner after all, and who knows if national security requests allow compelling SGX keys. USG did go after Lavabit RSA keys after all.

So I'd really want to see this either explained, or conveyed in the product's threat model documentation, and see that threat model offered on the front page of the project. Security is about knowing the limits of the privacy design so that the user can make an informed decision.

It wouldn't be long until Google and Gemini can read this information and Google knows you are using Confer.

Wouldn't trust it regardless if Email is available.

The fact that confer allows Google login shows that Confer doesn't care about users privacy.

This is a perfect example of purism vs. pragmatism. Moxie is a pragmatist who builds things that the average person can actually use. If it means that millions of people who would otherwise have used ChatGPT will migrate because of the reduced friction and get better privacy as a result, that's a win even if at the margin they're still leaking one insignificant piece of metadata to Google.

ChatGPT already knows more about me than Google did before LLMs, but would I switch to inferior models to preserve privacy? Hard tradeoff.

Few in this world have done as much for privacy as the people who built Signal. Yes, it’s not perfect, but building security systems with good UX is hard. There are all sorts of tradeoffs and sacrifices one needs to make.

For those interested in the underlying technology, they’re basically combining reproducible builds, remote attestation, and transparency logs. They’re doing the same thing that Apple Private Cloud Compute is doing, and a few others. I call it system transparency, or runtime transparency. Here’s a lighting talk I did last year: https://youtu.be/Lo0gxBWwwQE

Signal's achievement is that it's very private while being extremely usable (it just works). Under that lens, I don't think it could be improved much.

Exactly. Plus it basically pioneered the multi-device E2EE. E.g., Telegram claimed defaulting to E2EE would kill multi-client support:

"Unlike WhatsApp, we can allow our users to access their Telegram message history from several devices at once thanks to our built-in instant cloud sync"

https://web.archive.org/web/20200226124508/https://tgraph.io...

Signal just did it, and in a fantastic way given that there's no cross device key verification hassle or anything. And Telegram never caught up.

>Data and conversations originating from users and the resulting responses from the LLMs are encrypted in a trusted execution environment (TEE) that prevents even server administrators from peeking at or tampering with them.

I think what they meant to say is that data is decrypted only in a trusted execution environment, and otherwise is stored/transmitted in an encrypted format.

The web app itself feels poorly made—almost vibe-coded in places: nonsensical gradients, UI elements rendering in flashes of white, and subtly off margins and padding.

The model itself is unknown, but speaks with the cadence reminiscent of GPT-4o.

I'm no expert, but calling this "end-to-end encrypted" is only accurate if one end is your client and the other is a very much interposable GPU (assuming vendor’s TEE actually works—something that, in light of tee.fail, feels rather optimistic).

Thank you! :)

> .. assuming vendor’s TEE actually works

For sure TEEs have a rich history of vulnerabilities and nuanced limitations in their threat models. As a concept however, it is really powerful, and implementers will likely get things more and more right.

As for GPUs, some of Nvidia’s hardware does support remote attestation.

https://docs.nvidia.com/attestation/index.html