News for hackers. Stuff that matters.
Re:Cost Per 1M tokens Of Self Hosting Llama-3 by aborsy on Friday June 14, 2024 @05:02PM (#40685103)
Why do you have to trust a third party?

It’s end to end encrypted, and with tail lock enabled, nodes can not be added without user’s permission.

Re:Cost Per 1M tokens Of Self Hosting Llama-3 by sneak on Saturday June 15, 2024 @02:09AM (#40687835)
The idea of “user’s permission” is determined by tailscale and/or the oidc provider. I don’t know anything about “tail lock”, perhaps it is a new mitigation for this issue?

I didn’t start with tailscale because the only way you could log into it was with Google or GitHub or something. I don’t trust Microsoft or Google with auth for my internal network. I thought about running Headscale but Nebula was faster/easier for me.

Re:Cost Per 1M tokens Of Self Hosting Llama-3 by aborsy on Saturday June 15, 2024 @04:50AM (#40688429)
Yes, Microsoft and Google will not be able to authenticate to your network if you enable tail lock. A node in your network has to sign.
Re:Cost Per 1M tokens Of Self Hosting Llama-3 by hammyhavoc on Saturday June 15, 2024 @10:54AM (#40690184)
Well, one example, depending on your threat model—their privacy policy states that they retain info and comply with subpoenas.

There's also potential for malicious updates to compromise a network (as there is with most software unless you're auditing the source for each update).

E2EE is only as meaningful as where the keys reside, and how easily those keys are abused.

Re:Cost Per 1M tokens Of Self Hosting Llama-3 by aborsy on Sunday June 16, 2024 @04:15AM (#40695516)
That’s interesting!

The metadata is generally public information, I don’t care about that.

The malicious updates and key abuse are more concerning. It’s true for all software, and probably better done with OS, like on iOS.

The VPN could steal the keys, but that’s a lawsuit!

Re:Cost Per 1M tokens Of Self Hosting Llama-3 by hammyhavoc on Sunday June 16, 2024 @01:01PM (#40698493)
Are the keys not already kept on their own infra?
Re:Cost Per 1M tokens Of Self Hosting Llama-3 by aborsy on Monday June 17, 2024 @11:16AM (#40706461)
No, private keys don’t leave user’s devices. This is the case in all such products.

But with a malicious update, they could ship them to their infra, targeting some users. The product then becomes malware!