News for hackers. Stuff that matters.
Re:Cost Per 1M tokens Of Self Hosting Llama-3 by aborsy on Sunday June 16, 2024 @04:15AM (#40695516)
That’s interesting!

The metadata is generally public information, I don’t care about that.

The malicious updates and key abuse are more concerning. It’s true for all software, and probably better done with OS, like on iOS.

The VPN could steal the keys, but that’s a lawsuit!

Re:Cost Per 1M tokens Of Self Hosting Llama-3 by hammyhavoc on Sunday June 16, 2024 @01:01PM (#40698493)
Are the keys not already kept on their own infra?
Re:Cost Per 1M tokens Of Self Hosting Llama-3 by aborsy on Monday June 17, 2024 @11:16AM (#40706461)
No, private keys don’t leave user’s devices. This is the case in all such products.

But with a malicious update, they could ship them to their infra, targeting some users. The product then becomes malware!