News for hackers. Stuff that matters.
Re:Mind the Trust Gap: Fast, Private Local-to-Cloud LLM Chat by wolecki on Tuesday May 13, 2025 @11:31AM (#43974049)
A fast Trusted Execution Environment protocol utilizing the H100 confidential mode. Prompts are decrypted and processed in the GPU enclave. Key innovation is that it can be really fast especially on ≥10B parameter models, with latency overhead less than 1%. Like in CPU confidential cloud computing, that opens up a channel for communication with cloud GenAI models that even the provider cannot intercept. Wonder whether something like that could boost the trust for all the AI neoclouds out there.
Re:Mind the Trust Gap: Fast, Private Local-to-Cloud LLM Chat by danbiderman on Tuesday May 13, 2025 @11:43AM (#43974188)
author here!