News for hackers. Stuff that matters.
Re:How I Program with LLMs by pama on Monday January 06, 2025 @09:39PM (#42618566)
Your company could locally host LLMs; you wont get chatGPT or Claude quality, but you can get something that would have been SOTA a year ago. You can vet the public inference codebases (they are only of moderate complexity), and you control your own firewalls.
Re:How I Program with LLMs by Kostchei on Monday January 06, 2025 @10:56PM (#42619028)
You can get standalone/isolated versions of chatGPT, if your org is large enough, in partnership with OpenAI. And others. They run on the same infra but in accounts you set up, cost the same, but you have visibility on the compute, and control of data exfil - ie is there is none.
Re:How I Program with LLMs by CubsFan1060 on Monday January 06, 2025 @10:50PM (#42618985)
You can run Claude on both AWS and Google Cloud. I’m fairly certain they don’t share data, but would need to verify to be sure.
Re:How I Program with LLMs by evilduck on Monday January 06, 2025 @11:44PM (#42619331)
You can also run Llama 405B and the latest (huge) DeepSeek on your own hardware and get LLMs that trade blows with Claude and ChatGPT, while being fully isolated and offline if needed.
Re:How I Program with LLMs by krembo on Tuesday January 07, 2025 @01:31AM (#42619897)
With Amazon Bedrock you can get an isolated serverless Claude or llama with a few clicks
Re:How I Program with LLMs by evilduck on Wednesday January 08, 2025 @12:07AM (#42631136)
True, but if your org is super paranoid about data exfiltration you're probably not sending it to AWS either.