Re:How I Program with LLMs
by pama on Monday January 06, 2025 @09:39PM
(#42618566)
Your company could locally host LLMs; you wont get chatGPT or Claude quality, but you can get something that would have been SOTA a year ago. You can vet the public inference codebases (they are only of moderate complexity), and you control your own firewalls.
[ Parent
| on HN ]
You can get standalone/isolated versions of chatGPT, if your org is large enough, in partnership with OpenAI. And others. They run on the same infra but in accounts you set up, cost the same, but you have visibility on the compute, and control of data exfil - ie is there is none.
Re:How I Program with LLMs
by CubsFan1060 on Monday January 06, 2025 @10:50PM
(#42618985)
You can run Claude on both AWS and Google Cloud. I’m fairly certain they don’t share data, but would need to verify to be sure.
You can also run Llama 405B and the latest (huge) DeepSeek on your own hardware and get LLMs that trade blows with Claude and ChatGPT, while being fully isolated and offline if needed.
With Amazon Bedrock you can get an isolated serverless Claude or llama with a few clicks