News for hackers. Stuff that matters.

A comment on Building a personal, private AI computer on a budget. Parent comment

Re:Building a personal, private AI computer on a budget by nickthegreek on Tuesday February 11, 2025 @09:54AM (#43013409)
You shouldn't be having slow response issues with LLM/TTS/STT for HA on a mbp m3 max 128gb. I'd either limit the entities exposed or choose a smaller model.
[ Parent | on HN ]
Re:Building a personal, private AI computer on a budget by joshstrange on Tuesday February 11, 2025 @01:34PM (#43016408)
Oh, I can get smaller models to run reasonably fast but I'm very interested in tool calling and I'm having a hard time finding a model that runs fast and is good at calling tools locally (I'm sure that's due to my own ignorance).
[ Parent | on HN ]
Re:Building a personal, private AI computer on a budget by nickthegreek on Tuesday February 11, 2025 @02:39PM (#43017335)
I decided on openai api for now after setting up so many differnt methods. the local stuff isn't up to snuff yet for what I am trying to accomplish but decent for basic control.
[ Parent | on HN ]
Re:Building a personal, private AI computer on a budget by joshstrange on Tuesday February 11, 2025 @02:59PM (#43017602)
I use a combo of Anthropic and OpenAI for now through my bots and my chat UIs and that lets me iterate faster. My hope is once I've done all my testing I could consider moving to local models if it made sense.
[ Parent | on HN ]