News for hackers. Stuff that matters.

A comment on Mistral ships le chat – enterprise AI assistant that can run on prem. Parent comment

Re:Mistral ships le chat – enterprise AI assistant that can ... by martin_a on Thursday May 08, 2025 @05:10AM (#43924440)
Strange idea, but if I'd like to set up a solid LLM for use in my home network, how much processing power would I need for a multi-purpose model?

A Raspberry Pi? And old ThinkPad? A fully speced-out latest gen Macbook?

edit: One of those old Mac Pros?

[ Parent | on HN ]
Re:Mistral ships le chat – enterprise AI assistant that can ... by wsintra2022 on Thursday May 08, 2025 @09:04AM (#43925722)
That’s what I tried initially, an old black tin can Mac Pro, but it couldn’t do it. Next splashed on an m2 ultra 64gb mpro, runs ollama with qwen3 32b - reverse shell into the localhost with open web-ui and automatic111 and voila AI on my home network
[ Parent | on HN ]
Re:Mistral ships le chat – enterprise AI assistant that can ... by martin_a on Thursday May 08, 2025 @10:14AM (#43926287)
Hm, that seems like a lot of power use. I thought I could get away with somewhat less.
[ Parent | on HN ]