News for hackers. Stuff that matters.
Re:My local model setup on an M4 Pro Mac Mini by JKCalhoun on Wednesday September 02, 2026 @08:37AM (#49535340)
I like the in-depth description. Everything from the naming convention of the models (and how much RAM they require) as well as all the components needed underscores just how complicated this all still is.

I suppose I am waiting for AI-in-a-Box to come along so I can (painlessly) join in.

(I'm sure wrangling with all these esoteric aspects of LLMs though is fun for some people.)

Re:My local model setup on an M4 Pro Mac Mini by maherbeg on Wednesday September 02, 2026 @11:26AM (#49537741)
I've seen https://www.lucebox.com/ as an interesting option
Re:My local model setup on an M4 Pro Mac Mini by manueltgomes on Thursday September 03, 2026 @04:22AM (#49547350)
It’s $7900! “Interesting” and “option” are doing a lot of heavy lifting there :).
Re:My local model setup on an M4 Pro Mac Mini by Bluestein on Thursday September 03, 2026 @07:34AM (#49548687)
... if they are doing heavy lifting, are they ... load-bearing? :)
Re:My local model setup on an M4 Pro Mac Mini by JKCalhoun on Thursday September 03, 2026 @03:08PM (#49555098)
I was hoping for a software "LLM in a Box".
Re:My local model setup on an M4 Pro Mac Mini by lwsio on Wednesday September 02, 2026 @09:20AM (#49535808)
Hehe I really am just working it out as I go along - I promise it is fairly painless. Hugging Face allow you to specify your machine and then browse models that fit. And then you can just vibe out 'oh this one is a bit slow let me try another' etc