Re:When Free Software Isn't Free Enough
by lrvick on Wednesday November 18, 2026 @04:46PM
(#50552607)
Local AI can already do all of those things. I build operating systems and compilers using only consumer GPUs in my garage.
[ Parent
| on HN ]
Re:When Free Software Isn't Free Enough
by iFreilicht on Wednesday November 18, 2026 @05:03PM
(#50552838)
Which models do you recommend running? What amount of RAM do your GPUs have?
Re:When Free Software Isn't Free Enough
by lrvick on Wednesday November 18, 2026 @06:26PM
(#50553799)
I could run larger models with fewer sessions, but with my cluster of 8x r9700s I get 12 sessions of qwen 3.8 flash next 135b at > 100tps each and that is the sweet spot for me.
With radiance and/or strata a single card is enough for one session though.