ericdotlee
However, the internet seems littered with "clever" loca ai monstrosities that gang together 4-6 ancient nVidia GPU's (priced today to seem like overpriced e-waste) to get lackluster performance from piles of nVidia m60's and P100's? In 2025 this kind of seems like a waste or just bad advice to use hardware this old?
Curious if this find seems like a good source of info regarding staying away from Intel and AMD GPU's for local inference? Might do some training but right now more interested in light RAG and maybe some local coding.
Hoping to build something before the holiday season to keep my office warm with GPU's :).
Thanks!
Been looking for more details about software configs on https://llamabuilds.ai
Curious if anyone here has a similar config of 1-4 RTX 3060s? Trying to decide if picking up a few of these is a good value or if I should just continue renting cloud GPU's?
Is the Nvidia RTX 3060 the best value for entry level local AI?
Is there a big reason why Triton is considered a "failure"?
That said - I don't think I want to opt for one of the Ryzen "ai mini pc's" they seem solidly a worse option than a 3090?
So far I've just been renting gpu time but it's getting expensive. I also occasionally use qwen3-coder in my homelab but my headless "inference" machine is quite limited with an RTX 3080.
Curious if this build from llamabuilds.ai looks good? In terms of gpu etc? I'm maybe considering buying more GPUs in the future - hard to tell if stacking RTX 6000 ADA's is worthwhile!
Thanks HN!
Good starting point for professional AI workstation?
Initially, ChatGPT was kind of a cool toy that felt like something new but not really a threat. Now, even people who aren't engineers see the threat this has to destroy the dignity and financial stability of millions of people.
I like Ai - but at the same time I hope our country picks people over corporate profits... unfortunately I don't see that being likely.
Companies like BetterHelp have not helped people improve really just training them to have a constantly affirming voice at arms length at all hours.
That said, grinding through middling startup jobs also sucks.
The ability to get vc attention seems incredibly cliquey - not a game I really want to play again. On top of just wanting a normal / decent salary as a founder.
Love to see it.
Have you guys added the ability to cleanse / watch command strings for sensitive items like secrets, keys etc? Seems easier than ever with advances in lightweight LLMs.
It's a core tenant of my business and a handful of side projects. Wish you and your team the best!
Seems like hand waving and layoffs will have to stop before we get real data
Very interesting to already see rust based inference frameworks as well.