Re:Building a personal, private AI computer on a budget
by xienze on Tuesday February 11, 2025 @09:39AM
(#43013174)
I get where you’re coming from, but the problem with LLMs is that you very regularly need a lot of back-and-forth with them to tease out the information you’re looking for. A more apt analogy might be a coworker that you have to follow up with three or four times, at an hour per. Not so appealing anymore. Doubly so when you have to stand up $2k+ of hardware for the privilege. If I’m paying good money to host something locally, I want decent performance.
[ Parent
| on HN ]
Re:Building a personal, private AI computer on a budget
by unshavedyak on Tuesday February 11, 2025 @03:59PM
(#43018287)
Agreed. Furthermore, for some tasks like large context code assistant windows i want really fast responses. I've not found a UX i'm happy with yet but for anything i care about i'd want very fast token responses. Small blocks of code which instantly autocomplete, basically.
Re:Building a personal, private AI computer on a budget
by MonkeyClub on Tuesday February 11, 2025 @04:44PM
(#43018744)
> If I’m paying good money to host something locally
The thing is, however, that at 2k one is not paying good money, one is paying near the least amount possible. TFA specifically is about building a machine on a budget, and as such cuts corners to save costs, e.g. by buying older cards.
Just because 2k is not a negligible amount in itself, that doesn't also automatically make it adequate for the purpose. Look for example at the 15k, 25k, and 40k price range tinyboxes:
It's like buying a 2k-worth used car, and expecting it to perform as well as a 40k one.