News for hackers. Stuff that matters.
Re:Ask HN: Affordable hardware for running local large langu... by ysleepy on Sunday May 05, 2024 @02:17PM (#40266958)
I simply bought 4x32GB ddr4 memory (~200 bucks) for a normal desktop mainboard and a high-thread-count cpu.

You can experiment with a lot of models, it's just going to be slow.

With ddr5 you can even go higher with 48GB modules.

Otherwise I got a 3060 12G, which can be had for 200€ used.

Its a very affordable setup.

There are no comments on this comment yet.