News for hackers. Stuff that matters.
Re:Show HN: I fine-tuned Flan-T5. Can it cook? by vorpalhex on Friday December 09, 2022 @03:10PM (#33925910)
You can get a lot of success by buying used gpus of the right sort (usually pro gpus, not gaming cards).

There is still cost in terms of power - gpus run hot and loud.

Re:Show HN: I fine-tuned Flan-T5. Can it cook? by askiiart on Friday December 09, 2022 @04:14PM (#33926528)
I bought a Tesla K80 for ~$200 off eBay, and now it's going for less than $150, and you can essentially get the same but just 1 GPU for $100. It has 12 GB of VRAM, and works great for "medium-duty" machine learning and AIs like Stable Diffusion.

It does run hot (150W per GPU), and I've had to make a [custom solution to cool it](https://github.com/askiiart/k80-linux-cooling), but it works great considering the price.

By the way, you can essentially never have an application use the 2 GPUs in parallel in any useful way; I'd suggest just getting a K20X instead.