Re:Show HN: I fine-tuned Flan-T5. Can it cook?
by spion on Friday December 09, 2022 @03:04PM
(#33925862)
For some reason, I find it really grating how most deep learning stuff are pretty much impossible to do without paying quite a bit to a 3rd party service by the hour. Its one of the rare things in computing that you can't get started with unless you have some significant resources - most other things you can theoretically do with any old laptop and internet connection (ok - you may need to pay once you get users, but not just to get started)
[ Parent
| on HN ]
Re:Show HN: I fine-tuned Flan-T5. Can it cook?
by mattnewton on Friday December 09, 2022 @03:22PM
(#33926034)
I’ve found Google’s Colab to be a pretty great deal for getting started and playing around.
Re:Show HN: I fine-tuned Flan-T5. Can it cook?
by izzygonzalez on Friday December 09, 2022 @03:11PM
(#33925930)
At certain scales that’s true but there’s plenty you can experiment with with just Colaboratory or Kaggle… at least to get some intuition before jumping to spending money.
Re:Show HN: I fine-tuned Flan-T5. Can it cook?
by vorpalhex on Friday December 09, 2022 @03:10PM
(#33925910)
You can get a lot of success by buying used gpus of the right sort (usually pro gpus, not gaming cards).
There is still cost in terms of power - gpus run hot and loud.
Re:Show HN: I fine-tuned Flan-T5. Can it cook?
by askiiart on Friday December 09, 2022 @04:14PM
(#33926528)
I bought a Tesla K80 for ~$200 off eBay, and now it's going for less than $150, and you can essentially get the same but just 1 GPU for $100. It has 12 GB of VRAM, and works great for "medium-duty" machine learning and AIs like Stable Diffusion.
It does run hot (150W per GPU), and I've had to make a [custom solution to cool it](https://github.com/askiiart/k80-linux-cooling), but it works great considering the price.
By the way, you can essentially never have an application use the 2 GPUs in parallel in any useful way; I'd suggest just getting a K20X instead.
Re:Show HN: I fine-tuned Flan-T5. Can it cook?
by riku_iki on Friday December 09, 2022 @03:17PM
(#33925985)
I think that model he trained can be trained on consumer RTX 3060, you just maybe would need to wait 4 times longer comparing to A100 he used.
Re:Show HN: I fine-tuned Flan-T5. Can it cook?
by cweill on Sunday December 11, 2022 @06:16PM
(#33948706)
Google colab is $49/mo to get an A100. Trust me when I say you can build multimillion-dollar ML companies with just that (and maybe $100 extra dollars per month of spot credits).
Re:Show HN: I fine-tuned Flan-T5. Can it cook?
by cactusplant7374 on Friday December 09, 2022 @04:01PM
(#33926417)
Runpod and Colab are the easiest to get started. Google cloud has a crazy sales funnel that will make you go crazy.
$20 should get you pretty far I would think.