News for hackers. Stuff that matters.
Re:How to train large models on many GPUs? by cinntaile on Monday September 27, 2021 @09:05AM (#28670163)
Do you need watercooling to keep them from running too hot?
Re:How to train large models on many GPUs? by kkielhofner on Monday September 27, 2021 @09:42AM (#28670530)
You can tweak the power limit settings for your application. In many cases you can drop the power consumption (and heat generated) while still maintaining > 90% performance but this will depend on your actual use case [0].

In my experience for many models you can reduce the power limit even further than what has been tested in this guide while barely impacting performance.

[0] https://timdettmers.com/2020/09/07/which-gpu-for-deep-learni...

Re:How to train large models on many GPUs? by maxwells-daemon on Monday September 27, 2021 @09:15AM (#28670265)
I use 2x3090 to train large language models, and mine don't thermal-throttle with air cooling even though they're right next to each other. Eth mining does generate too much heat though.
Re:How to train large models on many GPUs? by lvl100 on Monday September 27, 2021 @05:36PM (#28676324)
For ML? Nope. I think overheating issues are mostly for mining. I run models and 3D render quite a bit and never ran into problems.