News for hackers. Stuff that matters.

A comment on Are GPUs Worth It for ML?. Parent comment

Re:Are GPUs Worth It for ML? by scosman on Monday August 29, 2022 @04:26PM (#32643099)
As mentioned in the comment, ML training workloads tend to be super chunky (at least in my experience). Some days we want to train 50 models, some weeks we are evaluating and don’t need any compute.

I’d rather be able to spin up 200 gpus in parallel when needed (yes, at a premium), but ramp to 0 when not. Data scientists waiting around are more expensive than GPUs. Replacing/maintaining servers is more work/money than you expect. And for us the training data was cloud native, so transfer/privacy/security is easier; nothing on prem, data scientists can design models without having access to raw data, etc.

[ Parent | on HN ]
Re:Are GPUs Worth It for ML? by machinekob on Monday August 29, 2022 @04:33PM (#32643186)
If you are cloud only company then for sure it is just easier but still it wont be cheaper just more convenient to use. If data science team is very big probably "best" solution without unlimited money is just to run local and go cloud [premium] if you dont have free resources for your teams (It was the case when i was working in pretty big EU bank but it wasn't "true" Deep learning yet [about 4-5 years ago]).
[ Parent | on HN ]