News for hackers. Stuff that matters.

A comment on GPUs for Deep Learning in 2023 – An In-depth Analysis. Parent comment

Re:GPUs for Deep Learning in 2023 – An In-depth Analysis by habibur on Thursday January 19, 2023 @01:19AM (#34437296)
> If you can do 8bit inference, don't use a GPU. CPU will be much cheaper and potentially also lower latency.

Good advice. Does that mean that I can install like 64 gb ram on a PC and run those models in comparable time?

[ Parent | on HN ]
Re:GPUs for Deep Learning in 2023 – An In-depth Analysis by fxtentacle on Thursday January 19, 2023 @09:21AM (#34440526)
That's how cloud speech recognition is usually deployed. OpenAI whisper is faster than realtime on regular desktop CPUs, which I guess is good enough.

And for a datacenter, a few $100 AMD CPUs will beat a single $20k NVIDIA A100 at throughput per dollar.

[ Parent | on HN ]