YOLOv5 on CPUs: Sparsifying to Achieve GPU-Level Performance

Re:YOLOv5 on CPUs: Sparsifying to Achieve GPU-Level Performance by 37ef_ced3 on Friday September 10, 2021 @07:04PM (#28487849)

Many businesses/services can't saturate the hardware you describe. It's just too much compute power. With CPUs you can scale down to fit your actual needs: all the way down to a single AVX-512 core doing maybe 24 inferences per second (costing a few dollars PER MONTH).

Also, your cost/inference results will change if you use a fast CPU inference engine, instead of something slow like PyTorch (which you appear to be using).

[ Parent | on HN ]

Re:YOLOv5 on CPUs: Sparsifying to Achieve GPU-Level Performance by carbocation on Saturday September 11, 2021 @09:01AM (#28491641)

Thanks - this is something I wasn’t familiar with. Do you have any pointers for CPU inference engines that you’ve had good experience with or that I can look into further?

[ Parent | on HN ]

Re:YOLOv5 on CPUs: Sparsifying to Achieve GPU-Level Performance by markurtz on Saturday September 11, 2021 @09:27AM (#28491805)

Disclosure: I work for Neural Magic.

Hi carbocation, we'd love to see what you think of the performance using the DeepSparse engine for CPU inference: https://github.com/neuralmagic/deepsparse

Take a look through our getting started pages that walk through performance benchmarking, training, and deployment for our featured models: https://sparsezoo.neuralmagic.com/getting-started

[ Parent | on HN ]