GPUs for Deep Learning in 2023 – An In-depth Analysis

Re:GPUs for Deep Learning in 2023 – An In-depth Analysis by BatteryMountain on Thursday January 19, 2023 @03:18AM (#34437942)

With this card you can also run Open AI's Whisper with the Large model (the multilingual one!), as it requires 10GB.

[ Parent | on HN ]

Re:GPUs for Deep Learning in 2023 – An In-depth Analysis by Const-me on Thursday January 19, 2023 @04:47AM (#34438472)

My implementation of Whisper uses slightly over 4GB VRAM running their large multilingual model: https://github.com/Const-me/Whisper

[ Parent | on HN ]

Re:GPUs for Deep Learning in 2023 – An In-depth Analysis by crabbycarrot on Thursday January 19, 2023 @01:56PM (#34444057)

Highly recommend quantizing the model (https://pytorch.org/tutorials/recipes/recipes/dynamic_quanti...). I converted the large model to use int8, and I'm able to run it 5x real-time on CPU with pretty low RAM requirements with still very good quality.

[ Parent | on HN ]