News for hackers. Stuff that matters.
Re:Universal Speech Model by rolisz on Thursday March 30, 2023 @03:14AM (#35369174)
On a 1080Ti (6 year old GPU) I found Whisper large models to take around as much time to transcribe as the length of the audio.
Re:Universal Speech Model by inciampati on Thursday March 30, 2023 @04:43AM (#35369786)
That's very similar to CPU-based performance with modern CPUs and parallelization! Frankly, with whisper.cpp it tends to be a little faster than the length of the audio for the "small" model, and much faster for "base" and "tiny".
Re:Universal Speech Model by pantalaimon on Thursday March 30, 2023 @03:18PM (#35377623)
Doesn't even have to be that modern, my Ivy Bridge CPU already achieves faster than realtime performance - which makes me wonder if there is maybe some upstart cost for the GPU based solution and it would outperform the CPU only with longer clips.
Re:Universal Speech Model by selfhoster11 on Thursday March 30, 2023 @11:12AM (#35373898)
Try quantised models. They perform reasonably well, although you probably want to run some benchmarks if you really want to get it done properly.