News for hackers. Stuff that matters.
Re:Universal Speech Model by abraxas on Thursday March 30, 2023 @12:20AM (#35367930)
Yeah, I'm not sure why people get so hyped up about Whisper. In production use it's middling at best and there are commercial offerings the handily beat it in both accuracy and speed.

Whisper is mostly an academic toy.

Re:Universal Speech Model by dontreact on Thursday March 30, 2023 @12:51AM (#35368152)
If there is a better mode I can easily integrate using python I’m all ears. For what I’m building quality is most important
Re:Universal Speech Model by abraxas on Thursday March 30, 2023 @12:58AM (#35368211)
Oh, Azure's speech recognition API beats it handily on English language. Both in accuracy and speed.

Another is Deepgram. Even this obscure vendor seems to be able to handle the samples I tried better than Whisper: https://picovoice.ai/platform/cat/

But yeah, go with Azure as your starting point. It is good and the price is likely acceptable unless you're transcribing all of youtube.

Re:Universal Speech Model by dontreact on Thursday March 30, 2023 @10:41AM (#35373400)
Umm I want to pay zero and run locally
Re:Universal Speech Model by generalizations on Thursday March 30, 2023 @04:43PM (#35378582)
If you use the large_v2 version of whisper, and give it a prompt to indicate what it's transcribing, it can do extremely well. But do use the prompt feature.
Re:Universal Speech Model by dontreact on Friday March 31, 2023 @03:45PM (#35392442)
Yeah exactly this is why there’s hype. It’s the best model that you can use for free easily
Re:Universal Speech Model by selfhoster11 on Thursday March 30, 2023 @01:05AM (#35368262)
Whisper democratises high-quality transcription of the languages I personally care about, whether using a CPU or a GPU. It's FOSS, self-hostable, and very easy to use.

That's a big deal for someone who wants to build something using speech recognition (voice assistant, transcription of audio or video) without resorting to APIs.

Re:Universal Speech Model by abraxas on Thursday March 30, 2023 @01:20AM (#35368387)
Is this in the realm of aspiration or something you've actually worked on? Because Whisper is incredibly difficult (I'd say impossible) to use in a real time conversational setting. The transcription speed is too slow for interactive use even on a GPU once you step up above tiny or base. And when you step down this low the accuracy is attrocious (especially in noisy settings or with accented voices) and then you have to post process the output with a good NLP to make it usable in whatever actions you're driving.

Look, it's nice that it's out there and free to use. For the sake of my wallet I hope it gets really good. But it isn't competitive with top of the line commercial offerings if you need to ship something today.

Re:Universal Speech Model by aftbit on Thursday March 30, 2023 @01:51AM (#35368595)
What if you are operating in an internet-denied application? A remote radio sensing system with a low bandwidth back channel or an airplane cabin, just to name two.
Re:Universal Speech Model by codethief on Thursday March 30, 2023 @03:08AM (#35369134)
At the company I work for we are currently in the process of transitioning from being a German-only to an international company. Recently, we started using Whisper to live-transcribe + translate all our (German) all-hands meetings to English. Yes, it required some fine-tuning (i.e. what size do you choose for the audio chunks you pass to Whisper) but overall it's been working incredibly well – almost flawlessly so. I don't recall which model size we use but it does run on a beefy GPU.
Re:Universal Speech Model by rolisz on Thursday March 30, 2023 @03:14AM (#35369174)
On a 1080Ti (6 year old GPU) I found Whisper large models to take around as much time to transcribe as the length of the audio.
Re:Universal Speech Model by inciampati on Thursday March 30, 2023 @04:43AM (#35369786)
That's very similar to CPU-based performance with modern CPUs and parallelization! Frankly, with whisper.cpp it tends to be a little faster than the length of the audio for the "small" model, and much faster for "base" and "tiny".
Re:Universal Speech Model by pantalaimon on Thursday March 30, 2023 @03:18PM (#35377623)
Doesn't even have to be that modern, my Ivy Bridge CPU already achieves faster than realtime performance - which makes me wonder if there is maybe some upstart cost for the GPU based solution and it would outperform the CPU only with longer clips.
Re:Universal Speech Model by selfhoster11 on Thursday March 30, 2023 @11:12AM (#35373898)
Try quantised models. They perform reasonably well, although you probably want to run some benchmarks if you really want to get it done properly.
Re:Universal Speech Model by selfhoster11 on Thursday March 30, 2023 @11:09AM (#35373831)
Actively working on it. I've not noticed any performance problems even with the large model (though the plan was always to run the speech recognition on a GPU - your use case may differ). It seems to be doing fairly well even with slightly noisy inputs, and certainly has better bang/$ than other non-API solutions that service my native language.

While true real-time would definitely be nice, I can approximate it well enough with various audio slicing techniques.

Re:Universal Speech Model by MacsHeadroom on Thursday March 30, 2023 @03:01PM (#35377437)
Faster Whisper is 8x faster than real time on CPU and even faster on GPU. https://github.com/guillaumekln/faster-whisper

Vocode uses Whisper for real-time zero latency voicechat with chatGPT. Give their demo line a call to see how well it works: +1-650-729-9536

Re:Universal Speech Model by inciampati on Thursday March 30, 2023 @02:49AM (#35368983)
In most real world settings, at least in my personal use, latency to a remote AI comprises most of the usability difficulty with automated speech recognition. The larger whisper models can be run directly on a laptop using multi threading and achieve speech to text transcription that is fully sufficient to almost completely write whole emails, papers, documents with them. In fact, I've written most of this comment using an ASR system on my phone that uses whisper. While the smaller models (like the one user here) can need some correction, the bigger ones are almost perfect. They are both very sufficient and for realtime interactive use I see no future market for paid APIs.

Yesterday I wrote virtually all the prose in the manuscript while walking around with a friend and discussing it. We didn't even look at the phone.

Obviously there's an academic element here because I'm saying I'm using it for writing. But it's more of a human-centric computing thing. I'm replacing a lot of time that my thumbs are spent tapping on keys, my fingers are spent tapping on keyboard, and my eyes are spent staring at the words that are appearing, looking for typographical errors to correct, with time organizing my thoughts in a coherent way that can be spoken and read easily. I'm basically using whisper to create a new way to write that's more fluid, direct, and flows exactly as my speech does. I've tried this for years with all of the various ASR models on all the phones I've had and never been satisfied in the same way.

Re:Universal Speech Model by DougBTX on Thursday March 30, 2023 @04:15AM (#35369581)
Sounds great! Which app are you using for this?
Re:Universal Speech Model by inciampati on Thursday March 30, 2023 @04:24AM (#35369640)
"Openai Whisper Keyboard" is good. It doesn't use whisper.cpp but rather a pytorch implementation that runs on Android.

I also use whisper.el on emacs. It's amazing. Much more powerful but computer based of course.

Re:Universal Speech Model by DougBTX on Thursday March 30, 2023 @04:30AM (#35369685)
Thanks!