Re:GPT-4 API General Availability
by leodriesch on Thursday July 06, 2023 @04:27PM
(#36622395)
I'm interested in how the transformer based speech recognition from iOS 17 will perform compared to Whisper. I guess it will work more "real-time" like the current dictation on iOS/macOS, but I'm unsure as I am not on the beta right now.
My guess is the reason that apple invested so heavily in this [0] is because they are going to train a big transformer in their datacenter and apply it as an RNN on your phone.
Superficially, I think this will work very well, but slightly worse than whisper (with the advantage ofc being that its better at real-time transcription).
[0]https://machinelearning.apple.com/research/attention-free-tr...
I am a beta user and I am seriously impressed if I’m being honest. It works fully offline, is multilingual and is fast, even on the non-latest iPhone (I am using an iPhone 12 Pro Max). It is way better than apples previous version and better than locally installed whisper. They’ve done incredible work. Same with the new, transformer-based keyboard on iOS which is way better. And if you type in English, it sometimes shows word suggestions in the text field itself (similar how copilot works in an IDE).
You can run whisper.cpp locally in real time: https://github.com/ggerganov/whisper.cpp/tree/master/example...
Re:GPT-4 API General Availability
by ProllyInfamous on Thursday July 06, 2023 @06:09PM
(#36623861)
My M2 Pro (mac mini) will run Whisper much faster than "real time."
Pretty crazy stuff — perfectly understandable translations.
Re:GPT-4 API General Availability
by ycombinatornews on Thursday July 06, 2023 @04:17PM
(#36622205)
Echoing this - saying the whole text at once in one shot is very challenging for long batches of text.
Using built-in text input showed quite good results since ChatGPT is still understanding the ask quite well
One speculative thought about the purpose of Whisper is that this will help unlock additional high-quality training data that's only available in audio/video format.
Re:GPT-4 API General Availability
by PeterStuer on Friday July 07, 2023 @02:36AM
(#36628006)
Main reason for the lack of excitement is probably that it is fairly easy to self host Wisper, so people interested in it would have been doing that all along.
From using voice in the ChatGPT iOS app, I surmise that Whisper is very good at working out what you've actually said.
But it's really annoying to have to say my whole bit before getting any feedback about what it's gonna think I said. Even if it's getting it right at an impressive rate.
Given this is how OpenAI themselves use it (say your whole thing before getting feedback), I don't know that the API is set up to be able to mitigate that at all, but it would be really nice to have something closer to the responsiveness of on-device dictation with the quality of Whisper.