News for hackers. Stuff that matters.
Re:Whisper – open source speech recognition by OpenAI by dom96 on Wednesday September 21, 2022 @03:22PM (#32930149)
This really makes me want to build a Amazon Echo/Google Nest/etc replacement that's open hardware, open source and most importantly recognises voice completely offline. I find that I don't use these smart devices for much more than setting timers anyway so this seems like an easy project.

I just wonder what system requirements Whisper has and whether there are open source voice recognition models that are specifically built for embedded devices.

Re:Whisper – open source speech recognition by OpenAI by MacsHeadroom on Wednesday September 21, 2022 @03:46PM (#32930530)
I really want all this too. The smallest model is ~80mb and the largest is 3gb. Not sure about system requirements yet; but models that small suggest this may be doable locally on a single board computer.

Edit: According to this comment[0] the base model runs in real time on an M1 CPU. The tiny model apparently decodes an audio file twice as fast. These are promising results.

[0] https://news.ycombinator.com/item?id=32927360#32929739

Re:Whisper – open source speech recognition by OpenAI by lunixbochs on Wednesday September 21, 2022 @07:34PM (#32933042)
For an offline (non-streaming) model, 1x realtime is actually kind of bad, because you need to wait for the audio to be available before you can start processing it. So if you wait 10 seconds for someone to finish speaking, you won't have the result until 10 seconds after that.

You could use really small chunk sizes and process them in a streaming fashion, but that would impact accuracy, as you're significantly limiting available context.

Re:Whisper – open source speech recognition by OpenAI by dom96 on Wednesday September 21, 2022 @04:55PM (#32931392)
I'd be interested to see how well it performs on something like an RPi. M1 is pretty beefy.
Re:Whisper – open source speech recognition by OpenAI by olao99 on Thursday September 22, 2022 @06:47AM (#32937135)
To be more precise the original comment said "M1 Max" which in itself is significantly beefier a bare "M1"
Re:Whisper – open source speech recognition by OpenAI by suyash on Wednesday September 21, 2022 @03:27PM (#32930250)
This is only one side of the coin, you still need really good models for Speech Synthesis and then be able to have it all working in almost real time, ideally locally on device.
Re:Whisper – open source speech recognition by OpenAI by ricopags on Wednesday September 21, 2022 @04:12PM (#32930892)
As far as TTS goes, Mycroft.ai[0] has released a decent offline one.

[0]https://mycroft.ai/

Re:Whisper – open source speech recognition by OpenAI by arbol on Tuesday September 27, 2022 @04:54PM (#33000970)
I'm pretty sure mycroft sends your speech snippets to Google for processing so it's not exactly offline.

https://mycroft-ai.gitbook.io/docs/using-mycroft-ai/customiz...

I'm currently trying to setup a deepspeech server on my raspberry pi to see if it works ok for commanding spotify.

Edit: just realised you said `TTS` not `STT`

Re:Whisper – open source speech recognition by OpenAI by smcameron on Thursday September 22, 2022 @07:31PM (#32946004)
pico2wave with "-l=en-GB" option to get the British lady voice is pretty decent (way better than the other voices it does for some reason).
Re:Whisper – open source speech recognition by OpenAI by solarkraft on Wednesday September 21, 2022 @04:30PM (#32931104)
Are you thinking about reimplementing Mycroft?

The Mycroft has done a lot of cool and important work in the field to ship an actual personal assistant product (stuff like wake word detection).

Re:Whisper – open source speech recognition by OpenAI by dom96 on Wednesday September 21, 2022 @04:54PM (#32931383)
hah, of course someone had the idea already and executed on it. But yeah, basically that but without the screen (probably would go a long way to decrease the cost, $299 is pretty steep for such a device)
Re:Whisper – open source speech recognition by OpenAI by sheepybloke on Wednesday September 21, 2022 @08:03PM (#32933245)
One thing they don't touch much on is the STT, as they use models from third parties. You could definitely do something that utilizes this model and then feeds the tokens to some of their parsing code. I've been working on something similar to this, but burned out around adding the STT portion [0].

[0]: https://github.com/Sheepybloke2-0/trashbot - It was called trashbot because the final implementation was going to look like oscar the grouch in a trashcan displaying the reminders.

Re:Whisper – open source speech recognition by OpenAI by MayeulC on Thursday September 22, 2022 @03:58AM (#32935989)
Well, you can always install Mycroft on a Pi, or on your computer.

Almond is also interesting as a voice assistant, though I think it doesn't perform speech recognition itself.