Re:I made a free transcription service powered by Whisper AI
by jcalvinowens on Friday November 18, 2022 @08:02PM
(#33665428)
> the sampling, the jargon, the 10 codes, the background noise, static on analog systems/drop outs on digital systems, rate of speech, etc
Is the reduced bandwidth really the most significant problem? Naively I'd think everything else you mentioned would matter a lot more, I'm curious how much you experimented with that specifically.
Re:I made a free transcription service powered by Whisper AI
by kkielhofner on Friday November 18, 2022 @08:08PM
(#33665488)
When it all comes together it's kind of a nightmare for an ASR model. I'm not sure which contributes most but I know from my prior experiences with ASR for telephony even clean speech on pristine connections does much worse with models trained on 16 kHz being fed native 8 kHz audio that gets resampled. There were plenty of times in reviewing the recordings and ASR output where I'd listen to the audio and have no idea what they said.
I've done some early work with Whisper in the telephony domain (transcribing voicemails on Asterisk and Freeswitch) and the accuracy already seems to be quite a bit worse.
Re:I made a free transcription service powered by Whisper AI
by ivalm on Saturday November 19, 2022 @12:07AM
(#33667140)
Could one train an interpolation layer (eg take a bunch of 16k audio, down sample to 8k, train 8k->16k upsampler)? Or better yet (but more expensive), take whisper, freeze it, and train the upsampler on whisper’s loss.
Re:I made a free transcription service powered by Whisper AI
by kkielhofner on Saturday November 19, 2022 @12:16AM
(#33667191)
"I understand some of those words."
Hah, in all seriousness I'm more of a practitioner in this space. If this was something I absolutely needed to get done who knows where it would have went. For a little side hacking project once I encountered these issues I moved on - back in the day expectations were lower for telephony and the 8 kHz aspire models and kaldi were adequate to get that "real work" done.
Re:I made a free transcription service powered by Whisper AI
by andrew3726 on Saturday November 19, 2022 @05:15AM
(#33668845)
Sure, that's called audio super resolution, there's a few papers/projects doing that. Haven't really seen models which are robust and have good generalization though.
Did you try a telephony oriented model like aspire or similar? They’re trained on sort-of 8 kHz audio and might work better.
I tried something similar for my SDR feeds and gave up because it’s just too challenging and niche - the sampling, the jargon, the 10 codes, the background noise, static on analog systems/drop outs on digital systems, rate of speech, etc all contribute to very challenges issues for an ML model.