News for hackers. Stuff that matters.

A comment on Universal Speech Model. Parent comment

Re:Universal Speech Model by taberiand on Wednesday March 29, 2023 @10:04PM (#35366835)
Does Whisper do streaming translation?

I'm imagining the near future will see a portable fast streaming model for real-time voice translation, piped into text to speech. Hook it up to an earpiece and you've got a real-life Babelfish

[ Parent | on HN ]
Re:Universal Speech Model by speedgoose on Thursday March 30, 2023 @12:36AM (#35368026)
Yes it does streaming but you may have to reduce the quality to keep up if your machine is not very fast.
[ Parent | on HN ]
Re:Universal Speech Model by woodson on Thursday March 30, 2023 @12:37AM (#35368033)
Whisper doesn’t do streaming transcription or translation. It’s one of its architectural disadvantages over more commonly used ASR approaches, such as transducers (see e.g. Microsoft Research’s system trained on ~350k hours that does streaming transcription and translation: https://arxiv.org/abs/2211.02499).
[ Parent | on HN ]