News for hackers. Stuff that matters.
Re:SoundStream: An End-to-End Neural Audio Codec by ngrilly on Sunday August 15, 2021 @03:53AM (#28186959)
Next step: train a NN to predict what we’re going to say to compensate network latency :)
Re:SoundStream: An End-to-End Neural Audio Codec by vletal on Sunday August 15, 2021 @04:01AM (#28186986)
Have you ever played a counterstrike with a lagging network? Observe the teammates running into walls and such. That's what the conversation would sound like.
Re:SoundStream: An End-to-End Neural Audio Codec by vbezhenar on Sunday August 15, 2021 @10:26AM (#28188994)
Running into a wall is not a result of NN prediction. It's a result of naïve linear prediction. NN will predict that he'll stop, turn around, may be shot someone. The hard thing is to not disappoint NN by actually running into a wall.
Re:SoundStream: An End-to-End Neural Audio Codec by vletal on Sunday August 15, 2021 @02:49PM (#28191314)
You need to take into consideration the complexity. Counterstrike has very tiny set of possible actions you can take. Compare that to the space of things one can say.

A NN is going to fail on that at least as missereably as linear interpolation fails in CS.

Re:SoundStream: An End-to-End Neural Audio Codec by hypertele-Xii on Monday August 16, 2021 @06:31AM (#28196617)
When speech prediction fails, it should sound clearly wrong. Otherwise we risk serious misunderstanding when the prediction says something that sounds good but means different.

We already have trouble like this with texting autocorrection.

Re:SoundStream: An End-to-End Neural Audio Codec by meowface on Sunday August 15, 2021 @06:47AM (#28187719)
It could make for a hilarious new comedic art form.
Re:SoundStream: An End-to-End Neural Audio Codec by lxgr on Sunday August 15, 2021 @04:18AM (#28187064)
Don‘t modern codecs already do something like this to conceal packet loss? (Obviously only at the phoneme and not word or semantic level.)
Re:SoundStream: An End-to-End Neural Audio Codec by amelius on Sunday August 15, 2021 @05:01AM (#28187275)
Does this work for the starting phoneme of a word?
Re:SoundStream: An End-to-End Neural Audio Codec by pbhjpbhj on Sunday August 15, 2021 @06:53AM (#28187748)
Just running with that a little ...

So, at what point during a phoneme does it become distinct?

After any one phoneme has been sent down the wire you can truncate the audio and send a phoneme identifier; at the other end they just replay the phoneme from a cache.

Like doing speech to text, but using phonemes, and sending the phoneme hash string instead of the actual audio.

Must have been done already?

Re:SoundStream: An End-to-End Neural Audio Codec by ngrilly on Sunday August 15, 2021 @04:12PM (#28192084)
Wondering the same.
Re:SoundStream: An End-to-End Neural Audio Codec by mchusma on Sunday August 15, 2021 @04:10PM (#28192067)
Googles duo already does this. In my experience, it's amazing. http://ai.googleblog.com/2020/04/improving-audio-quality-in-...
Re:SoundStream: An End-to-End Neural Audio Codec by ngrilly on Saturday August 21, 2021 @11:24AM (#28257722)
I remember reading that paper. This is amazing. But it's not really predicting what we're gone say beyond the current syllable:

> While our model learns how to plausibly continue speech, this is only true on a short scale — it can finish a syllable but does not predict words, per se.