Re:A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet
by bravura on Friday March 29, 2019 @09:42PM
(#19527058)
So can you do domain adaptation and get it to vocode my voice into Johnny Cash's? Larynx adaptation.
Re:A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet
by jmvalin on Friday March 29, 2019 @09:47PM
(#19527084)
In theory, it wouldn't be too hard to implement with an neural network. In theory. In practice, the problem is figuring out how to do the training because I don't have 2 hours of your voice saying the same thing as the target voice and with perfect alignment. I suspect it's still possible, but it's not a simple thing either.
Re:A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet
by krisoft on Saturday March 30, 2019 @05:00AM
(#19528517)
Perfect alignment, or any alignment for that matter, is not necessary. Check out adversarial networks. CycleGAN can be trained to do similar feats in image domain without aligned inputs. Shouldn't be hard to adopt it to audio.
Re:A Real-Time Wideband Neural Vocoder at 1.6 Kb/S Using LPCNet
by jmvalin on Saturday March 30, 2019 @10:38AM
(#19529609)
I didn't say "impossible", merely "not simple". The minute you bring in a GAN, things are already not simple. Also, I'm not aware of any work on a GAN that works with a network that does conditional sampling (like LPCNet/WaveNet), so it would mean starting from scratch.
The bitrate could definitely be reduced (possibly by 50%+) by using packets of 1 seconds along with entropy coding, but the resulting codec would not be very useful for voice communication. You want packets short enough to get decent latency and if you use RF, then VBR makes things a lot more complicated (and less robust).