I've also run a BBC news report through the program with better results although it demonstrates that any background noise at all can throw things off significantly: https://twitter.com/peterc/status/1111736029558517760 .. so at this low bitrate, it really is only good for plain speech without any other noise.
When it comes to noisy speech, it should be possible to improve things by actually training on noisy speech (the current model is trained only on clean speech). Stay tuned :-)
It holds up ridiculously well considering the entire song compresses down to 25392 bytes.
I am curious how it sounds when there is a really active bassline and lead synth.
The vocal sections just sound like someone clearing their throat out.
the music barely at all.
I suspect the reason that excerpt sounds so bad is because the music has several instruments playing at once. One doesn't generally design a vocoder to deal with more than one voice. As that except plays, you can hear that the most prominent instruments (eg: the bass at several moments) sound pleasing, albeit speech-like.It would probably different from the original music, but pleasant, if one processed each track separately.
Heavy compression of voice is vulnerable to background noise.
I miss the classic telco 8K samples per second, 8 bits. We used to think that was crappy audio.
Also based on work by Xiph. Possibly using the same LPC used here.
https://en.wikipedia.org/wiki/Adaptive_Multi-Rate_audio_code...
OPUSFLAGS = --vbr --bitrate 16
all: $(patsubst %.wav,%.opus,$(wildcard *.wav))
.PHONY: all
%.opus: %.wav
opusenc $(OPUSFLAGS) $< $@
Make -j4 or whatever. There are a few other ways to do this (e.g. xargs).
1.6 kbps is nuts! I like to re-encode audio books or podcasts in Opus at 32 kbps and I consider that stingy. The fact that speech is even comprehensible at 1.6 kbps is impressive. As the article explains, their technique is analogous to speech-to-text, then text-to-speech.
The original recordings are a little stiff, and the encoded speech is a little more stiff. It isn't perfect, but it's decent. It'll be interesting to hear this technique applied to normal conversation. If regular speech holds up as well as their samples, it should be perfectly adequate for conversational speech. At 1.6 kbps, which is absurd.
Also, I wonder how well this technique could be applied to music. My guess is that it won't do justice to great musicians ... but it might be good enough simple pop tunes.