Silero V3: fast high-quality text-to-speech in 20 languages with 173 voices

Re:Silero V3: fast high-quality text-to-speech in 20 languag... by rixrax on Monday June 20, 2022 @04:22AM (#31807672)

Loosely related, but what's the state of the art in natural text to speech ML/AI models?

[ Parent | on HN ]

Re:Silero V3: fast high-quality text-to-speech in 20 languag... by fxtentacle on Monday June 20, 2022 @04:25AM (#31807702)

text to WAV:

You predict mel spectrums with a transformer architecture (so word embedding + attention decoder) and then convert them into audio signals with Parallel WaveGAN or Hifi-GAN.

FastSpeech2 (Microsoft) is extremely good.

WAV to text:

You detect wave shapes with convolutions to generate an embedding, then attention layers to turn it into an encoding, then convert that to logits. Logits go into language model and that predicts the final sentence with a beam search decoder.

wav2vec 2.0 (Facebook) is amazing.

[ Parent | on HN ]

Re:Silero V3: fast high-quality text-to-speech in 20 languag... by forgingahead on Monday June 20, 2022 @04:23AM (#31807682)

Tortoise-TTS (stylised as "TorToiSe") is pretty amazing: https://github.com/neonbjb/tortoise-tts

[ Parent | on HN ]

Re:Silero V3: fast high-quality text-to-speech in 20 languag... by snakers41 on Monday June 20, 2022 @04:59AM (#31807942)

Silero TTS works fast even on one CPU thread, this is the point

[ Parent | on HN ]