News for hackers. Stuff that matters.

A comment on Show HN: Text-to-speech and speech-to-text open-source software stack.

Re:Show HN: Text-to-speech and speech-to-text open-source so... by ftreml on Sunday January 26, 2020 @02:22PM (#22154090)
This project is the result of a one year long learning process in speech recognition and speech synthesis.

The original task was to automate the testing of a voice-enabled IVR system. While we started with real audio recordings, very soon it was clear that this approach is not feasible for a non-trivial app and it will be impossible to reach a satisfying test coverage. On the other hand, we had to find a way to transcribe the voice app response to text for doing our automated assertions.

As cloud-based solutions where not an option (company policy), we very quickly got frustrated as there was no "get shit done" Open Source stack available for doing medium-quality text-to-speech and speech-to-text conversions. We learned how to train and use Kaldi, which is according to some benchmarks the best available system out there, but mainly targeting academic users and research. We made heavy-weight MaryTTS work to synthesize speech in reasonable quality.

And finally, we packaged all of this in a DevOps-friendly HTTP/JSON API with a Swagger definition.

As always, feedback and contributions are welcome!

[ Parent | on HN ]
Re:Show HN: Text-to-speech and speech-to-text open-source so... by ayumu722 on Monday January 27, 2020 @04:20AM (#22157932)
I've built a speech synthesis system with Marytts before, it works, but unfortunately, the quality is not very good, HMM and unit selection speech synthesis are now very old approaches, they are far from the current state of the art, you should try open-source implementations of Tacotron2 or wavnet, you will surely achieve better quality.
[ Parent | on HN ]
Re:Show HN: Text-to-speech and speech-to-text open-source so... by ftreml on Monday January 27, 2020 @05:29AM (#22158154)
are you ready for a pull request ?
[ Parent | on HN ]
Re:Show HN: Text-to-speech and speech-to-text open-source so... by ayumu722 on Monday January 27, 2020 @07:06AM (#22158564)
I'll be very pleased, but I can't promise anything. Good job by the way, Speech synthesis and Speech recognition are not easy subjects to work on.
[ Parent | on HN ]
Re:Show HN: Text-to-speech and speech-to-text open-source so... by yorwba on Sunday January 26, 2020 @05:17PM (#22155184)
You should indicate somewhere which languages are supported. You mention Deepspeech German in the credits, but did you use it to implement support for German or just to inform your architectural choices?
[ Parent | on HN ]
Re:Show HN: Text-to-speech and speech-to-text open-source so... by ftreml on Sunday January 26, 2020 @05:26PM (#22155231)
good point, will add this information. in short, german and english, because those are the languages i am comfortable with. i hope to find native speakers contributing more languages. deepspeech was evaluated, but right now kaldi provides better performance, thats why we stick to kaldi by default.
[ Parent | on HN ]
Re:Show HN: Text-to-speech and speech-to-text open-source so... by StudentStuff on Sunday January 26, 2020 @07:22PM (#22155878)
Mozilla DeepSpeech has seen significant improvements in the quality of results it provides (and RAM/CPU usage) since early December, the accuracy of transcriptions has substantially improved in our testing.
[ Parent | on HN ]
Re:Show HN: Text-to-speech and speech-to-text open-source so... by Erlich_Bachman on Wednesday January 29, 2020 @06:33AM (#22179448)
Can you provide some quantifiable results, like word error rate or something? Do you think it is better than what Kaldi offers currently? (In terms of accuracy, not in terms of computational performance.)
[ Parent | on HN ]
Re:Show HN: Text-to-speech and speech-to-text open-source so... by StudentStuff on Monday February 03, 2020 @09:11PM (#22231687)
Every release mentions the word error rate, its interesting to watch the WER improve over time: https://github.com/mozilla/DeepSpeech/releases
[ Parent | on HN ]
Re:Show HN: Text-to-speech and speech-to-text open-source so... by magicalhippo on Sunday January 26, 2020 @06:37PM (#22155659)
> deepspeech was evaluated, but right now kaldi provides better performance

Was this before the streaming API was added to DeepSpeech? I recently did some testing with it, and it provides text within ~100ms of last audio block on my PC.

edit: that is, the most significant latency I had was from having to wait a bit to detect end of speech.

[ Parent | on HN ]
Re:Show HN: Text-to-speech and speech-to-text open-source so... by ftreml on Sunday January 26, 2020 @06:46PM (#22155708)
with "performance" i meant the error rate, not the speed. speed was not a criteria for us, so we didnt evaluate it. i read that deepspeech is way quicker in training phase as it is smarter in using gpu computing power.
[ Parent | on HN ]
Re:Show HN: Text-to-speech and speech-to-text open-source so... by magicalhippo on Sunday January 26, 2020 @06:55PM (#22155750)
Ahh gotcha. Yes I got rather poor "hit rate" until I made my own language model.

Fortunately for me I only needed it for command recognition, so the process was quite quick, and results were very good. No need to retrain the net.

[ Parent | on HN ]
Re:Show HN: Text-to-speech and speech-to-text open-source so... by ftreml on Sunday January 26, 2020 @07:07PM (#22155799)
we used a context-free data set, augmented with additional domain-specific sound samples and it worked out fine, although the additiinal samples made nearly no difference
[ Parent | on HN ]
Re:Show HN: Text-to-speech and speech-to-text open-source so... by dillonmckay on Sunday January 26, 2020 @03:35PM (#22154591)
What was the IVR software you were using, was that bespoke?
[ Parent | on HN ]
Re:Show HN: Text-to-speech and speech-to-text open-source so... by ftreml on Sunday January 26, 2020 @04:13PM (#22154837)
no, it was a special call center software called vacapo.
[ Parent | on HN ]