> available data sets

Reminder that Mozilla's Common Voice project accepts voice donations! https://voice.mozilla.org/

Interesting to see Kabyle as the language with the third most validated hours at https://voice.mozilla.org/en/languages

Kabylia is a region in the north of Algeria mostly inhabited by Berber people who are bilingual in Algerian Arabic and Kabyle. In recent years, an independence movement has developed that emphasizes Kabyle over Arabic for reasons of internal cohesion. To confuse matters, there's also a pan-Berber movement denying the existence of a separate Kabyle language, classifying it as a dialect of Berber/Tamazight instead.

Those heated politics have led to a large number of Kabyles contributing to various linguistic corpus projects to gain visibility for their cause. E.g. trying to overtake Berber on https://tatoeba.org/stats/sentences_by_language (As far as I know, Mozilla's Common Voice shares data with the Tatoeba project.)

Given the value of open source training data (or scarcity of it), has anybody attempted to use LibriVox for training?

https://en.wikipedia.org/wiki/LibriVox

https://librivox.org/

The recordings are public domain audio books of public domain books, so the licensing should be fine. The audio isn't annotated, but given the value involved I think it would be worth attempting to use forced alignment to annotate the recordings with their public domain source texts. Forced alignment using the sort of speech recognizer you're trying to train in the first place may be a bit "chicken and the egg", but from some experiments I've run myself existing open source speech recognizers can do it reasonably well. Humans could manually tune up the alignment to improve the quality if necessary.

As for motivating people to actually do that mundane work... well these are audio books so maybe the work isn't so mundane after all! The LibriVox recording of Tom Sawyer (read by John Greenman: https://librivox.org/tom-sawyer-by-mark-twain/) is pretty great and has been listened to by millions of people. If somebody created a "read along" web app that showed you the text of the book from Project Gutenberg getting highlighted as the audiobook from LibriVox was played, users who have an interest in reading/hearing the book could have their attention held by Mark Twain and with the right UI provide fine tuning for the forced alignment at the same time.

LibriVox recordings are what the LibriSpeech corpus [1, 2] is based on. That's what they use in the paper.

[1] Panayotov, V., Chen, G., Povey, D. and Khudanpur, S. (2015). LibriSpeech: an ASR corpus based on public domain audio books. Proc. ICASSP. http://www.danielpovey.com/files/2015_icassp_librispeech.pdf

[2] http://openslr.org/12/

Librivox is commonly used as a training corpus however it's main weakness is that reading speech differs quite a lot from conversational speech.

I see. I hadn't considered that might be a weakness because to be honest the creation of forced alignments between LibriVox and the source texts was my objective (it's a feature that exists on some Kindle's when pairing ebooks with audible audio books. I believe the feature makes literature more accessible, a noble enough cause. Although I can understand why more effort is being spent on recognizing conversational speech.)

Have you seen https://www.readbeyond.it ?

> a huge pain to set up.

This project doesn't look like it's particularly easy to build: https://github.com/facebookresearch/wav2letter/blob/master/d...

The problem with Kaldi is that it's not a turnkey solution for a speech recognition system, but a collection of libraries and shell scripts that can be used to build your own system, assuming you're a researcher in speech recognition or are willing to put in the time to become one. A long list of dependencies appears less daunting in comparison.