News for hackers. Stuff that matters.
Re:Voice2json: Offline speech and intent recognition on Linux by sodality2 on Friday May 21, 2021 @01:15PM (#27236912)
Mozilla CommonVoice is definitely trying. I always do a few validations and a few clips if I have a few minutes to spare, and I recommend everyone does. They need volunteers to validate and upload speech clips to create a dataset.

https://commonvoice.mozilla.org/en

Re:Voice2json: Offline speech and intent recognition on Linux by teraflop on Friday May 21, 2021 @02:14PM (#27237760)
I like the idea, and decided to try doing some validation. The first thing I noticed is that it asks me to make a yes-or-no judgment of whether the sentence was spoken "accurately", but nowhere on the site is it explained what "accurate" means, or how strict I should be.

(The first clip I got was spoken more or less correctly, but a couple of words are slurred together and the prosody is awkward. Without having a good idea of the standards and goals of the project, I have no idea whether including this clip would make the overall dataset better or worse. My gut feeling is that it's good for training recognition, and bad for training synthesis.)

This seems to me like a major issue, since it should take a relatively small amount of effort to write up a list of guidelines, and it would be hugely beneficial to establish those guidelines before asking a lot of volunteers to donate their time. I don't find it encouraging that this has been an open issue for four years, with apparently no action except a bunch of bikeshedding: https://github.com/common-voice/common-voice/issues/273

Re:Voice2json: Offline speech and intent recognition on Linux by cptskippy on Friday May 21, 2021 @05:03PM (#27239948)
After listening to about 10 clips your point becomes abundantly clear.

One speaker, who sounded like they were from the mid-west United States, was dropping the S off words in a couple clips. I wasn't sure if it was misreads or some accent I'd never heard.

Another speaker, with a thick accent that sounded European, sounded out all the vowels in circuit. Had I not had the line being read, I don't think I'd have understood the word.

I heard a speaker with an Indian accent who added a preposition to the sentence that was inconsequential but incorrect none the less.

I hear these random prepositions added as flourishes frequently with some Indian coworkers, does anyone know the a reason? It's kind of like how American's interject "Umm..." or drop prepositions (e.g. "Are you done your meal?") and I almost didn't pick up on it. For that matter where did the American habit of dropping prepositions come from? It seems like it's people in the North East primarily.

Re:Voice2json: Offline speech and intent recognition on Linux by OJFord on Friday May 21, 2021 @05:28PM (#27240192)
I can't quite imagine superfluous prepositions (could you give an example?) but I have found it slightly amusing learning Hindi and coming across things where I think Oh! That's why you sometimes hear X from Indian English speakers, it's just a slightly 'too' literal¹ mapping from Hindi, or trying to use a grammatical construction that doesn't really exist in English, like 'topic marking'.

[¹] If that's even fair given it's a dialect in its own right - Americans also say things differently than I would as a 'Britisher'

Re:Voice2json: Offline speech and intent recognition on Linux by batch12 on Friday May 21, 2021 @07:19PM (#27241165)
Are you talking about phrases like "please do the needful"?
Re:Voice2json: Offline speech and intent recognition on Linux by OJFord on Friday May 21, 2021 @08:45PM (#27241995)
That's not one I've heard. Examples that come to mind are 'even I' (which seems closer to 'I too' than the 'you'd scarcely believe it but I' that it naturally sounds to me), 'he himself' (or similar subject emphasis), and adverb repetition.

I'd say it's mostly subtler (I suppose that should be the expected distribution!) things I've noticed though, they're just harder to recall as a result.

(Just want to emphasise I'm not making fun of anybody or saying anything's wrong, in case it's not clear in text. I'm just enjoying learning Hindi, fairly interested in language generally, and interested/amused to notice these things.)

Re:Voice2json: Offline speech and intent recognition on Linux by OJFord on Saturday May 22, 2021 @07:16PM (#27251086)
Just thought of another - '[something is] very less' - which comes, presumably, from कम being used for both little/few and less than.

Hindi is much more economical, to put it literally, one says things like 'than/from/compared to orange, lemon is sour', and 'orange is little/less [without comparison] sour'.

Which, I believe, is what gives rise to InE sentences like 'the salt in this is very less' (it needs more salt, there's very little).

Re:Voice2json: Offline speech and intent recognition on Linux by batch12 on Friday May 21, 2021 @07:17PM (#27241137)
I have never heard that example. Maybe it is regional. I have heard, "you done?" as an example.
Re:Voice2json: Offline speech and intent recognition on Linux by xwx on Friday May 21, 2021 @03:57PM (#27239144)
I downloaded the (unofficial) Common Voice app [1] and it provides a link to some guidelines [2], which also aren't official but look sensible and seem like the best there is at the moment.

[1] https://f-droid.org/packages/org.commonvoice.saverio/

[2] https://discourse.mozilla.org/t/discussion-of-new-guidelines...

Re:Voice2json: Offline speech and intent recognition on Linux by tootie on Friday May 21, 2021 @01:59PM (#27237550)
If you read the doc, it says voice2json is layer on top of the actual voice recognition engine. And it supports mozilla deep speech, pocket sphinx and a few others as the underlying engine.
Re:Voice2json: Offline speech and intent recognition on Linux by wcarss on Friday May 21, 2021 @01:41PM (#27237286)
I've used the deepspeech project a fair amount and it is good. It's not perfect, certainly, and it honestly isn't good enough yet for an accurate transcription in my mind, but it's good. Easy to work with, pretty good results, and all the right kinds of free.

Thanks for taking time to contribute!

Re:Voice2json: Offline speech and intent recognition on Linux by jfarina on Friday May 21, 2021 @01:37PM (#27237213)
I wonder if they use movies and tv; recordings where the script is already available.
Re:Voice2json: Offline speech and intent recognition on Linux by wongarsu on Friday May 21, 2021 @01:52PM (#27237466)
That's fine for training your own model, but I don't think you could distribute the training set. That seems like a clear copyright violation, against one of the groups that cares most about copyright.

Maybe you could convince a couple of indie creators or state-run programs to licence their audio? But I'm not sure if negotiating that is more efficient than just recording a bit more audio, or promoting the project to get more volunteers.

Re:Voice2json: Offline speech and intent recognition on Linux by akiselev on Friday May 21, 2021 @02:03PM (#27237605)
It would likely be a lot easier for someone from within the BBC, CBC, PBS, or another public broadcaster to convince their employer to contribute to the models. These organizations often have accessibility mandates with real teeth and real costs implementing that mandate. The work of closed captioning, for example, can realistically be improved by excellent open source speech recognition and TTS models without handing all of the power over to Youtube and the like.

It would still be an uphill battle to convince them to hand over the training set but the legal department can likely be convinced if the data set they contribute back is heavily chopped up audio of the original content, especially if they have the originals before mixing. I imagine short audio files without any of the music, sound effects, or visual content are pretty much worthless as far as IP goes.

Re:Voice2json: Offline speech and intent recognition on Linux by dec0dedab0de on Friday May 21, 2021 @02:05PM (#27237636)
That's fine for training your own model, but I don't think you could distribute the training set. That seems like a clear copyright violation, against one of the groups that cares most about copyright.

I'm not sure that is a clear copyright violation. Sure, at a glance it seems like a derivative work, but it may be altered enough that it is not. I believe that collages, and reference guides like cliff notes are both legal.

I think a bigger problem would be that the scripts, and even the closed captioning, rarely match the recorded audio 100%

Re:Voice2json: Offline speech and intent recognition on Linux by Wowfunhappy on Friday May 21, 2021 @05:07PM (#27239980)
And also... it's not like the program actually contains a copy of the training data, right? The training data is a tool which is used to build a model.
Re:Voice2json: Offline speech and intent recognition on Linux by taneq on Friday May 21, 2021 @05:18PM (#27240083)
How is it different from things like GPT3 which (unless I’m mistaken) is trained on a giant web scrape? I thought they didn’t release the model out of concerns for what people would do with a general prose generator rather than any copyright concerns?
Re:Voice2json: Offline speech and intent recognition on Linux by sodality2 on Friday May 21, 2021 @02:01PM (#27237575)
Does using copyrighted works to train a machine learning model make that model infringing?
Re:Voice2json: Offline speech and intent recognition on Linux by wongarsu on Friday May 21, 2021 @02:37PM (#27238083)
Generally a ML model transforms the copyrighted material to the point where it isn't recognizable, so it should be treated as its own unrelated work that isn't infringing or derivative. But then you have e.g. GPT that is reproducing some (largeish) parts of the training set word-for-word, which might be infringing.

Also I don't think there have been any major court cases about this, so there's no clear precedent in either direction.

Re:Voice2json: Offline speech and intent recognition on Linux by visarga on Friday May 21, 2021 @06:01PM (#27240515)
> But then you have e.g. GPT that is reproducing some (largeish) parts of the training set word-for-word, which might be infringing.

Easy fix - keep a bloom filter of hashed ngrams ensuring you don't repeat more than N words from the training set.

Re:Voice2json: Offline speech and intent recognition on Linux by pabs3 on Saturday May 22, 2021 @01:23AM (#27243747)
There are some that say that the Google Books court case is precedent for ML model stuff, if you search back through my comment history you will find links.
Re:Voice2json: Offline speech and intent recognition on Linux by sodality2 on Friday May 21, 2021 @04:16PM (#27239347)
Thanks!
Re:Voice2json: Offline speech and intent recognition on Linux by marcodiego on Friday May 21, 2021 @02:06PM (#27237652)
GP is not talking about the model but about the training data set.
Re:Voice2json: Offline speech and intent recognition on Linux by sodality2 on Friday May 21, 2021 @02:13PM (#27237735)
I am aware, I'm asking if the model, however, is infringing. Surely you can't distribute them in a dataset but is training on copyrighted data legal, and can you distribute that model?
Re:Voice2json: Offline speech and intent recognition on Linux by _jal on Friday May 21, 2021 @05:23PM (#27240138)
All text written by a human in the US is automatically copyright the author. So if an engine trained on works under copyright is a derivative work, GPT3 and friends have serious problems.
Re:Voice2json: Offline speech and intent recognition on Linux by kelnos on Friday May 21, 2021 @03:01PM (#27238432)
I expect that wouldn't be perfect, though. Sometimes the cut that makes it into the final product doesn't exactly match the script. Sometimes it's due to an edit, other times it's due to an actor saying something similar to but not exactly what the script says, but the director deciding to just go with it.

What might work better is using closed captions or subtitles, but I've also seen enough cases where those don't exactly match the actual speech either.

Re:Voice2json: Offline speech and intent recognition on Linux by taneq on Friday May 21, 2021 @05:19PM (#27240093)
They might work even better for interpreting the intent of spoken text. Not great for dictation though.
Re:Voice2json: Offline speech and intent recognition on Linux by habibur on Friday May 21, 2021 @03:14PM (#27238606)
He meant subtitle when he talked of script.
Re:Voice2json: Offline speech and intent recognition on Linux by cerved on Friday May 21, 2021 @05:26PM (#27240164)
Weird sentences