Mistakes are part of real life, just like how you spelled speach, instead of speech.

The parent comment has a failure rate of 1/38 words, which means a 2.6% failure rate, and this is one of the best commentators in the world, too.

The human error rate on this task is 4% (https://pdfs.semanticscholar.org/387e/7349b8e31e316c2a738060...). What is amazing is how consistently people overestimate human performance.

Even humans make mistakes: speach -> speech deliberatly -> deliberately I expect these systems to surpass human ability in the next few years.

My bet is that is not really about audio quality, but because in order to understand speech you need to fully understand the language. Try to write down something you hear in a foreign language. As someone who has learnt English as a foreign language I can assure you that at first I wasn't able to even separate between words, it just sounded as a continous stream of sound. Once you learn enough vocabulary you identify some words. Once you identify enough words you are able to fill the gaps using the context. With practice the whole process happens so fast you don't even notice.

I wish I only failed to recognize 6% of the words when listening to crappy teleconference calls...

You listen on teleconference calls?

I like it when people say "Can you repeat that, I was on mute"

Well, we have to put human into comparison in order to know exactly how impressive 6% metric is. I would guess it might be pretty close.

I wouldn't blame the audio quality. It is more about different humans have different kinds of accent which makes the same word sound differently. Considering the different accents, I would say 6% error rate is pretty good compared where we started.

Why shouldn't the audio quality factor in? Isn't it easier to understand 44.1 Khz 16 bit CD quality audio than 8khz 8 bit pcm (phone data).

It might be, but the resource demands of higher fidelity acoustic models slows processing down. 44.1/16 has an order of magnitude greater bitrate than 8/8.

I guess the point of this particular data set is to check performance on low quality phone audio

The base product use case has been to handle phone fidelity for many years. Think: legal dictation, retail digital recording hardware (phones), and medical transcription. Speech-to-text for recordings fed by a Telefunken U-47 is highly niche. :)

heh. It's not so much because of the microphones that the data sets are THAT narrow band - it's more phone bandwidth limitations. Even the cheapest electret and MEMS mics have pretty good freq response, far beyond 4khz (nyquist 8khz) this data set uses.

Now that bandwidth is becoming less of an issue, we will be getting less shitty sounding, wider bandwidth phone audio - https://en.wikipedia.org/wiki/Wideband_audio

Though if I had it my way, U87 or U47, or hey even SM7B would be mandatory for all speech recordings :)