Facebook open-sources a speech-recognition system and a machine learning library

Re:Facebook open-sources a speech-recognition system and a m... by nl on Saturday December 22, 2018 @07:07AM (#18740236)

I do a lot of sequence work (text).

While CNNs are tempting and fast to train I've never been able to get the accuracy I can from RNNs. In NLP, accuracy is important because for lots of tasks NLP is right at that inflection point of being good enough to be useful.. if it's good enough.

It's worth noting that this TCN paper gets a perplexity of 45.19 on WikiText-103. That was competitive in 2015.

The current state of the art is 29.2[1] - not their claim of 48.4 (unclear what that came from).

Still, CNNs are nice in an ensemble model if that's your thing. They do tend to pick up different things to RNNs, which can be useful.

Edit: I now understand why their reported metrics are so wrong. They use generic models to compare against. They list SOTA performance in their supplementary material (they still get them wrong though).

[1] http://nlpprogress.com/english/language_modeling.html

[ Parent | on HN ]

Re:Facebook open-sources a speech-recognition system and a m... by stochastic_monk on Saturday December 22, 2018 @03:58PM (#18742416)

Thanks for digging into how their claims are misleading. As someone in an adjacent subfield, I was wondering if I was missing something. It does at least sound like an interesting potential component.

[ Parent | on HN ]