Re:Oxford Deep NLP – An advanced course on natural language ...
by option_greek on Tuesday February 07, 2017 @07:33AM
(#13588912)
What are the practical uses of language modelling RNNs ? (apart from writing grammar/syntax checkers)
[ Parent
| on HN ]
Re:Oxford Deep NLP – An advanced course on natural language ...
by spangry on Tuesday February 07, 2017 @08:50AM
(#13589273)
I wonder if they could be used to improve speech recognition accuracy. So you'd have two models running when someone utters a sentence: the first would generate the x most likely phrases that it thinks were spoken, and the second (the RNN) would select the highest ranked 'plausible' sentence (i.e. a sentence it would have been able to generate itself).
I guess that's a bit indirect, but these RNNs are essentially learning the 'rules' that actual phrases conform to. It'd definitely be better than trying to hard-code the rules (especially for a language like English!). And the training data is very easy to get: just feed it a few thousand ebooks, the comments section from HN etc.
Re:Oxford Deep NLP – An advanced course on natural language ...
by bshillingford on Tuesday February 07, 2017 @11:41AM
(#13590528)
Standard neural network-based speech recognition pipelines (i.e. RNN + CTC) always use a language model. Unlike a seq2seq model (or any autoregressive model, or a structured prediction output), CTC models output timesteps as conditionally independent. Hence, everyone uses an RNN LM or n-gram LM or both when retrieving probable sequences from a CTC model (e.g. with beam search).
Re:Oxford Deep NLP – An advanced course on natural language ...
by rkjaran on Tuesday February 07, 2017 @11:35AM
(#13590486)
Language models are a standard component of speech recognition. Although N-gram LMs are used much more often than RNN LMs.
Re:Oxford Deep NLP – An advanced course on natural language ...
by circuithunter on Tuesday February 07, 2017 @10:49AM
(#13590095)
Machine translation is a big one. See the recent NY Times feature [1] and the arxiv paper [2]. Automated image captioning is another (used extensively by Facebook).
[1] https://www.nytimes.com/2016/12/14/magazine/the-great-ai-awa...
Re:Oxford Deep NLP – An advanced course on natural language ...
by visarga on Tuesday February 07, 2017 @07:50AM
(#13588981)
Language encoding and generation (sequence to sequence).
Re:Oxford Deep NLP – An advanced course on natural language ...
by option_greek on Tuesday February 07, 2017 @08:00AM
(#13589041)
Could you expand a bit on that. I have come across examples generating poems from Shakespeare works and realistic looking css/javascript but still trying to find out a more realistic usecase.
Re:Oxford Deep NLP – An advanced course on natural language ...
by matt4711 on Tuesday February 07, 2017 @08:10AM
(#13589085)
Say you want to translate a sentence from language A to language B. You have a system that generates 10 possible translations in language B. Now you use a language model of language B to figure out which is the best translation by asking the language model which of the translations has the highest probability of being generated by the model.
Re:Oxford Deep NLP – An advanced course on natural language ...
by greeneggs on Tuesday February 07, 2017 @08:58AM
(#13589321)
Is there any way to "translate" between writing styles of the same language? I'm thinking something analogous to the Van Gogh-ify image-processing techniques using deep convolutional networks.
Even very simple transformations, e.g., adding alliteration/assonance or adding rhymes everywhere, might be fun.
Re:Oxford Deep NLP – An advanced course on natural language ...
by stephancoral on Tuesday February 07, 2017 @11:03AM
(#13590214)
Look into Google's SyntaxNet - https://research.googleblog.com/2016/05/announcing-syntaxnet...
Should help you break down sentences into their semantic parts. The transformations are then made by walking the syntax tree and modifying the tagged parts of speech as you see fit.
Re:Oxford Deep NLP – An advanced course on natural language ...
by sprobertson on Tuesday February 07, 2017 @02:56PM
(#13592165)
Chat bots, image to caption, and question answering are some more realistic use cases of the generator side. In those cases there is some input (previous chat message, an image, a question) which is encoded into a vector sometimes referred to as a context or thought vector. The decoder/generator unfolds that vector into a series of words.