Re:An Introduction to Recurrent Neural Networks
by ma2rten on Thursday July 25, 2019 @04:55PM
(#20529604)
Mainly RNNs are much slower to train than transformers.
Re:An Introduction to Recurrent Neural Networks
by MiroF on Thursday July 25, 2019 @07:24PM
(#20530853)
As well as stability issues, long range dependencies, etc..
Re:An Introduction to Recurrent Neural Networks
by mrfox321 on Thursday July 25, 2019 @08:33PM
(#20531267)
It depends on the problem domain. Transformers are useful for NLP (language modeling, machine translation), but RNNs (and CNNs) are still being used in speech-to-text. When the input/output relationships are monotonic, transformers are probably too general.
I have to say that while I understand the problems with recurrent nets (which I've used many times), I haven't yet grokked the alternatives. Here are some decently looking search results for you as starting points. Warning, these can be longer and heavier reads probably not for beginners.
https://towardsdatascience.com/the-fall-of-rnn-lstm-2d1594c7... (there's some sensationalism here to be fair)
https://mchromiak.github.io/articles/2017/Sep/12/Transformer...
https://www.analyticsvidhya.com/blog/2019/06/understanding-t...
https://www.tensorflow.org/beta/tutorials/text/transformer
That being said, I think that understanding RNNs is very beneficial conceptually and nowadays there are relatively easy to use implementations that should be pretty good for many use cases.