News for hackers. Stuff that matters.
Re:An Introduction to Recurrent Neural Networks by stereolambda on Thursday July 25, 2019 @02:55PM (#20528289)
It's worth noting that apparently (as I learned lately) RNNs are going slightly out of fashion because they are hard to parallelize and have trouble remembering important stuff at larger distances. Transformers are proposed as a possible solution - very roughly speaking, they use attention mechanisms instead of recurrent memory and can run in parallel.

I have to say that while I understand the problems with recurrent nets (which I've used many times), I haven't yet grokked the alternatives. Here are some decently looking search results for you as starting points. Warning, these can be longer and heavier reads probably not for beginners.

https://towardsdatascience.com/the-fall-of-rnn-lstm-2d1594c7... (there's some sensationalism here to be fair)

https://mchromiak.github.io/articles/2017/Sep/12/Transformer...

https://www.analyticsvidhya.com/blog/2019/06/understanding-t...

https://www.tensorflow.org/beta/tutorials/text/transformer

That being said, I think that understanding RNNs is very beneficial conceptually and nowadays there are relatively easy to use implementations that should be pretty good for many use cases.

Re:An Introduction to Recurrent Neural Networks by ma2rten on Thursday July 25, 2019 @04:55PM (#20529604)
Mainly RNNs are much slower to train than transformers.
Re:An Introduction to Recurrent Neural Networks by MiroF on Thursday July 25, 2019 @07:24PM (#20530853)
As well as stability issues, long range dependencies, etc..
Re:An Introduction to Recurrent Neural Networks by mrfox321 on Thursday July 25, 2019 @08:33PM (#20531267)
It depends on the problem domain. Transformers are useful for NLP (language modeling, machine translation), but RNNs (and CNNs) are still being used in speech-to-text. When the input/output relationships are monotonic, transformers are probably too general.