Re:Understanding LSTM and its diagrams
by PaulHoule on Wednesday November 15, 2017 @06:54PM
(#15709158)
Pro Tip: The GRU is a much simpler unit (easier to explain) and usually works better than the LSTM.
[ Parent
| on HN ]
Re:Understanding LSTM and its diagrams
by reachtarunhere on Wednesday November 15, 2017 @11:05PM
(#15710195)
Interesting paper that compares performance of GRU vs LSTM. http://proceedings.mlr.press/v37/jozefowicz15.pdf
TL;DR - For most tasks other than Language modeling GRU is better.
Re:Understanding LSTM and its diagrams
by cdancette on Thursday November 16, 2017 @11:58AM
(#15714260)
why is LSTM better for language modeling?
Re:Understanding LSTM and its diagrams
by reachtarunhere on Sunday November 19, 2017 @05:57AM
(#15733346)
The paper doesn't speak much. Empirical results.
Re:Understanding LSTM and its diagrams
by nonbel on Wednesday November 15, 2017 @06:59PM
(#15709182)
What do you mean by "works better"? Do you mean something beyond having fewer parameters to store/calculate?
Re:Understanding LSTM and its diagrams
by PaulHoule on Wednesday November 15, 2017 @07:02PM
(#15709198)
Usually it requires fewer epochs to train the GRU than the LSTM.
The learning capacity of the LSTM and GRU is proportional to the number of weights, so you need more GRUs to do the job of a given number of LSTMs.
It drives me nuts to see so many tutorials where people do the LSTM because they saw other people do the LSTM when doing the same with the GRU is simpler and better!