Re:Exploring LSTMs
by griffinkelly on Saturday June 10, 2017 @11:53AM
(#14527770)
So if LSTMs are purposely forgetting, do you need less training data than a CNN?
Re:Exploring LSTMs
by agibsonccc on Saturday June 10, 2017 @05:47PM
(#14529364)
LSTMs don't "forget" more than "remember the things that matter". They don't necessarily need less data. They do have a limit on the "length" of time steps they can handle though.
Eg: You can't do thousands in to the future (maybe a few hundred or so)
The long part of "LSTM" means remember good long ranging dependencies.
Usually as long as you have a high amount of regularization and use truncated backprop through time in training you can learn some fairly useful classification and forecasting problems.
Beyond that standard neural net tuning applies. Eg: normalize your data, pay attention to your weight initialization, understand what loss function you're using,..