Neural nets that update over time need a way to know what to keep and what to forget from last time. So let's learn what to keep and forget... by using more neural nets.

Disclaimer: I just learned what an LSTM was. But it's a good article.

This is a surprisingly accurate tl;dr for someone who just learnt what LSTMs are.

One more key point:. Memory is additive, not multiplicative.

And LSTM stands for Long/Short Term Memory, the definition is only found about a third of the way through the article.

Knowing what the actual units do, I prefer spelling out the acronym as Long Short-Term Memory. They implement short-term memory, like all RNNs do, with a slight improvement to make it long.