News for hackers. Stuff that matters.
Re:ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based... by jszymborski on Friday January 20, 2023 @01:20AM (#34449798)
While this is true, and was a major advantage of LSTMs/GRUs, they still suffer from vanishing gradients.

w.r.t proteins, our sequences often surpass 1500 amino acids and that is really tough for an LSTM to stably train on.

There are no comments on this comment yet.