Re:RWKV: Reinventing RNNs for the Transformer Era
by MontyCarloHall on Tuesday May 23, 2023 @08:23AM
(#36043267)
>Many people could reason about the book in its entirety
That’s analogous to having War and Peace in the training set. When actually reading War and Peace, nobody‘s working memory includes a precise recall of everything they’ve read thus far, which is more analogous to an LLM’s context size.
That’s analogous to having War and Peace in the training set. When actually reading War and Peace, nobody‘s working memory includes a precise recall of everything they’ve read thus far, which is more analogous to an LLM’s context size.