News for hackers. Stuff that matters.
Re:ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based... by solomatov on Thursday January 19, 2023 @08:20PM (#34447989)
My understanding is that RNNs aren't worse than Transformers per se, they are just slower to train, and use GPU much more efficiently, i.e. much more stuff could be run in parallel.
Re:ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based... by Hendrikto on Friday January 20, 2023 @07:14AM (#34452016)
Also slower to perform inference on. RNNs have to be much more sequential.
Re:ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based... by euclaise on Friday January 20, 2023 @01:42PM (#34457680)
We also don't have evidence that they scale the way transformers do