Re:RWKV Language Model
by low_tech_punk on Thursday January 02, 2025 @02:58AM
(#42572617)
Thanks! The 0.1B version looks perfect for embedded system. What is the key benefit of attention-free architecture?
[ Parent
| on HN ]
Re:RWKV Language Model
by pico_creator on Thursday January 02, 2025 @07:28AM
(#42573843)
lower compute cost especially over longer sequence length. Depending on context length, its 10x, 100x, or even 1000x+ cheaper. (quadratic vs linear cost difference)