News for hackers. Stuff that matters.

A comment on RWKV Language Model. Parent comment

Re:RWKV Language Model by Ey7NFZ3P0nzAe on Thursday January 02, 2025 @03:37AM (#42572766)
Has there been progress towards making RWKV multimodal? Can be use projector layers to send images to RWKV?
[ Parent | on HN ]
Re:RWKV Language Model by pico_creator on Thursday January 02, 2025 @07:35AM (#42573888)
There is work done for Vision RWKV, and audio RWKV, an example paper is here: https://arxiv.org/abs/2403.02308

Its the same principle as open transformer models where an adapter is used to generate the embedding

However currently the core team focus is in scaling the core text model, as this would be the key performance driver, before adapting multi-modal.

The tech is there, the base model needs to be better

[ Parent | on HN ]