News for hackers. Stuff that matters.

A comment on Paving the way to efficient architectures: StripedHyena-7B. Parent comment

Re:Paving the way to efficient architectures: StripedHyena-7B by refulgentis on Friday December 08, 2023 @06:39PM (#38576393)
There are RWKV numbers in there, so you sort of got your wish sideways. :)

Has RWKV not released anything until now? I thought it was an open project that was in use, if sort of by a hipster 1%

[ Parent | on HN ]
Re:Paving the way to efficient architectures: StripedHyena-7B by stellaathena on Friday December 08, 2023 @07:34PM (#38576910)
Paper: https://arxiv.org/abs/2305.13048

Models (v4 are the ones from the paper): https://huggingface.co/RWKV

GitHub: https://github.com/BlinkDL/RWKV-LM

[ Parent | on HN ]
Re:Paving the way to efficient architectures: StripedHyena-7B by senseiV on Friday December 08, 2023 @08:19PM (#38577287)
V5 7b is out, close to hyena, gets 1400 t/s on a 3090, while an h100 llama 7b 8bit is 1200 t/s
[ Parent | on HN ]