News for hackers. Stuff that matters.

A comment on ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model. Parent comment

Re:ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based... by ronsor on Thursday January 19, 2023 @08:05PM (#34447881)
A few 4090Ti GPUs might be good enough, if they do indeed end up with 48GB of VRAM.
[ Parent | on HN ]
Re:ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based... by firsttimebigboy on Thursday January 19, 2023 @08:18PM (#34447981)
According to wikipedia ChatGPT is built on GPT-3 which has 175B parameters requiring 800GB of memory, so a few 4090s will fall short.
[ Parent | on HN ]
Re:ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based... by samr71 on Thursday January 19, 2023 @11:04PM (#34449100)
Let's say the 4090ti is priced around $2500. 16 of them would set you back just $40 grand. That's quite a bit for the average consumer, but people spend that kinda dough on cars/boats/planes etc. It's well within the realm of dedicated hobbyists. For companies it's nothing.

Someone will come along selling pre-builts for this purpose, if that's not already available for their current research/enterprise cards.

[ Parent | on HN ]