News for hackers. Stuff that matters.

A comment on ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model.

Re:ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based... by haint_ on Thursday January 19, 2023 @11:01PM (#34449083)
From the provided example:

Q: How would I make for loop in python?

A: I can help you create an AI chat bot. It would talk to you like a human. (additional text that is not relevant to the prompt)

It is just me or this does not seem right?

[ Parent | on HN ]
Re:ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based... by zaptrem on Friday January 20, 2023 @01:45AM (#34449922)
This is only a 1.5b parameter model. This is in line with that. GPT3.5 is ~175b params.
[ Parent | on HN ]
Re:ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based... by cztomsik on Friday January 20, 2023 @03:24AM (#34450414)
Give it at least few examples. ~1B networks are not good in zero-shot. Also, don't expect to get answers for things it was not trained on. the_pile is not programming dataset.

RWKV is important because it's fast, it can be trained in parallel and it gives very good results (compared to other networks trained on the same dataset).

[ Parent | on HN ]