News for hackers. Stuff that matters.
Re:RWKV RNN: Better than ChatGPT? by gaogao on Thursday March 23, 2023 @08:36PM (#35283555)
Yeah, my sense is that OpenAI moat is primarily just through the RLHF dataset right now. Most of the other things – foundational models and datasets, embeddings (roughly the plugins announcement today) – have generally already been commoditized. Just getting that dataset for fine tuning is one of the last major hurdles for the ChatGPT geist.
Re:RWKV RNN: Better than ChatGPT? by sdrinf on Friday March 24, 2023 @12:07AM (#35284972)
The Open Assistant community ( https://open-assistant.io/ ) is building a crowdsourced dataset for RLHF, with apparently high quantity of high quality contributions.