News for hackers. Stuff that matters.
Re:Emmett Shear becomes interim OpenAI CEO as Altman talks b... by bufferoverflow on Monday November 20, 2023 @12:59AM (#38343081)
I thought GPT-4 was not trained on labeled data, but simply on a large volume of text / code. Most of it is publicly accessible: wikipedia, archives of scientific articles, books, github, plus probably purchased data from text-heavy sites like Reddit.
Re:Emmett Shear becomes interim OpenAI CEO as Altman talks b... by frabcus on Monday November 20, 2023 @01:19AM (#38343294)
Whatever they've built this year presumably uses all the positive/negative feedback on ChatGPT that they have a year worth of data now...

Another examples is the Be My Eyes data - presumably the vision part of GPT-4 was trained on the archive of data the blind assistance app has, and that could be an exclusive deal with OpenAI.

Re:Emmett Shear becomes interim OpenAI CEO as Altman talks b... by enigmurl on Monday November 20, 2023 @01:06AM (#38343149)
Assuming it's a reference to RLHF? Not sure
Re:Emmett Shear becomes interim OpenAI CEO as Altman talks b... by lyu07282 on Monday November 20, 2023 @01:14AM (#38343243)
No it's reinforcement learning with human feedback, RLHF lots of labeling