News for hackers. Stuff that matters.

A comment on Study shows over time, ChatGPT lost accuracy. Parent comment

Re:Study shows over time, ChatGPT lost accuracy by mov_eax_ecx on Tuesday September 05, 2023 @12:28PM (#37394087)
How can i locate this study?. I think you are misrepresenting something.

In the gpt4 paper they specifically address this, and find that "Averaged across all exams, the base model achieves a score of 73.7% while the RLHF model achieves a score of 74.0%, suggesting that post-training does not substantially alter base model capability."

[ Parent | on HN ]
Re:Study shows over time, ChatGPT lost accuracy by nicce on Tuesday September 05, 2023 @12:30PM (#37394131)
The problem with these studies is that we really still don’t know. Nobody can replicate the papers of OpenAI.
[ Parent | on HN ]
Re:Study shows over time, ChatGPT lost accuracy by adamsb6 on Tuesday September 05, 2023 @12:37PM (#37394276)
Given the homogeneity of responses on taboo subjects, there's probably something exogenous to the model at work.
[ Parent | on HN ]
Re:Study shows over time, ChatGPT lost accuracy by galaxytachyon on Tuesday September 05, 2023 @12:52PM (#37394533)
Found it, it is a pretty recent paper.

https://arxiv.org/pdf/2308.13449.pdf

[ Parent | on HN ]