News for hackers. Stuff that matters.
Re:PaLM 2 Technical Report [pdf] by heliophobicdude on Wednesday May 10, 2023 @04:40PM (#35893179)
Humaneval needs careful consideration though.

In the GPT-4 technical report, they reported contamination of humaneval data in the training data.

They did measure against a "non-contaminated" training set but no idea if that can still be trusted.

https://cdn.openai.com/papers/gpt-4.pdf

There are no comments on this comment yet.