News for hackers. Stuff that matters.
Re:GPT-5.2 by Stevvo on Thursday December 11, 2025 @03:40PM (#46236849)
The variance is way too high for this test to have any value at all. I ran it 10 times, and each pelican on a bicycle was a better rendition than that, about half of them you could say were perfect.
Re:GPT-5.2 by golly_ned on Thursday December 11, 2025 @04:40PM (#46237560)
Compared to the other benchmarks which are much more gameable, I trust PelicanBikeEval way more.
Re:GPT-5.2 by prodigycorp on Thursday December 11, 2025 @06:48PM (#46239011)
But, to do what exactly?
Re:GPT-5.2 by refulgentis on Thursday December 11, 2025 @07:32PM (#46239406)
What?
Re:GPT-5.2 by getnormality on Friday December 12, 2025 @01:25AM (#46241401)
Well, the variance is itself interesting.