News for hackers. Stuff that matters.

A comment on GPT-5.5. Parent comment

Re:GPT-5.5 by XCSme on Thursday April 23, 2026 @03:51PM (#47880833)
They mentioned in their release page, that the Claude team noticed memorization of the SWE-bench test, so the test is actually in the training data.

Here: https://www.anthropic.com/news/claude-opus-4-7#:~:text=memor...

[ Parent | on HN ]
Re:GPT-5.5 by William_BB on Friday April 24, 2026 @02:44AM (#47886485)
Good luck arguing with SWE benchmark purists
[ Parent | on HN ]
Re:GPT-5.5 by sigmoid10 on Friday April 24, 2026 @11:28AM (#47891610)
Any static benchmark older than 12-18 months is basically worthless, because the content will have spread all over the internet and have found its way into the latest model's training set.
[ Parent | on HN ]