Claude Sonnet 4.6

Re:Claude Sonnet 4.6 by deanc on Tuesday February 17, 2026 @01:24PM (#47050999)

I really don't get these companies posting disingenuous benchmarks. Every time, they pick and choose who to compare against. Not comparing to the latest 5.3-codex is absurd when it's been out a couple of weeks now. Who are they trying to kid?

[ Parent | on HN ]

Re:Claude Sonnet 4.6 by tomlis on Tuesday February 17, 2026 @03:55PM (#47053191)

gpt-5.3-codex isn't available via the API yet. Pretty sure they were only testing via API access.

[ Parent | on HN ]

Re:Claude Sonnet 4.6 by falloon on Tuesday February 17, 2026 @01:31PM (#47051132)

If you were writing a promotional post for your new model, would you include benchmarks of a competitor that's spanking you across the board? This is marketing.

[ Parent | on HN ]

Re:Claude Sonnet 4.6 by AdamConwayIE on Tuesday February 17, 2026 @01:27PM (#47051075)

There aren't really any of the typical benchmark suites targeting Codex 5.3 because it's still not in the API.

SWE bench for example creates a predictions file and evaluates the results in the harness. Without Codex 5.3 being in the API, it can't.

[ Parent | on HN ]

Re:Claude Sonnet 4.6 by rvz on Tuesday February 17, 2026 @01:28PM (#47051091)

> Who are they trying to kid?

People who do not know how reproducible research works.

Any benchmark that is presented by AI labs must be reproduced reliably by someone else independent of that AI lab presenting these results.

Otherwise, not only it is biased, these numbers can be just made up for marketing purposes.

[ Parent | on HN ]