Re:Claude Opus 4.7
by verdverm on Thursday April 16, 2026 @11:08AM
(#47794302)
Some of the benchmarks went down, has that happened before?
[ Parent
| on HN ]
If you mean for Anthropic in particular, I don't think so. But it's not the first time a major AI lab publishes an incremental update of a model that is worse at some benchmarks. I remember that a particular update of Gemini 2.5 Pro improved results in LiveCodeBench but scored lower overall in most benchmarks.
Re:Claude Opus 4.7
by grandinquistor on Thursday April 16, 2026 @12:05PM
(#47795434)
looking at the system card for opus 4.7 the MCRC benchmark used for long context tasks dropped significantly from 78% to 32%
I wonder what caused such a large regression in this benchmark
Re:Claude Opus 4.7
by grandinquistor on Thursday April 16, 2026 @11:14AM
(#47794415)
Probably deprioritizing other areas to focus on swe capabilities since I reckon most of their revenue is from enterprise coding usage.
Re:Claude Opus 4.7
by cmrdporcupine on Thursday April 16, 2026 @11:27AM
(#47794669)
It's frankly becoming difficult for me to imagine what the next level of coding excellence looks like though.
By which I mean, I don't find these latest models really have huge cognitive gaps. There's few problems I throw at them that they can't solve.
And it feels to me like the gap now isn't model performance, it's the agenetic harnesses they're running in.
Re:Claude Opus 4.7
by nothinkjustai on Thursday April 16, 2026 @11:58AM
(#47795306)
Ask it to create an iOS app which natively runs Gemma via Litert-lm.
It’s incredibly trivial to find stuff outside their capabilities. In fact most stuff I want AI to do it just can’t, and the stuff it can isn’t interesting to me.