Re:GPT-4.1 in the API
by asdev on Monday April 14, 2025 @01:30PM
(#43683829)
it's worse than 4.5 on nearly every benchmark. just an incremental improvement. AI is slowing down
[ Parent
| on HN ]
They don't disclose parameter counts so it's hard to say exactly how far apart they are in terms of size, but based on the pricing it seems like a pretty wild comparison, with one being an attempt at an ultra-massive SOTA model and one being a model scaled down for efficiency and probably distilled from the big one. The way they're presented as version numbers is business nonsense which obscures a lot about what's going on.
Or OpenAI is? After using Gemini 2.5, I did not feel "AI is slowing down". It's just this model isn't SOTA.
It's like 30x cheaper though. Probably just distilled 4.5
Re:GPT-4.1 in the API
by HDThoreaun on Monday April 14, 2025 @03:29PM
(#43685246)
Maybe progress is slowing down but after using gemini 2.5 there clearly is still a lot being made.
Re:GPT-4.1 in the API
by simianwords on Monday April 14, 2025 @01:58PM
(#43684214)
Sorry what is the source for this?
It's better on AIME '24, Multilingual MMLU, SWE-bench, Aider’s polyglot, MMMU, ComplexFuncBench while being much much cheaper and smaller.
and it's worse on just as many benchmarks by a significant amount. as a consumer I don't care about cheapness, I want the maximum accuracy and performance