Depends on how they're applied. I've had success using LLama and while we check to see if OpenAI or Google's Gemini would give us any noticeable improvement, it really doesn't for our use case.

While certainly newer models are more capable on the whole, it doesn't mean I need all that capability to accomplish the business goal.

This is kind of a useless statement. If your use case is so easy that "old" models work for it then obviously you won't care about or be following the latest developments but its just not accurate to say that Deepseek R1 is equivalent to o3 or Gemini 2.5.

Producing quality results is not the same thing as saying Deepseek R1 is the equivalent of o3 or Gemini 2.5

Again, its not about capabilities alone, (on this, many models lag behind, I already said as much). I follow these developments quite closely, and I purposely said results as to not say they're equivalent in capability. They aren't.

However, if a business is getting acceptable results from older models or cheaper models than capability doesn't matter, the results do. Gemini 2.5 can be best of breed but why switch if it shows no meaningful improvement in results for the business?

If I need more capability or results are substandard, I can always upgrade, but its like saying there's no room for cheaper processors and you'd be out of your mind not to be using only the latest at all times no matter the results.

That's not what GP was saying though. To stay with that analogy, the assertion was that "all processors are kinda the same, there's no real qualitative difference", which sounds pretty strange. It's somewhat accurate if your use-case is covered by the average processor and the faster one doesn't benefit you. They're not equal, but all of them surpass your needs.

> If I need more capability or results are substandard, I can always upgrade

You wouldn't be able to upgrade (and see improved results) if the model you use today was close to equal to the top of the line.

That wasn’t the assertion. The results - not the models themselves, not strictly speaking their over all capabilities - if they have no meaningful improvement by moving to a newer model, why then would I want to switch if I’m not getting any tangible improvement in results?

> The big companies in this space doing the research are not making leap over leap with each release, and the downstream open source projects are coming closer to the same quality or in fact can produce the same quality (e.g DeepSeek or LLAMA) hence why it’s becoming a commodity.

This was the assertion. "Open source is close/equal in quality", not "open source is enough for plenty of use-cases, not everyone needs the top of the line".

Your belief that O3 and O4 are that superior to open source models comes from the fact that models are often using shit, garbage, trash samplers like top_p and top_k.

Switch them to good samplers and write the tool calling code to allow tool calls in the reasoning chain and you’ll see close to parity in performance.

The remaining advantages left to closed source come from better long context, and later data cutoff points.

If you don’t believe me let’s see the receipts of your ICLR or NeurIPS publications - otherwise sit down and listen to your elders.