Why?

Because we see the products they release even after the breathless "we do 30000 PRs per day" marketing posts.

No, the labs' models are not light-years ahead. That's what labs claim with literally no independent verification of these claims. They want you (and investors, and politicians) to believe these models are light-years ahead.

What they do have is unlimited and unrestricted access to their own models.

I dunno. They sat on GPT-4 for 8 months before actually releasing it and that was a major stride ahead of 3. It's kind of silly not to realize they do indeed have access to models months ahead of the public at minimum. Nothing you have access to is the true frontier. It only takes some common sense to acknowledge that.

> OpenAI sat on GPT-4 for 8 months before actually releasing it and that was a major stride over 3.

OpenAI said they sat on it for 8 months

> It's kind of silly not to realize they do indeed have access to models months ahead of the public at minimum

Yes, they do. Nothing in this sentence implies that what they have access to is "light years ahead of the public models".

>OpenAI said they sat on it for 8 months

They sat in it. We had concrete details('rumors' that turned out surprisingly accurate) months before the public release.

>Yes, they do. Nothing in this sentence implies that what they have access to is "light years ahead of the public models".

The accomplishments of those internal models we're priven to should be enough to give you an indication. You can fire up as many Astra agents as you like you're not going to get the performance of Open AI's internal model in math. Just as you can throw as much compute and agents at GPT-4 and you're not going to get anything resembling Opus 5.5.

Again, this is all common sense. That you refuse to pull your head out of the sand doesn't mean it's unbelievable. You just don't want to believe it.

> They sat in it. We had concrete details ('rumors' that turned out surprisingly accurate) months before the public release.

So all you had was rumours then "confirmed" by OpenAI marketing

> "Light years" is subjective

Yes. And this subjective description invokes a certain image that I objected to.

> but the accomplishments of those internal models we're proven to should be enough to give you some indication.

Yes. The indication is: there are some improvements, with diminishing returns. A lot of perceived improvements now come from tooling, harnesses etc.

And the people working at the labs have access to unrestricted models and unlimited compute that will not flag you for "bio research" and will not stop running after 5 hours, which adds to the marketing myths.

> You can fire up as many Astra agents as you like you're not going to get the performance of Open AI's internal model in math.

Of course I won't. I don't have the same access.

Doesn't mean any of their models are light years ahead. And as for maths, again, you literally have no idea what they were actually doing besides their marketing materials. But even their marketing materials let slip some details that amount to "commandeered a full cluster for 20 000 hours and let it rip"

No it wasn't. GPT-4 was indistinguishable from 3.5.

For the same reason as a child I was sceptical towards claims along the lines of 'my girlfriend goes to another school out of town'. If it can't be externally verified to a high degree of confidence, we should employ a high degree of scepticism towards extraordinary claims.

It's really easy to understand, what they have are the same model with less restrictions and gaurdrails, they also have infinite amount of usage and no worry about cost, they can burn a million dollars for an article in the news.