News for hackers. Stuff that matters.
Re:Claude Opus 4.6 by tedsanders on Thursday February 05, 2026 @03:09PM (#46904493)
We don't vary our model quality with time of day or load (beyond negligible non-determinism). It's the same weights all day long with no quantization or other gimmicks. They can get slower under heavy load, though.

(I'm from OpenAI.)

Re:Claude Opus 4.6 by Someone1234 on Thursday February 05, 2026 @03:29PM (#46904759)
Specifically including routing (i.e. which model you route to based on load/ToD)?

PS - I appreciate you coming here and commenting!

Re:Claude Opus 4.6 by hhh on Thursday February 05, 2026 @03:32PM (#46904798)
There is no routing with API, or when you choose a specific model in chatGPT.
Re:Claude Opus 4.6 by zwaps on Friday February 06, 2026 @01:09AM (#46909670)
In the past it seemed there was routing based on context-length. So the model was always the same, but optimized for different lengths. Is this still the case?
Re:Claude Opus 4.6 by Trufa on Thursday February 05, 2026 @03:18PM (#46904616)
Can you be more specific than this? does it vary in time from launch of a model to the next few months, beyond tinkering and optimization?
Re:Claude Opus 4.6 by tedsanders on Thursday February 05, 2026 @04:11PM (#46905388)
Yeah, happy to be more specific. No intention of making any technically true but misleading statements.

The following are true:

- In our API, we don't change model weights or model behavior over time (e.g., by time of day, or weeks/months after release)

- Tiny caveats include: there is a bit of non-determinism in batched non-associative math, bugs or API downtime can obviously change behavior, heavy load can slow down speeds, and this of course doesn't apply to the 'unpinned' models that are clearly supposed to change over time (e.g., xxx-latest). But we don't do any quantization or routing gimmicks that would change model weights.

- In ChatGPT and Codex CLI, model behavior can change over time (e.g., we might change a tool, update a system prompt, tweak default thinking time, run an A/B test, or ship other updates); we try to be transparent with our changelogs (listed below) but to be honest not 100% of changes are logged. But even here we're not doing any gimmicks to cut quality by time of day or intentionally dumb down models after launch. Model behavior can change though.

ChatGPT release notes: https://help.openai.com/en/articles/6825453-chatgpt-release-...

Codex changelog: https://developers.openai.com/codex/changelog/

Codex CLI commit history: https://github.com/openai/codex/commits/main/

Re:Claude Opus 4.6 by ComplexSystems on Thursday February 05, 2026 @04:22PM (#46905541)
Do you ever replace ChatGPT models with cheaper, distilled, quantized, etc ones to save cost?
Re:Claude Opus 4.6 by jghn on Thursday February 05, 2026 @04:28PM (#46905636)
He literally said no to this in his GP post
Re:Claude Opus 4.6 by tedsanders on Friday February 06, 2026 @01:52AM (#46909905)
We do care about cost, of course. If money didn't matter, everyone would get infinite rate limits, 10M context windows, and free subscriptions. So if we make new models more efficient without nerfing them, that's great. And that's generally what's happened over the past few years. If you look at GPT-4 (from 2023), it was far less efficient than today's models, which meant it had slower latency, lower rate limits, and tiny context windows (I think it might have been like 4K originally, which sounds insanely low now). Today, GPT-5 Thinking is way more efficient than GPT-4 was, but it's also way more useful and way more reliable. So we're big fans of efficiency as long as it doesn't nerf the utility of the models. The more efficient the models are, the more we can crank up speeds and rate limits and context windows.

That said, there are definitely cases where we intentionally trade off intelligence for greater efficiency. For example, we never made GPT-4.5 the default model in ChatGPT, even though it was an awesome model at writing and other tasks, because it was quite costly to serve and the juice wasn't worth the squeeze for the average person (no one wants to get rate limited after 10 messages). A second example: in our API, we intentionally serve dumber mini and nano models for developers who prioritize speed and cost. A third example: we recently reduced the default thinking times in ChatGPT to speed up the times that people were having to wait for answers, which in a sense is a bit of a nerf, though this decision was purely about listening to feedback to make ChatGPT better and had nothing to do with cost (and for the people who want longer thinking times, they can still manually select Extended/Heavy).

I'm not going to comment on the specific techniques used to make GPT-5 so much more efficient than GPT-4, but I will say that we don't do any gimmicks like nerfing by time of day or nerfing after launch. And when we do make newer models more efficient than older models, it mostly gets returned to people in the form of better speeds, rate limits, context windows, and new features.

Re:Claude Opus 4.6 by acuozzo on Saturday February 07, 2026 @07:12PM (#46929813)
> we never made GPT-4.5 the default model in ChatGPT

Just wondering: Why was it never made available via API? You can just charge whatever per token to make sure it's profitable like o1-pro.

I use it via my ChatGPT-Pro subscription, but I still find the API omission weird.

Re:Claude Opus 4.6 by tedsanders on Sunday February 08, 2026 @12:16PM (#46936333)
It was available in the API from Feb 2025 to July 2025, I believe. There's probably another world where we could have kept it around longer, but there's a surprising amount of fixed cost in maintaining / optimizing / serving models, so we made the call to focus our resources on accelerating the next gen instead. A bit of a bummer, as it had some unique qualities.
Re:Claude Opus 4.6 by jychang on Thursday February 05, 2026 @04:30PM (#46905649)
Re:Claude Opus 4.6 by tedsanders on Thursday February 05, 2026 @04:43PM (#46905795)
Yep, we recently sped up default thinking times in ChatGPT, as now documented in the release notes: https://help.openai.com/en/articles/6825453-chatgpt-release-...

The intention was purely making the product experience better, based on common feedback from people (including myself) that wait times were too long. Cost was not a goal here.

If you still want the higher reliability of longer thinking times, that option is not gone. You can manually select Extended (or Heavy, if you're a Pro user). It's the same as at launch (though we did inadvertently drop it last month and restored it yesterday after Tibor and others pointed it out).

Re:Claude Opus 4.6 by tgrowazay on Thursday February 05, 2026 @04:40PM (#46905769)
Isn’t that just how many steps at most a reasoning model should do?
Re:Claude Opus 4.6 by Trufa on Thursday February 05, 2026 @06:37PM (#46907018)
I ask then unironically then, am I imagining that models are great when they start and degrade over time?

I've had this perceived experience so many times, and while of course it's almost impossible to be objective about this, it just seem so in your face.

I don't discard being novelty plus getting used to it, plus psychological factors, do you have any takes on this?

Re:Claude Opus 4.6 by jason_oster on Thursday February 05, 2026 @08:21PM (#46907806)
You might be susceptible to the honeymoon effect. If you have ever felt a dopamine rush when learning a new programming language or framework, this might be a good indication.

Once the honeymoon wears off, the tool is the same, but you get less satisfaction from it.

Just a guess! Not trying to psychoanalyze anyone.

Re:Claude Opus 4.6 by wasmainiac on Friday February 06, 2026 @04:16AM (#46910698)
I don’t think so. I notice the same thing, but I just use it like google most of the time, a service that used to be good. I’m not getting a dopamine rush off this, it’s just part of my day.
Re:Claude Opus 4.6 by newswasboring on Friday February 06, 2026 @04:53AM (#46910938)
>there is a bit of non-determinism in batched non-associative math that can vary by batch / hardware

Maybe a dumb question but does this mean model quality may vary based on which hardware your request gets routed to?

Re:Claude Opus 4.6 by qingcharles on Friday February 06, 2026 @12:33PM (#46915705)
Thank you for saying this publically.

I feel like you need to be making a bigger statement about this. If you go onto various parts of the Net (Reddit, the bird site etc) half the posts about AI are seemingly conspiracy theories that AI companies are watering down their products after release week.

Re:Claude Opus 4.6 by joshvm on Thursday February 05, 2026 @03:58PM (#46905212)
My gut feeling is that performance is more heavily affected by harnesses which get updated frequently. This would explain why people feel that Claude is sometimes more stupid - that's actually accurate phrasing, because Sonnet is probably unchanged. Unless Anthropic also makes small A/B adjustments to weights and technically claims they don't do dynamic degradation/quantization based on load. Either way, both affect the quality of your responses.

It's worth checking different versions of Claude Code, and updating your tools if you don't do it automatically. Also run the same prompts through VS Code, Cursor, Claude Code in terminal, etc. You can get very different model responses based on the system prompt, what context is passed via the harness, how the rules are loaded and all sorts of minor tweaks.

If you make raw API calls and see behavioural changes over time, that would be another concern.

Re:Claude Opus 4.6 by zamadatix on Thursday February 05, 2026 @04:43PM (#46905796)
I appreciate you taking the time to respond to these kinds of questions the last few days.
Re:Claude Opus 4.6 by wasmainiac on Thursday February 05, 2026 @05:50PM (#46906548)
Thanks for the response, I appreciate it. I do notice variation in quality throughout the day. I use it primarily for searching documentation since it’s faster than google in most case, often it is on point, but also it seems off at times, inaccurate or shallow maybe. In some cases I just end the session.
Re:Claude Opus 4.6 by nl on Thursday February 05, 2026 @06:02PM (#46906667)
Usually I find this kind of variation is due to context management.

Accuracy can decreases at large context sizes. OpenAI's compaction handles this better than anyone else, but it's still an issue.

If you are seeing this kind of thing start a new chat and re-run the same query. You'll usually see an improvement.

Re:Claude Opus 4.6 by repeekad on Thursday February 05, 2026 @08:38PM (#46907915)
This is called context rot
Re:Claude Opus 4.6 by charcircuit on Friday February 06, 2026 @01:04AM (#46909650)
I thought context rot was only for long distance queries.
Re:Claude Opus 4.6 by wasmainiac on Friday February 06, 2026 @01:14AM (#46909689)
I don't think so. I am aware that large contexts impacts performance. In long chats an old topic will someone be brought up in new responses, and the direction of the mode is not as focused.

Regardless I tend to use new chats often.

Re:Claude Opus 4.6 by fragmede on Thursday February 05, 2026 @06:25PM (#46906906)
I believe you when you say you're not changing the model file loaded onto the H100s or whatever, but there's something going on, beyond just being slower, when the GPUs are heavily loaded.
Re:Claude Opus 4.6 by clbrmbr on Thursday February 05, 2026 @07:27PM (#46907435)
I do wonder about reasoning effort.
Re:Claude Opus 4.6 by hauntsaninja on Friday February 06, 2026 @03:40AM (#46910479)
Reasoning effort is denominated in tokens, not time, so no difference beyond slowness at heavy load

(I work at OpenAI)

Re:Claude Opus 4.6 by derwiki on Thursday February 05, 2026 @07:07PM (#46907274)
Has this always been the case?
Re:Claude Opus 4.6 by GorbachevyChase on Thursday February 05, 2026 @08:35PM (#46907902)
Hi Ted. I think that language models are great, and they’ve enabled me to do passion projects I never would have attempted before. I just want to say thanks.
Re:Claude Opus 4.6 by robertclaus on Friday February 06, 2026 @01:11AM (#46909678)
Hi Ted! Small world to see you here!
Re:Claude Opus 4.6 by a456463 on Friday February 06, 2026 @11:32AM (#46914948)
sure. we believe you
Re:Claude Opus 4.6 by smugtrain on Friday February 06, 2026 @07:56PM (#46920198)
It will give the user lower quality if it finds them “distressed” however, choosing paternalistic safety over epistemic accuracy. As a user gets more frustrated with the system, it will pick up the distress signal even more so, a kind of feedback loop toward degraded service quality. In my experience.