They basically dynamically allocate compute into this pool from the general pool to meet demand and SLAs. If demand for priority processing spikes latency in the general pool spikes.
Almost all subscriptions use general pool. That's why it gets really slow sometimes. Unless you use "fast" which acts like priority processing and is super expensive.
So that's part of how they balance compute based on demand and fund the "excess" capacity.
Im asking on how lower compute could translate to worse model performance. That's the implicit claim being made when people are saying Anthropic nerfs a model (result is bad, not necessarily slow). The only way I can think that could happen is if they actually swapped Opus 5.5 out for a worse model that takes less resources to do inference.
There is no proof, the frontier labs deny it, and certainly it's not happening on hosts like Azure, Amazon, and etc.. The claim is that it's "trivial" to detect observation, so they serve the normal model if you try to measure it..
Basically the conspricy theory is:
* OpenAI/Anthropic nerf their models regularly to reduce compute demand
* It's impossible to measure and thus prove
However, how would less compute result in degraded performance as opposed to more latency? Like is amount of compute something you can actually tune to control for latency? Wouldn't that imply they are actually serving requests in other models? If I run a very large model locally on a shit computer vs. something maxed out, it's the latency that differs.