News for hackers. Stuff that matters.
Re:There is minimal downside to switching to open models by LPisGood on Monday June 22, 2026 @01:02AM (#48625901)
People talk about this a lot. What I have never seen is a discussion of methods they might employ to degrade the models.

Let’s say I’m a bad faith LLM operator, and I want to degrade my model so the next release looks better and people want to switch to the more expensive one. How would I do that?

Re:There is minimal downside to switching to open models by nessex on Monday June 22, 2026 @01:38AM (#48626136)
They would quantize the model. That'd make it cheaper to run, and have slightly worse output but it would still generate outputs with a similar feel, derived from a compressed version of the same knowledge base etc.

They wouldn't even need to do this uniformly, quantized versions of the model could be routed only a subset of the requests. They could do this to nerf the old model, or more likely just to give themselves more hardware to run the new one on by handling more requests on less hardware. Or to handle increased request volume as traffic ramps up faster than hardware can be provisioned.

Playing with local models at various quants, the degradation can be hard to spot. Sometimes it's only noticeable in aggregate. And even then, you never really know if you just got unlucky with a bad response due to RNG.

I've had Opus 4.6 fall into some weirdly incoherent loops that I rarely see from even Sonnet, that felt like the kind of thing I got frequently with Qwen3.5 9B on local. And the above applies... Was that just bad RNG? Or was my request to Opus routed to some lower quality variant? There's no great way for me to tell for any given request, nor any way to guarantee Anthropic _didn't_ do that.

Re:There is minimal downside to switching to open models by OccamsMirror on Monday June 22, 2026 @03:12AM (#48626797)
I have had the same experiences you've had with 4.6 and it was ever since they brought out 4.7. It's fairly obvious they're doing something like you've said here.
Re:There is minimal downside to switching to open models by nessex on Monday June 22, 2026 @03:32AM (#48626949)
Forgot to mention, but it was after the 4.7 release when I was still using 4.6 that I saw those loops too... Before that, 4.6 had been a pretty seamless experience.
Re:There is minimal downside to switching to open models by tsss on Monday June 22, 2026 @07:38AM (#48628808)
And guess what all the providers of open models do: They quantize, badly.
Re:There is minimal downside to switching to open models by csunbird on Monday June 22, 2026 @08:15AM (#48629116)
This is why you pay premium for trusted providers, who are verified to not quantize
Re:There is minimal downside to switching to open models by maybe_pablo on Monday June 22, 2026 @01:35AM (#48626113)
Weight quantization, n-expert capping, routing to smaller model, context window truncation, aggressive sampling constraints, lossy speculative decoding and probably more.
Re:There is minimal downside to switching to open models by alfiedotwtf on Monday June 22, 2026 @03:51AM (#48627081)
I'm pretty sure you could do n-expert capping on any MoE model with only a handful lines of changes to ik_llama.cpp, but yeah... my bet is the have various quantisations and run the lower ones at peak (along with different system prompts i.e we're GPU-bound right now. Get to the point with less chatter)
Re:There is minimal downside to switching to open models by trollbridge on Monday June 22, 2026 @06:48AM (#48628432)
I can't prove any of it, but it sure feels like that happens sometimes on Anthropic's platform.

I don't seem to get any of this with GPT-5.5 or GPT-5.5-Pro (not that I use 5.5-Pro enough to know for sure, but when I do use it, it never seems nerfed).

Re:There is minimal downside to switching to open models by Tepix on Monday June 22, 2026 @01:35AM (#48626115)
Use quantisation.