News for hackers. Stuff that matters.
Re:Claude Code Daily Benchmarks for Degradation Tracking by codeflo on Thursday January 29, 2026 @11:21AM (#46812314)
This is intentional? I think delivering lower quality than what was advertised and benchmarked is borderline fraud, but YMMV.
Re:Claude Code Daily Benchmarks for Degradation Tracking by TedDallas on Thursday January 29, 2026 @11:38AM (#46812552)
Per Anthropic’s RCA linked in Ops post for September 2025 issues:

“… To state it plainly: We never reduce model quality due to demand, time of day, or server load. …”

So according to Anthropic they are not tweaking quality setting due to demand.

Re:Claude Code Daily Benchmarks for Degradation Tracking by rootnod3 on Thursday January 29, 2026 @11:44AM (#46812632)
And according to Google, they always delete data if requested.

And according to Meta, they always give you ALL the data they have on you when requested.

Re:Claude Code Daily Benchmarks for Degradation Tracking by groundzeros2015 on Thursday January 29, 2026 @12:19PM (#46813211)
What would you like?
Re:Claude Code Daily Benchmarks for Degradation Tracking by AlexandrB on Thursday January 29, 2026 @12:37PM (#46813489)
An SLA-style contractually binding agreement.
Re:Claude Code Daily Benchmarks for Degradation Tracking by edmundsauto on Thursday January 29, 2026 @02:02PM (#46814861)
I bet this is available in large enterprise agreements. How much are you willing to pay for it?
Re:Claude Code Daily Benchmarks for Degradation Tracking by Onavo on Thursday January 29, 2026 @03:42PM (#46816302)
Priced in.
Re:Claude Code Daily Benchmarks for Degradation Tracking by entropicdrifter on Thursday January 29, 2026 @12:13PM (#46813109)
>And according to Google, they always delete data if requested.

However, the request form is on display in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying ‘Beware of the Leopard'.

Re:Claude Code Daily Benchmarks for Degradation Tracking by cmrdporcupine on Thursday January 29, 2026 @11:58AM (#46812854)
I guess I just don't know how to square that with my actual experiences then.

I've seen sporadic drops in reasoning skills that made me feel like it was January 2025, not 2026 ... inconsistent.

Re:Claude Code Daily Benchmarks for Degradation Tracking by root_axis on Thursday January 29, 2026 @12:29PM (#46813377)
I wouldn't doubt that these companies would deliberately degrade performance to manage load, but it's also true that humans are notoriously terrible at identifying random distributions, even with something as simple as a coin flip. It's very possible that what you view as degradation is just "bad RNG".
Re:Claude Code Daily Benchmarks for Degradation Tracking by cmrdporcupine on Thursday January 29, 2026 @12:31PM (#46813407)
yep stochastic fantastic

these things are by definition hard to reason about

Re:Claude Code Daily Benchmarks for Degradation Tracking by quadrature on Thursday January 29, 2026 @12:40PM (#46813537)
LLMs sample the next token from a conditional probability distribution, the hope is that dumb sequences are less probable but they will just happen naturally.
Re:Claude Code Daily Benchmarks for Degradation Tracking by tempaccount420 on Thursday January 29, 2026 @01:48PM (#46814608)
It's more like the choice between "the" and "a" than "yes" and "no".
Re:Claude Code Daily Benchmarks for Degradation Tracking by mattmanser on Thursday January 29, 2026 @01:57PM (#46814778)
Funny how those probabilities consistently at 2pm UK time when all the Americans come online...
Re:Claude Code Daily Benchmarks for Degradation Tracking by chrisjj on Thursday January 29, 2026 @12:45PM (#46813605)
That's about model quality. Nothing about output quality.
Re:Claude Code Daily Benchmarks for Degradation Tracking by stefan_ on Thursday January 29, 2026 @01:16PM (#46814088)
Thats what is called an "overly specific denial". It sounds more palatable if you say "we deployed a newly quantized model of Opus and here are cherry picked benchmarks to show its the same", and even that they don't announce publicly.
Re:Claude Code Daily Benchmarks for Degradation Tracking by mcny on Thursday January 29, 2026 @11:27AM (#46812411)
Personally, I'd rather get queued up on a long wait time I mean not ridiculously long but I am ok waiting five minutes to get correct it at least more correct responses.

Sure, I'll take a cup of coffee while I wait (:

Re:Claude Code Daily Benchmarks for Degradation Tracking by lurking_swe on Thursday January 29, 2026 @11:38AM (#46812550)
i’d wait any amount of time lol.

at least i would KNOW it’s overloaded and i should use a different model, try again later, or just skip AI assistance for the task altogether.

Re:Claude Code Daily Benchmarks for Degradation Tracking by direwolf20 on Thursday January 29, 2026 @11:46AM (#46812648)
They don't advertise a certain quality. You take what they have or leave it.
Re:Claude Code Daily Benchmarks for Degradation Tracking by denysvitali on Thursday January 29, 2026 @11:27AM (#46812409)
If there's no way to check, then how can you claim it's fraud? :)
Re:Claude Code Daily Benchmarks for Degradation Tracking by chrisjj on Thursday January 29, 2026 @11:26AM (#46812387)
There is no level of quality advertised, as far as I can see.
Re:Claude Code Daily Benchmarks for Degradation Tracking by pseidemann on Thursday January 29, 2026 @01:25PM (#46814238)
What is "level of quality"? Doesn't this apply to any product?
Re:Claude Code Daily Benchmarks for Degradation Tracking by chrisjj on Thursday January 29, 2026 @01:43PM (#46814538)
In this case, it is benchmark performance. See the root post.
Re:Claude Code Daily Benchmarks for Degradation Tracking by bpavuk on Thursday January 29, 2026 @11:28AM (#46812422)
> I think delivering lower quality than what was advertised and benchmarked is borderline fraud

welcome to the Silicon Valley, I guess. everything from Google Search to Uber is fraud. Uber is a classic example of this playbook, even.

Re:Claude Code Daily Benchmarks for Degradation Tracking by copilot_king on Thursday January 29, 2026 @11:33AM (#46812486)
If you aren't defrauding your customers you will be left behind in 2026
Re:Claude Code Daily Benchmarks for Degradation Tracking by rootnod3 on Thursday January 29, 2026 @11:45AM (#46812640)
That number is a sliding window, isn't it?