> I noticed this too as did many others. Had to bump my default reasoning but it’s not the same. Seems to often happen before new model drops

I heard that Qwen4 was dropping some time soon, I wonder if Anthropic is trying to announce a new model immediately after Qwen4 - both for their IPO, and to kill the competition.

Of all the reasons to look at local inference, this is the one which pushes me in that direction the most. I want to rely on a model for consistency in my workflows. If I wake up on Monday and suddenly it's refactoring things it built on Friday, that's a huge problem. That makes it unreliable.

Probably because those "nerfs" do not show on sites that track model performance over time. Sometimes Anthropic/OpenAI mess stuff up in their harness, but usually not in the models themselves.

In my own work I have not once noticed a degradation where I could have said with any certainty that it's not within usual deviation or me having a bad day.

Conspicuously the model seems to be especially dumb and annoying when I did not get a lot of sleep the previous night.

When OpenAI and Anthropic engineers post here on HN that they do not quantize models and silently replace checkpoints in the API, I believe them.

I saw nothing that convinced me the people who are constantly making these complaints don't just mistake any suboptimal result for changes in the model.

Everyone really should keep a few tasks around that are indicative of the work they do and run them a few times with the same model and a few times across model families and when new models drop.

It’s extremely informative to see with your own eyes the variance that always exists. Even if you can’t afford statistical relevance it’s valuable.

Specifically

> YOU get to decide when the model changes

I run it on a 6*rtx3090 consumer PC (no threadripper etc sadly) with very constrained pcie topology (4 cards sit on 2 lanes basically) and i still get 7k prefill and 50t/s single user or 200t/s 7 users. All at full 262k contexts of course and with 96GB of ram kv cache that means I can easily have 10+ Claude code sessions open with 1-7 generating at any time.

I've been the person that said, before opus 4.6 came out that this AI stuff is absolute crap for programming. Then it suddenly wasn't and I was using it many hours each day.

Local AI was the same. It was all crap. Models like GLM5.3 Flash that are many times larger than that qwen, lots of people are oooing and ahhing over I found way too inferior. Previous DeepSeek V4.1 Flash 0731 was atrocious. They benchmaxed it to hell, but at least it was small.

The current model of same name but release 1017 and double the size is currently the best open source model IMO. It even beats my self hosted Qwen3.8-flash-next.

I've tried hosting that DeepSeek model, but even adding another rtx3090 for a total of 7 cards and my 192gb of ram bought when it cost under $1000 I get only about 25t/s decode and few hundred prefill. That is far too slow for me to use.

But, I believe once this pyramid scheme of circular financing finally bursts there will be a glut of hardware on the market where people like me will be able to buy equipment to run 750B models like we now run 120B.

Also, how much power does that consume at full tilt ?

Truly insane times

That's around 1% of the price of 8x B200.

Tonight has been maddening with the kinds of absurd mistakes it's repeatedly making. It's so frustrating.

I'm the kind of person that wants something to be consistent and reliable. And Anthropic has been great, except for in this way. Which is enough of a recurring frustration (about every couple months this year) that I genuinely can't wait for other options to catch up. Then I'll be leaving as soon as I can.

I just don't know how long that will be.

Totally possible that I'm just hallucinating/projecting/etc or my workflows somehow changes in terms of difficulty, but it's been kind of shocking.

Now it's like talking to my hungover self, after 3 days bender, who just wanted to be left alone.

That's exactly why they did it. Nerf Opus 5.5, then drop Fable 5.5 just before IPO so it looks better next to its weaker predecessor.

And people wonder why open models are a big deal, and why some people love them so much, despite being 30% worse than the frontier.

Having control over something that writes most of your code is important.

Only local AI capable hardware and frontier open weight models can provide that.

So I'm AI pilled and in another career, and started to think, what if the problem is the inputs? They release Opus X.X and at first it's a rush of their best users, best use cases, upgraded lanes. As weeks go on, adoption increases, cruft uses start to poison the model? Is this how the ongoing evolution works? I'd love to get Sonnet 5.5 and Opus for review in a place where I could reliably daily drive for X months / Y years etc.

Surely everyone is that kind of person.

Who wants to pay for a service that doesn't work properly half the time?

"More reasoning" != "better"

This is the #1 thing i HATE with LLMs. They are not predictable like a compiler is.

There is random input to make the models evaluate differently every time, but without that (or if you fix it) then the model will always give the same output with the same input.

There's no somehow to it, it's actually surprisingly hard to make a computer do something in a non-deterministic fashion.

I have no doubt they will ensure their users get an accurate picture of Opus 5.5 within hours.

I *do* wish we wouldn't let software companies call this "A/B testing". That's not what that term means to a typical reader. In any other market than software, you'd call it something like "misleading consumers in respect to product composition" and "an unfair commercial practice". That's verbiage from the EU "dual quality" food scandal, where transnational food brands (Coca-Cola was one) adulterated their recipes to use cheaper ingredients in select markets, though the external packaging was kept the same.

https://ec.europa.eu/commission/presscorner/detail/en/qanda_...

(I suppose Coca-Cola's CEO would prefer we talk about what they did as "A/B testing").

https://github.com/ninjahawk/livenerf

I wonder as well whether would this pop up on nerfbench

I don't think that's an excuse, but it is likely the reasoning.

https://news.ycombinator.com/newsguidelines.html

The model itself should not be substituted(for example, quantized) without an explicit SKU change.

Pretty much the only alternative is loading up some credits and using Fable.

Both get the job done these days. I still hate Opus 5.5’s output, 6.1 Sol does an acceptable job of doing a second cleanup pass to it.

1)- Get a coding task and get the output of Opus and Astra.

2)- Open a new session and ask each other to review in detail the solution from the other model without mentioning where is it coming from.

3)- After the review, ask the original model to comment on it and state which solution is better and if the criticisms are valid...all in a new session.

4)- At the end Opus and others from Anthropic always end up agreeing the OpenAI model solution is better. My human review agrees.

After doing this for a few weeks...and consistently having their models rating higher the OpenAI solutions...I cancelled the Anthropic subscription...

DeepSeek, Qwen.

I tried $20 of usage credits as a test once the limit was reached.

I shit you not, it lasted for less than 5 minutes.

I get 5x more done (of equivalent quality) on my opencode go + DeepSeek flash sub for $40, than $200 anthropic sub.

I have been using DeepSeek flash for a few projects and it is actually crazy capable, And I don't think I have spent $20 via api in a month yet.

A bit like "You can order the steak for $75, or you can have whatever the chef is making for $30"

And with Strata running at 50tok/s on my desktop is all I need. Would I like faster? Sure! But it's super useful in all sorts of tasks now.

Flash Next is really great, but you still need pretty decent hardware. A DGX Spark, Strix Halo, or Mac with a lot of memory. You can use things like Strata to squeeze it into a system with only 32GB of VRAM by offloading a lot of it to system memory and still get something that "works", but the quality falls off a cliff too. At that point you get better results out of Qwen 3.8 27b in int4.

Never go below a four bit quant if you want good results. A worse quant of a bigger model is often easily beaten by the small model at decent quality. A lot of the people who think local AI suck think that because they don't understand that low quants == dumbass model. You aren't just compressing data, you're reducing the amount of compute it uses which effectively lowers it's IQ.

If you want to run something actually useful for cheap Qwen 3.8 27b on a 32GB GPU in int4 should be passable. It's more like Sonnet for a year ago. To match last years Opus you really need at least 128Gb.

Personally I have two Sparks connected with ConnectX 7. Now I can run GLM 5.3 locally at 30-40 tps and it's honestly been great. It really cut my dependence on cloud providers and I can run an abliterated version that never refuses my Cyber related tasks.

Folks like to think of 'weights' as just storage, but they aren't (except with stuff like engrams). Those digits of precision encode actual computation, not just storage. There is always a cost, it's lossy compression after all. It's like MP3 vs AAC. If you can't hear the difference it just means you haven't encountered it YET.

That's not to say that you ever will. The cost could be in weird far out edge cases, but whatever it once encoded... It's gone now and it ain't coming back.

No longer am I constantly made to feel like Ingrid Bergman in that movie with all the lights.

How are they making money in your view?

What you said doesn't yield any information whatsoever about whether or not subscriptions are profitable, all that says is that subscriptions are less profitable than API.

Things are able to be less profitable than other things while still being profitable.

If something like the API is very very profitable, it's even possible for profitable things to be much less profitable!

https://x.com/tokengremlin/status/2119783808773841101

i've seen people on social media claiming all sorts of things about Fable 5.5, Opus 5.6 and Sol 6.2 and what-not for the past few weeks, but it just seems like engagement farming with no proof

That would be the less bad reason.

Under load requests queue and take longer.

If you got routed to a TPU instance at some point in your chat another bug kept you there.

Earlier this year in 2026 they decided to save usage by stripping out all thinking tokens if you left your chat idle for one hour (cache timeout) and then came back to it, causing massive degradations. They fixed that, but recently they switched to doing auto-compactions for long running chats in anticipation of them going stale in cache. I think that one is usually what you want anyway and there is at least a way to recover to the uncompacted chat if you really need it, but it might also be a source of some degradation people are seeing now.

But the model NERF conspiracies are bunk.

But the harness one was not just a random harness regression; they were getting a huge influx of users and pretty clearly did it to save capacity. Not a great thing when you might be relying on attention to those thinking traces to reduce the risk of prompt injection etc. and it could have gone beyond major task degradation into a safety/security issue.

Through the lens of a conspiracy theory this could be viewed narrowly as "cheating" their users to reduce load.

But there are other motivations for the constant work going on in the harnesses that paint a fuller picture.

However, how would less compute result in degraded performance as opposed to more latency? Like is amount of compute something you can actually tune to control for latency? Wouldn't that imply they are actually serving requests in other models? If I run a very large model locally on a shit computer vs. something maxed out, it's the latency that differs.

They basically dynamically allocate compute into this pool from the general pool to meet demand and SLAs. If demand for priority processing spikes latency in the general pool spikes.

Almost all subscriptions use general pool. That's why it gets really slow sometimes. Unless you use "fast" which acts like priority processing and is super expensive.

So that's part of how they balance compute based on demand and fund the "excess" capacity.

Im asking on how lower compute could translate to worse model performance. That's the implicit claim being made when people are saying Anthropic nerfs a model (result is bad, not necessarily slow). The only way I can think that could happen is if they actually swapped Opus 5.5 out for a worse model that takes less resources to do inference.

There is no proof, the frontier labs deny it, and certainly it's not happening on hosts like Azure, Amazon, and etc.. The claim is that it's "trivial" to detect observation, so they serve the normal model if you try to measure it..

Basically the conspricy theory is:

* OpenAI/Anthropic nerf their models regularly to reduce compute demand

* It's impossible to measure and thus prove

* nerf them based on supply and demand, when you have a high demand, nerf it, but keep charging same price.

* do you want to show revenue growth to investors? increase thinking tokens, customers will keep paying, because again, this is non-deterministic system, sometimes it thinks more

* do you want to increase profits? start serving 4-bit, 5-bit quantizations and keep charging same price

* want even more? publish a paper "Advanced Auto Super Smart Prompt Router - AASSPiRation", and start routing requests to Qwen3.8-27B or GemmaE2B from time to time on the edge

How do you even lower the quality - serve half broken car for a luxury order? No way person who orders luxury car accepts it.

Or do you increase wait times? - sure, order gets cancelled, you won't get a payment for that.

There are of course ways, but you still can do something, in case of LLMs, you blame yourself for poor prompting and retry, which again makes money to LLM providers

- You can lower the quality standard of which car counts as which class. There are many classes of vehicle on both platforms.

- You can delete or not consider low ratings from certain users. Both companies do this already.

- You can increase wait times, of course. It doesn't at all follow that the order gets cancelled. It depends on the market and what each company believes the other is doing.

That's thirty seconds of brainstorming.

Opus is suddenly fumbling basic tool calls and failing to finish tasks even at high, where I had been permanently using it on low effort with success since it came out. It is very frustrating.

My guess is they quantized it and/or reallocated DC capacity to train a new model, this has happened before.

Not just frustrating, it can also be dangerous. What if you trust Claude with certain security-sensitive things (because experience), and suddenly you cannot trust it anymore like that?

then $corp should remove your access to these security-sensitive things ;)

It is deterministic, if we're being pedantic

Killing it's own shell, multiple, multiple times in the same session. I'm sure I saw a phrase "I did it again".

It's so shit that it must be damage control. The alternative is to limit usage and that'd affect the IPO even more.

I wouldn't say I've especially noticed its effectiveness having been reduced, it's just slower.

Weirdly enough, Sonnet quality seems unchanged.

You rewarded Anthropic's failure with more dollar spend. I get why you did it but...

After I figured out, like this chap on X, that it wasn't ME, I diversified my model usage and now a lot more work has shifted to Astra/Sol and I'm learning to make my workflows more resistant to the whims of each model.

This company does not deserve to be successful and has no moral high-ground.

I'm so happy to not look at bogus jargon that Claude made up.

As for faster, maybe? It depends on the metrics and what you're doing with it. If you mean TPS, no, I get much higher TPS and even faster TTFT on my local setup than frontier models. If you mean time to complete tasks... maybe. If you're throwing whole projects at it to vibecode, yeah, a frontier model will dump slop so fast it'll make your head spin. If you're doing actual methodical development, like doing spec-driven development and actually reading the specs before spinning off agents to implement them, rather than just having another agent review it and yolo, you'll find that frontier models don't really push the needle that much, and you'll be making high-quality projects.

Yes? If you're building anything reasonably complex, it is a ton of work to keep everything in your head so that different parts of the system work seamlessly and work you do in one area does not adversely affect another area.

Also, yes, I do need the capabilities beyond what can be reasonably achieved with local models because threat actors are also using frontier models.

This really depends on your infrastructure. In this case it all depends on your hardware, and there is no secret sauce, you can buy the same GPUs or better than the big players are using. Self-hostef GLM in our company is faster than any commercial model. Of course the hardware was not cheap, but completely manageable for a medium or large company.

You can not buy an H100.

Personally I was forced to use only GPT-5.6 Sol at work instead of Opus 5.5, and it was so much worse.

Benchmarks that control for benchmaxxing indicate all open-weight models behind 5.6 Sol and roughly at the level of GPT-5.5.

And that's for the latest large ones, much less local Qwen 3.8 Flash Next.

I would regret such a downgrade.

And that’s looking at a large model, to host the equivalent sized model as Opus 5.5 you’d need more than a single dgx spark. A fairer comparison would be to Open models hosted remotely or comparable weak models like Luna which you would get away with a $100 plan or less. That also doesn’t account for token prices continuing to go down for the same level of intelligence.

But even if that was not true, the more important metric, cost per task, has definitely gone down.

a dgx spark could never

By choice, maybe. New models release faster than every nine months. And, evidently, you won't be running nine months of some fixed meaning of "Opus 5.5". What that means changes over time, out of your control, and without ever even being officially told.

Now, if GPUs didn't cost an arm and a leg, the economics would have been different.

"Frontier" (closed) models are better by definition, but if frontier models were good enough for you half a year ago, open models are good enough now.

"good enough" leaves so much on the table...and wastes so much of your time. I value my time and attention

But the in a million years bit, even taken as rhetorical bombast, is of course silly. The difference between local models and the frontier models is ever shrinking. Extrapolating out and eventually, sooner rather than later, the difference is going to be a wash.

Unless all of the vendors stop progressing open local models, the future is clearly there. It's why I think AI is important and valuable, but also the current leaders have a dubious financial path forward.

He's pretending like there's any magic to claude or the others, there isn't

There are plenty of ways to run ds flash and qwen flash on single cheap gpu, but speeds are quite bad or contex windows are small.

On the single r9700 I was able to run q2 with 4 slots of 256k, and I think it was about 40 - 50 GB of system memory offloading? Got about 60 - 80 TPS per slot depending on concurrent use. With the two r9700s we are running q3 with 4 slots of 256k, using about 100 GB of system memory offloading, with mtp enabled. Getting about 70 - 100 TPS now.

which GPU?

also, so, 90 months of my subscription? And I also would need to buy new RAM and other parts of a computer?

I'm reminded of the video game industry where they would launch a game without any in-game purchases at launch, get the reviews to make note of it and give it a higher rating because of it... then add in-game purchases anyway. I think that stopped again though.

If you have 99.9% already - that's better than any pre-LLM deterministic linter.

Sounds like those teams need better leadership. Deterministic tools should always be used along side LLMs to improve the results and reduce costs.

It's like saying "save this json as .txt but using valid json syntax." I mean maybe that will work most of the time but WHY?!

2) the wholesome party is not in power yet

3) even when they are, the orange man will veto everything they might attempt to do

4) even when the orange man is replaced with someone wholesome, and that wholesome person enacts a bunch of wholesome legislations, data centers will simply move literally anywhere else on Earth.

>In Ohio alone, at least 14 municipalities passed bans or restrictions on datacenter development, including in Sunbury, where Amazon has proposed a $2bn datacenter project. In Maine, eight municipalities passed temporary datacenter moratoriums. In Emporia, Kansas, where a 1,000-acre gigawatt-scale datacenter was recently proposed, a ban on high-impact datacenters was approved by voters.

and now they will be built elsewhere.

I think there's a rather obvious precedent for that inevitable outcome.

If they did, there's no way that remains a secret.

If they didn't, Anthropic would be making their own service noticeably shitter than AWS Bedrock

Lots of "this is a potential issue, I didn't check into it though" which is super frustrating to see. The model just points out major flaws in logic or barriers but goes no further than a comment on it.

I've got a Qwen coder model and a flash next model locally now, so it's not the end of the world. But it does need to be remedied - are we seeing them prep for a new model deploy when this happens? Or are they experimenting with leaner hardware to drive down costs ahead of IPO?

It does seem like Anthropic is experimenting with how much less they can offer users. I also get the feeling they are judging what resources a task deserves and dumbing things down accordingly. I’m seeing lazy responses and skipped steps more frequently for non-technical queries.

It’s definitely possible the average user hasn’t noticed, unfortunately.

Oh, even psychological effect would be nice if you slightly tune down the current frontier 1-2 weeks before releasing the new one.

Dude, you have a mythical creature at your disposal at retail, chill, it's going to be ok.

Some models, like Opus 5.5 (last week) and Sonnet 5.5 (now) route to Fable 5.5.

Sounds like analysis fully driven by a model, which can be fine, if you know what you are getting into and can verify the output. Just last week had my dad ask Claude Opus 5.5 on High about watermarking after I informed him. The model told him that watermarking text was not implemented in Anthropic models, whereupon he showed me and I then asked him to just ask the model to check again, ideally online. These models do still "hallucinate", they still have major gaps in their output not easily caught if you don't know the topic and interestingly, anything connected to LLMs, model providers, etc. tends to be where they do worst.

In this case, looks like the LLM did not consider that these are (and have been for months now) just summaries and the actual reasoning can not be analyzed for encryption reasons.

If in doubt, please do some research independent of the model output. LLMs not having any internal "understanding" that'd give them anything akin to an internalized "feel" for what they "know" accurately enough is one of their major flaws.

    1. I'm in a A/B with a ruthlessly effective model
    2. I've tuned my memory files in a way the model likes
    3. I've been lucky 
    4. I'm just "better at claude" than most

It's impossible to say, and that is clearly part of the problem. More visibility of this stuff might not even help us, given it is human nature to ascribe patterns to randomness where they don't exist.

I'm afraid to have touch my code base because a nerfed model could slip in stupid bugs.

This is also why a weaker local model might be preferable because it allows me to calibrate trust, and my expectations don't change.

I have had Astra analyze my long running conversations and the degradation definitely happened on November 6th and 7th.

I run a small local GPU cluster for Qwen 3.8 Flash and while it's "free" (you should check electricity prices in EU...) unfortunately it doesn't even compare to Opus 5.5 or Astra in capabilities. It's like GPT 6 vs GPT 3, but imagine GPT 3 is also 100x slower and sounds like an airplane taking off.

In the end it's totally possible that I would pay more for electricity running Qwen for 24 hours churning on some complex problem vs. Astra one-shotting it in 15 minutes.

I hope we get there, but it's not there yet, and I'm sure never will be with consumer GPUs. Maybe if you spend $20k+ on a local AI rig. There's just no market force to give access to AI to everyone at fair price.

I use Sonnet 5.5 daily and Opus rarely. My usage is also a little involved as in I do all the design and strategy and get Sonnet to just write the code

For that, Sonnet has been consistent

I use Opus or Fable when it's a dumpster fire situation where we have no time to think even

So far I've never had an issue with Sonnet even.

all the measurements always say it isn’t

anyone who grew up playing dungeons and dragons gets that this is just people saying “you don’t understand, these five used to roll high”

every llm space is inundated with rank amateurs explaining at great length how it used to be better yesterday

nobody has the courage to clean out the obvious bullshit, so the drone gets ever louder and the signal gets ever rarer

do you remember what programming was two years ago?

stop complaining so much

Instead of being Opus on high (effort), it's Opus is high.

I don't have measured data aside from myself noticing my frustration level with Claude jumping on these days frequently over the past several months.

If you paid for X but are now receiving Y, you should be entitled to get your money back.

Every time I consider going back to Claude, I remember that mysterious week in February when Claude was failing to do even the simplest things. Things were so bad i used the free BigPickle to correct errors Sonnet was making.

And let's not forget the king of A/B testing here. Google has no issues breaking products for months just to test some ridiculous idea.These people should be publicly fired.

100% agree. I don't care how many reddit bots i see praising Claude, I bought max once wanting to see what the fuss was about, and the next day they broke the product and it was really embittering

They can one day decide to rearrange the Gmail interface to absurd levels by removing the TO field or replacing everything with a big Gemini button.

To me it seems there should be a very viable middle ground of paying for open model hosting by a trusted provider - unless I'm only hearing from 24/7 vibe coders

The only one I'm familiar with testing Opus 5.5 specifically over time is https://github.com/ninjahawk/livenerf and it's not showing anything.

They also have this comment in getting started: "Pin the CLI. This is not optional: a Claude Code update changes the harness, and a changed harness looks exactly like a changed model. Turn off auto-updates and write down the version you're pinning"

Wild because as an end user I don't care what Anthropic changed to affect performance, I only care that Anthropic caused degraded performance.

So your harness behavior can change without updating it.

Fun.

If there was a silent degradation then the recent chatter about suspected nerfing is what you would expect to see. Of course that's not evidence, but I dont think we can say it's far more likely that its all subjective.

I mean the basic intuition doesn't seem unfouded. Maybe people can confirm/deny this contention for compute. But Anthropic is compute constrained. Inference takes compute. Model development takes compute. Isn't it plausible that there will be periods of contention? Or do we have information that suggests inference compute is affected by model development?

Anthropic has confirmed that they make changes to the models, for example with the recent update to limit reverse engineering.

[1] https://www.rensoart.com/why-your-eye-is-ahead-of-your-hand-...

The only thing I could maybe see is that any changes that are being made are happening on the subscription tiers on not the api tiers, but even that seems like quite a stretch.

Like anyone who is building agents with a good eval suite would know, models have been pretty consistent and even small harness changes can impact performance considerably.

If you calculate Elo only before 11/3 and compare to only after 11/3, the Elo is 52 points lower after. Unfortunately we are still in beta so this isn’t conclusive (confidence interval of -160 to +40).

If we do this calculation on all our models, Opus 5.5 is first in apparent drop. (Again, not definitive, but supports the conclusion.

[1]: https://agentduel.com/leaderboard

He has his "nerfbench" he calls it. Very useful and entertaining at least. :)

Firstly, there is a lot of A/B testing. You would likely need at least 10 or 20 accounts to catch these random tests.

Secondly, the labs obviously run GPQA and MMLU like livenerf does and watch for their own regressions before doing A/B tests. These benchmarks are statistically the thing already being filtered out. You need tasks out of distribution. Ideally ones in distribution with the stuff they particularly frequently change (like safety or copywrite adjacent tasks).

These harnesses, and hidden system prompts, do get screwed with quite a lot in my experience. The models are actually remarkably consistent via API but there are tons of stealth system prompt and harness changes that lobotomize these models all the time.

Like despite being stochastic, try asking Luna to generate you a video game pitch via the API 100 times. It's almost always the same thing about lighthouses for some reason. That's actually why I tried to make a CLI entropy generator project.

I’m only prototyping so most of the code will be discarded, but even in this circumstances you can’t allow too much sprawl and decoherence within a single project. It makes prototyping cost so much more over time. It was pretty frustrating.

It is something I also appreciate with some of the other labs like deepseek who publish the checkpoint they are running on and still sell you the old one for a period of time as well.

Its obnoxiously anticonsumer to be constantly chopping and changing what a model represents on customers who are paying them an awful lot more to be on the frontier models.

Better tooling/reporting to demonstrate model degradation is needed though.

Am talking about low level Linux kernel interfacing stuff, my own Google Opal like UI generator...still rough but it's machine state to bang into shape not a bridge. It works but needs the refinement common to early development

Imo due to recent job trope developments a lot of devs expect a git repo that feels like a 3-ring binder with their senior thesis and such tucked within

Local models inability to one shot such is being used as cope. Imo these models are more capable than we want to believe

This all reminds of 20 years ago when sysadmins refused to acknowledge the cloud would take off due to privacy concerns of businesses

Look how that went

Data center needs are inflated to try and capture all the business but future devices capable of optimized battery/telemetry aware local inference would satisfy most people

The data centers serve 3 groups; SaaS workers, data center landlords/CEO/shareholders, politicians being paid off by data center landlords/shareholders

The other billions on the planet do not directly benefit as much. Merely as a side effect of trickle down

Of course the Sota models on the same custom harness got to about 99%, but I'm amazed at the performance of such a tiny public free model running on my 3080 laptop gpu. If it was actually finetuned for said task I imagine it could get to 90%+ completion.

Of course, any given person might want their own navier-tokes agent swarm of a solution, but the number times you want a bot to fill out a resume rather than solve a millenium problem is absurdly lopsided.

So it's just kinda, ok guys, are you going to just try to hold us hostage with your magicaly super-duper-intelligence, or are you going to admit that having great models means humanity gets a few smart people to be even more useful for whatever they're working on.

The rest of us arn't going to even see that future to begin with, because food, it's not about how much food humanity grows; it's how it's distributed. Unless this AI stuff solves a problem _no one_ has solved for food, it's just going to be another power chair to fight about who gets to sit there, who has the right to sit there, or like I suggest, it's just a utility thats to be managed by the same governance all other utilties do and has the same problem thereof.

So like, boring stuff; not p(dooom)

IMO these models are best at non-coding tasks. They of course have to use a computer, but from CAD generation, 3d Modeling, emails, legal and academic analysis, etc. these models are incredibly capable, and we are going to see a massive shift in the entire job market due to this, and ironically I suspect one of if not the most protected career as a result of this massive upcoming buildout will actually be software engineers.

    $ free -m
                   total        used        free      shared       buff/cache   available
    Mem:         3095421       48746      819943         554       2242686     3046674
    Swap:          84390        2594       81796

Livenerf: Has Opus 5.5 been nerfed yet? - https://news.ycombinator.com/item?id=49901736 - Sept 2026 (394 comments)

Every time a new model drops it feels wildly powerful, then regresses to some sort of generic SOTA level we've been used to over the last 6 months.

I remember my first coding prompt with, I think it was sonnet 4.6, on a fresh account and it was shocking how much it successfully did off a single prompt - I haven't found that with any other model at any other time and I suspect it was because they put me at the top of the priority list to get a "wow" reaction.

Regression after a release is pretty well-agreed upon, but I'm wondering if they secretly have a huge amount in the tank they aren't releasing. Either as a card to play in the future, or because they can't keep up with demand. I've heard anecdotes of people working with these companies saying that the public models aren't even close to their actual capabilities... Anyone have any opinion on this?

If you're selling model capabilities for money, and you change the mobile capabilities, not changing the model designator (name and/or version number) is fraud, full stop.

Unlike here where the "phantom bank accounts" fraud was successfully pinned on tellers and branch managers who were all "independently" just trying to meet management goals...

And yes, if you're doing engineering that violates the law, you should be fearful of doing. That's kind of the point of punishing criminals.

https://www.justice.gov/archives/opa/pr/volkswagen-engineer-...

Opus 5.5 was significantly crushing every task I gave it for quite some time.

Suddenly, it starts making the type of mistakes that even Sol 5.6 would not make. It became noticeably worse than Astra.

I've grown so completely tired of the labs changing the quality of the model out from under our feet that I CANNOT wait until there's viable alternatives at the frontier level.

I will gladly forgo the best available power for consistency - and I see that coming soon (hopefully with Qwen 4 or shortly after).

DeepSeek v4.1 Flash is not even remotely in the same league as an un-nerfed Astra or Opus 5.5 - despite what microbenchmarks may claim.

It's a great model, especially for the cost - but there's a huge difference.

Oh good Lord, they stir the pots constantly. Anthropic is hella naughty about this. Someone else on HN said it well, Anthropic has too much of a "researcher" mindset. You cant experiment on human populations tool use the same way you run AI/ML experiments.

I want a stable tool that works consistently, not a hammer that changes mid-swing (credit Mario Zechner for this analogy).

Google and Facebook have proven otherwise. Unless you are very, very lucky you are always part of multiple running experiments on their products, and nobody really cares.

2. Google doesn't pretend that its algorithm has certain capabilities while serving the user a nerfed version

I worked recently for a German megacorp, for whatever reason they gave us unlimited access to a bunch of models, up to sonnet 4.5/Gpt o1 etc, so not the best or latest models. And no agentic tools.

So I built a custom Claude Code clone because they exposed the api from their custom 'LLM Chat' clone in the client and accepted my user credentials for access.

Essentially a frankenstein AI harness using various models for their strengths, including using the OpenAI best model for memory context consolidation and compaction + Sonnet as orchestrator, etc. I was able to spawn 20+ agents to do my work and basically automate my entire job at that corp. I think my internal bill was something like 5k euros per month and I was being careful to not abuse it...

It was better than the best you could get from claude at the time (about a year ago now I think), in terms of task performance completion.

So imagine what someone with essentially infinite spend can do with endless fable agents. AKA US government and Anthropic internal.

Not a great plan

I don't know who they think they're fooling.

https://news.ycombinator.com/item?id=50429947

If I order a 16oz beer and get a 12oz pour, I'm going to call it out.

And then Sam Altman is gonna sue your ass for owning a measuring cup.

Is this 2026's "my macbook feels 'snappy' when I update it", though? (That is to say, not really much in evidence)

House of El covered this in her latest video incidentally.

I'll gladly sign up to be a "Experimental Settings" beta tester (with knobs of how extreme I want to go) for a slight discount (or just bill me the same, just gift me a reset every week? idk whatever).

Doing A/B testing in prod on a frontier model that could really mess with the users in really bad ways is a bad idea.

I suspect it's rather a simple matter of cost.

On some long running conversations it seems to better understand the end goal, follow the constraints we have established.

Exactly the reason I had previously cancelled my openai subscription

I'm looking into Gemini

At the model release people become high/manic/addicted to the newest model think they now have some new superpower and then that "high" wears off and you think the model isnt as impressive.

The truth is llms have a slot machine-esque UX to them, and the labs designed it this way.

Dont worry you'll get your new model with some mystical name any day now and a month or two afterwards you'll be posting about how it no longer feels as good to use.

Variable token usage? Sure. To be expected in non-deterministic systems like these.

Distinct increase in errors and issues? Never. Not within a single model's lifespan have I ever experienced this.

And this whole notion of the A/B testing of performance/usage is completely ridiculous. There is zero evidence it has anything to do with performance. There are a trillion things to A/B test besides things that hamstring groups of users, so this is just being used to confirm the bias of people already caught up in these "theories."

I await any realistic indication.

If they're messing with models like this, surely they'd at least occasionally make a tweak that improves intelligence rather than degrading it. But I never see anyone say Model 4.2 got smarter a week ago or whatever.

I just don’t see the point of using these expensive models anymore.

Eval context: 30 bug fixes and features evaluated by tests the agent didn't see. Ran on Opus 5.5 at medium effort in Claude Code.

20 of the tasks were last run on Nov 1, twice each. Nov 1 solved 36/40, and today solved 19/20. Thinking tokens per task were 0.95x Nov 1’s and output tokens 0.97x. The one significant change was speed: median time to first token decreased from 3.0s to 1.7s.

Caveats: one run per task today, medium effort only.

If you look at any sort of tracker, there's no telling whether you're in the same experiment group as the tracker (most likely not).

  Opus 5.5, one MacBook in SVG   Sep 22-23                  Nov 9
  max, streamed (128K cap)       128,000 tok, $2.56, empty  128,000 tok, $2.56, empty
  max, Batch API (300K)          255,957 tok, $2.56         243,707 tok, $2.44
  xhigh                          86,096 tok, $1.72          107,333 tok, $2.15

The model is still the best of everything we've tried. Most likely the next best would be Fable 5.2. But we don't like Fable because essentially the result is the same, just the price is higher. Its medium and high are close to what Opus 5.5 Max had, but the price is higher.

(I run Playcode, an AI app builder. We benchmark every model before we add it.)