It's because the frontier models are superior and much faster.

Superior? Sure. But do people need the capabilities beyond what can be reasonably achieved with local models? I think the people that actually sit in that gap are vanishingly small, and if your title contains "Engineer" or "Developer" or something like that, you are not one of those people (at least for doing your job)

As for faster, maybe? It depends on the metrics and what you're doing with it. If you mean TPS, no, I get much higher TPS and even faster TTFT on my local setup than frontier models. If you mean time to complete tasks... maybe. If you're throwing whole projects at it to vibecode, yeah, a frontier model will dump slop so fast it'll make your head spin. If you're doing actual methodical development, like doing spec-driven development and actually reading the specs before spinning off agents to implement them, rather than just having another agent review it and yolo, you'll find that frontier models don't really push the needle that much, and you'll be making high-quality projects.

Every hour baby-sitting a local model can be extremely expensive in billable hours.

Is there a model that doesn't need babysitting? Half my day is spent untangling BS that Claude and Codex hallucinate. Particularly around hallucinated product requirements. Its still faster than hand-writing code but only just. I'm seriously considering just going back to the early copilot days where I lead coding and just let it try to auto-correct. That was a great balance of AI enhancing my speed and workflows without removing any confidence in what was being shipped.

give your models a more organized context/harness if they are hallucinating product requirements... and I'm not talking about some hulking context framework. just organize your documentation in a canonical way. This product requirements hallucination is something that would bite me constantly if...they ever actually hallucinated them (and they don't, ever). This is most certainly user error.

You're untangling hallucinated BS from Opus 5.5/Fable?

We are literally in this thread because so-called frontier SOTA Claude models are shitting the bed and require babysitting. So this argument is essentially moot.

What’s your local setup? I’d love to give it a spin; lack of 5 hours limits is much needed.

Qwen 3.8 (and maybe soon 4) series models are probably the best right now for local AI. Specifically, for lower end hardware like 8gb GPUs you might find a lot of use in a Qwen 3.8 variant like Underdog Saluki 27B.

I might not need them but I'll sure as hell take them if they are available

>> But do people need the capabilities beyond what can be reasonably achieved with local models?

Yes? If you're building anything reasonably complex, it is a ton of work to keep everything in your head so that different parts of the system work seamlessly and work you do in one area does not adversely affect another area.

How much did you spend on your local setup out of curiosity?

Also, yes, I do need the capabilities beyond what can be reasonably achieved with local models because threat actors are also using frontier models.

They are faster, but the distance from superior is changing daily. Naturally, things like purposefully kneecapping the product does not help.

>much faster.

This really depends on your infrastructure. In this case it all depends on your hardware, and there is no secret sauce, you can buy the same GPUs or better than the big players are using. Self-hostef GLM in our company is faster than any commercial model. Of course the hardware was not cheap, but completely manageable for a medium or large company.

Speed can mean two things: rate of token generation or time to finish a task. What maters it the later and a model that takes a faster path to the result and uses fewer tokens because of that comes out ahead in practice.

> you can buy the same GPUs or better than the big players are using

You can not buy an H100.

Voluntarily downgrading to Sonnet 4.5 level performance sure is one way of avoiding imagined degradation in Opus 5.5...

Have you ever heard someone say they did so and regretted it?

First time I hear someone praise Qwen 3.8 Flash Next specifically.

Personally I was forced to use only GPT-5.6 Sol at work instead of Opus 5.5, and it was so much worse.

Benchmarks that control for benchmaxxing indicate all open-weight models behind 5.6 Sol and roughly at the level of GPT-5.5.

And that's for the latest large ones, much less local Qwen 3.8 Flash Next.

I would regret such a downgrade.

i haven’t heard anyone say they downgraded their car to the children’s toy power wheels and regretted it. all the same, i’m confident that it’s a bad idea

Compared to Opus local models Opus 5.5 is far superior, also for something like a dgx spark at 5k that would be 25 months ignoring power costs if you chose the $200 plan.

And that’s looking at a large model, to host the equivalent sized model as Opus 5.5 you’d need more than a single dgx spark. A fairer comparison would be to Open models hosted remotely or comparable weak models like Luna which you would get away with a $100 plan or less. That also doesn’t account for token prices continuing to go down for the same level of intelligence.

Are token prices going down?

Yes, by a lot. You can get opus 4 level of intelligence like 100 times cheaper now than at premiere.

It has been trending downwards. See recent Anthropic and OpenAI price changes.

But even if that was not true, the more important metric, cost per task, has definitely gone down.

it is believed that opus5.5 needs 8x h200 to run (~1t vram)

a dgx spark could never

I can buy a GPU that can run Qwen 3.8 Flash Next for 9 months worth of the highest tier Claude Code subscription, but then I'm running Qwen 3.8 Flash Next when I could have had nine months of unlimited Opus 5.5.

> then I'm running Qwen 3.8 Flash Next when I could have had nine months of unlimited Opus 5.5.

By choice, maybe. New models release faster than every nine months. And, evidently, you won't be running nine months of some fixed meaning of "Opus 5.5". What that means changes over time, out of your control, and without ever even being officially told.

Sure, but, even when it degrades, it's better than Qwen 3.8 Flash Next.

Now, if GPUs didn't cost an arm and a leg, the economics would have been different.

i dont understand how people keep comparing local models with frontier models, they dont belong in the same discussion especially for coding usecases. they solve different problems under different constraints.

Indeed. Local models remain rock solid, while cloud models are a shitshow of stability.

I don't understand what do you mean. They both solve the same problem. Some models solve it better, some worse. And in some cases you can only use local models, so they solve more problems.

"Frontier" (closed) models are better by definition, but if frontier models were good enough for you half a year ago, open models are good enough now.

This whole recent "good enough" phrase that is being thrown around more and more is completely asinine.

"good enough" leaves so much on the table...and wastes so much of your time. I value my time and attention

nobody with the competence to judge the quality difference between frontier models and local models would ever, in a million years, use local models

Why not? There are a lot of tasks that a local model can do cheaper.

In a workplace environment I don't think "cheaper" is necessarily the thing to optimize for. Subscription models save you a lot of time (and money) by getting things right faster, solving bigger problems, etc. A cheaper model just consumes more human time.

But with a local model with essentially free tokens, I can set it to work on tasks in the background without having to worry about any usage limits. Deferring simple bug fix tickets or other low complexity to an essentially unattended local model works just fine. And if the result occasionally fails review, it still didn't burn paid usage credits.

A local model tops 60 tokens/s max. Running it on the background means not being able to use on the task you are working on. When working on local hardware you cannot do concurrent requests. In deepseek for example you can have 100 agents running each getting at 120 tokens/s easily. In local models you are limited by your own infrastructure.

I do concurrent requests just fine and they have no noticeable performance impact. Not all local hardware is equal.

With cloud agents I also process stuff overnight. But my gpu is limited. If I want to OCR a high number of documents or vectorize data, I can't run an LLM at the same time. I use remote models and use my compute for running commands on my desktop machine.

You seem really upset that people do things differently than you. May I suggest trying Tetris to help you feel better? https://play.tetris.com

This only holds if you do not value your time meaningfully.

Today this is absolutely true. Local models have gotten very good, but they aren't there yet. The cost of a Max subscription or even AI pricing is overwhelmingly worth it for most users, even if you ignore the enormous capital costs of running the competitive models locally.

But the in a million years bit, even taken as rhetorical bombast, is of course silly. The difference between local models and the frontier models is ever shrinking. Extrapolating out and eventually, sooner rather than later, the difference is going to be a wash.

Unless all of the vendors stop progressing open local models, the future is clearly there. It's why I think AI is important and valuable, but also the current leaders have a dubious financial path forward.

It's an easily reproducible technology, clearly...

He's pretending like there's any magic to claude or the others, there isn't

Enjoy renting everything you depend on!

I sure do. And 2-4 years in the future when this insanity is over I will buy and own local llm hardware. But certainly not now with those prices and speed of progress that obsoletes everything in a year or two.

I made some assumptions and was foolish. Sadly not a new tale. But still in context I do think for every You there are several other Not Yous who are planning infrastructure who aren’t keeping an eye on the local escape hatch.

What GPU can you buy that will run a 180B model for $1800?

You asking wrong question. You need to ask what tks/seconds it's gomna run at.

There are plenty of ways to run ds flash and qwen flash on single cheap gpu, but speeds are quite bad or contex windows are small.

Qwen 3.8 Flash Next is 125B with 6B active, which with quantizing and system memory offloading (strata) runs nicely on 32 GB cards (I have actually run it on as low as a 16 GB GPU and 16 GB of system ram, but I don't recommend it for serious work). I was running it on a single r9700, but now I'm using 2 r9700s because I'm splitting my server with a friend, and they bought one as well.

On the single r9700 I was able to run q2 with 4 slots of 256k, and I think it was about 40 - 50 GB of system memory offloading? Got about 60 - 80 TPS per slot depending on concurrent use. With the two r9700s we are running q3 with 4 slots of 256k, using about 100 GB of system memory offloading, with mtp enabled. Getting about 70 - 100 TPS now.

What is the alternative? Local models? I do not think so. Maybe Kimi K3 (still not local) as this model can be served by multiple providers? However Kimi K3 sits between Sonnet/ Sol and Astra/Opus so still not the same level.

Lots of models are being served by many providers. Even Anthropic models can be served by Azure. Check for example https://www.requesty.ai/models

Yeah but what about your duty to consume and be part of the economy?

Why lease a Ferrari when you can buy a Corolla.

This, but unironically.

You two actually summarized the whole discussion beautifully.

Why indeed. (The Ferrari's always in the shop, and the Toyota gets great mileage, and you need to commute.)

If you’re commuting then sure, but most people are racing.

If the car's in the shop, you're not winning any races.

can u tell me the specs or hardware requirements to run qwen3.8 flash

> You can buy a GPU that can run Qwen 3.8 flash next for about 9 months worth of the highest tier Claude Code subscription.

which GPU?

also, so, 90 months of my subscription? And I also would need to buy new RAM and other parts of a computer?

20 euro claude sub is too good compared to a 6-10k hardware purchase to get something thats not even as good. Once the enshittification starts on full force it will be a different situation.

The value of a Claude sub has been dropping by like 50% every two weeks at this point due to usage reductions.

I honestly thought usage was going up. Now, I dont use LLMs nearly as much as a lot of people here, but I used to hit my limit very easily, and now I haven't hit it in wmquite some time.

I don't know how you thought that when real usage is down 10x since 2 months ago, they publicly announced announced several usage decreases and related issues, and its a constant point of discussion/criticism online.

Problem is that everyone loves to pretend their CRUD SaaS is special and requires the best of everything, so they 'need' the absolute frontier with as many tok/s as money can buy.