> It was easy to get comfortable with using the best model at the highest setting for everything for a while, but as the models continue to scale and reasoning token budgets grow, that's no longer a safe default unless you have unlimited budgets.

For a while I used Cerebras Code for 50 USD a month with them running a GLM model and giving you millions of tokens per day. It did a lot of heavy lifting in a software migration I was doing at the time (and made it DOABLE in the first place), BUT there were about 10 different places where the migration got fucked up and had to manually be fixed - files left over after refactoring (what's worse, duplicated ones basically), some constants and routes that are dead code, some development pages that weren't removed when they were superseded by others and so on.

I would say that Claude Code with throwing Opus at most problems (and it using Sonnet or Haiku for sub-agents for simple and well specified tasks) is actually way better, simply because it fucks things up less often and review iterations at least catch when things are going wrong like that. Worse models (and pretty much every one that I can afford to launch locally, even ones that need around ~80 GB of VRAM in the context of an org wanting to self-host stuff) will be confidently wrong and place time bombs in your codebases that you won't even be aware of if you don't pay enough attention to everything - even when the task was rote bullshit that any model worth its salt should have resolved with 0 issues.

My fear is that models that would let me truly be as productive as I want with any degree of confidence might be Mythos tier and the economics of that just wouldn't work out.

The GLM-4.7 model isn't that great. I was on their $200/month plan for a while. It was really hard to keep up with how fast it works. Going back to Claude seems like everything takes forever. GLM got much better in 5.1 but Cerebras still doesn't offer that yet (it's a bit heavier). I have a year of Z.ai that I got as a bargain and I use GLM-5.1 for some open source stuff but I am a bit nervous about sending data into their API.

The new one is quite a bit heavier!

GLM 4.7 is 358B parameters: https://huggingface.co/zai-org/GLM-4.7

GLM 5.1 is 754B parameters: https://huggingface.co/zai-org/GLM-5.1

That said, 5.1 is indeed a bunch better and I could definitely see myself using it for some tasks! Sadly all of the stuff I can actually run locally is still trash (I appreciate the effort behind Qwen 3.6, Gemma 4 and Mistral Small 4 though, alongside others).

Good points. I was speaking from a position of using an LLM in a pair programming style where I'm interactive with each request.

For handing work off to an LLM in large chunks, picking the best model available is the only way to go right now.

I have this exact same fear as an IC.

I wonder if Engineering Managers have this same fear, or they’re used to having to distribute complex tasks to senior engineers and gamble with seeming less risky tasks to juniors that may leave ticking time bombs in their code. Just the nature of code written by agents or humans?

Yes, that is absolutely a dynamic in managing an engineering team, and I'd argue that knowing the right person to give a particular task to, and how much detail they're going to need to get it done, is what separates good engineering managers from bad ones.

Yes, that definitely happens as an EM. You want your Senior/Staff engineers to architect out the new high-risk functionality into a doc for review. Then that Staff engineer either implements or has a junior/senior under their wing helping implement some of the scaffolding.

In this [common] paradigm the Staff Engineer acts as a architect/programmer and project manager in one. The EM should be there to guide and unblock.

> I know not everyone wants this mental overhead, though.

I’m curious how to even do it. I have no idea how to choose which model to use in advance of a given task, regardless of the mental overhead.

And unless you can predict perfectly what you need, there’s going to be some overuse due to choosing the wrong model and having to redo some work with a better model, I assume?

Human dev labor cost is still the high pole in the tent, even multiplying today's subsidized subscription cost by 10x. If the capability improvement trajectory continues, developers should prepare for a new economy where more productivity is achieved by fewer devs by shifting substantial labor budget to AI.

I'm getting a lot more done by handing off the code writing parts of my tasks to many agents running simultaneously. But my attention still has its limits.

Your employer doesn’t pay the subscription cost, they pay per token. So it’s already way more than 10x the cost.

Depends on the type of subscription. We have Codex Team and have a monthly subscription, no per-token costs.

Isn't that essentially GPT Pro Extended Thinking?

Except developers can’t even do that. Estimation of any not-small task that hasn’t been done before is essentially a random guess.

That's why you split tasks and do project management 101.

That's how things worked pre-AI, and old problems are new problems again.

When you run any bigger project, you have senior folks who tackle hardest parts of it, experienced folks who can churn out massive amounts of code, junior folks who target smaller/simpler/better scoped problems, etc.

We don't default to tell the most senior engineer "you solve all of those problems". But they're often involved in evaluation/scoping down/breakdown of problem/supervising/correcting/etc.

There's tons of analogies and decades of industry experience to apply here.

Yeah... you split tasks into consecutively smaller tasks until it's estimateable.

I'm not saying that can't be done, but taking a large task that hasn't been broken down needs, you guessed it, a powerful agent. that's your senior engineer who can figure out the rote parts, the medium parts, and the thorny parts.

the goal isn't to have an engineer do that. we should still be throwing powerful agents at a problem, they should just be delegating the work more efficiently.

throwing either an engineer or an agent at any unexplored work means you just have to delegate the most experienced resource to, or suffer the consequences.

I don't completely agree. Estimation is nontrivial, but not necessarily a random guess. Teams of human engineers have been doing this for decades -- not always with great success, but better than random. Deciding whether to put an intern or your best staff engineer on a problem is a challenge known to any engineering manager and TPM.

or tech lead. or whoever. the point is, someone has to do the sizing. I think applying an underpowered agent to a task of unknown size is about as good as getting the intern to do it.

Even EMs and TPMs are assigning people based on their previous experience, which generally boils down to "i've seen this task before and I know what's involved," "this task is small, and I know what's involved," or "this task is too big and needs to be understood better."

We should lean on the LLMs to do estimation. Claude loves to estimate that a plan will take two weeks, then implements the whole thing in 45 minutes.

The problem is half the time you don't know you need the better model until the lesser model has made a massive mess. Then you have to do it again on the good model, wasting money. The "auto" modes don't seem to do a good job at picking a model IME.

The problem is that without a lot of human involvement (which is even more expensive), even the best still make a lot of mistakes.

I'd be fine with right-sizing the model and effort if there was any semblance of reasoning stability. I thought we were getting there a couple months ago, but last night Opus couldn't even insert a new item in a numbered list without getting confused and ending up with a duplicate item and the wrong number order.