So it can never be right?

We’ve been two years away from nuclear fusion for 50 years. Sure, at some point, maybe we will genuinely be two years away, but without a crystal ball it’s not much use to keep saying it.

Terrible analogy, nobody thinks we're 2 years away from fusion, or ever has. 20 years, sure, maybe 10 even. But not 2.

As-is we're not going to get commercial fusion until at least what, 2040? 50?

I think its more likely that llms got better at using harnesses, which help force them to make sure everything compiles/runs before passing back to the user. This caused people to form habits around this where they slowly stopped checking the output.

So people are super impressed because things just work, and they do.. Until they dont.

Just because things compile doesnt mean they're going to hold up, and most of this slop wouldn't, ive seen it first hand and you can also tell by the increased amount of outtages across different platforms.

It was crap, it's good now is the positive equivalent of the boy who cried wolf. We've been hearing this bullshit so, so many times that no sane person would care to listen without some serious hard evidence and no your zero user vibe slop repo launched yesterday or "I'm so much more productive…dicking around the codebase but zero useful features to be seen by the user" doesn't cut it.

It's just obviously true if you use the models regularly. I don't know how anybody can come away feeling otherwise.

Through just using them. Some of us really have some expectations.

Could you share something that we can replicate, where SOTA models and harnesses, i.e. a Claude, does not work where say a junior or medior engineer new to your team would solve the problem?

Do people expect an eternal junior engineer in their teams? Even with a recent hire, there’s a deadline to the onboarding and hand holding process. Generally after three months, you were expected to deliver with no oversight other than peer reviews. It was a good deal for the team. Spend three months teaching, and get a productive team member for two years or more.

Now it’s permanent baby sitting.

Ok, so NOW that the models are awesome, i assume you will drop a lot of impressive software, correct? You were not able to do that a year ago because they were bad, however, that bottleneck is gone NOW, so NOW you will deliver, right?

It's so obviously true until the next model comes out. Or until a non-American model comes out matching it, then it's "no one could using AI tools for coding until this new GPT/Claude model came out"

Maybe it can be but it seems like the it needs more substance, no?

It means it can’t be given a free pass.

Repeated forever, a statement always has a chance to be right.

After a point it’s going to be challenged

If a boy always cries wolf, are you going to say "well there's a wolf we have to go get it."

I think you’re just lying. Where did you hear this? Some randos online? Years, really? Years ago LLMs couldn’t even program at all.

Every person I’m meeting in the real world who use agents have roughly the same experience. About a year ago they started working reliably and the side projects where we played around with it started seeing actual results.

About 6 months ago I started seeing colleagues and friends all around the same time saying that LLMs solve real problems for them.

For my own case, I’ve replaced years of sloppy spaghetti code in our internal tools and services, from a bunch of engineers many of which has left the company, with code that’s objectively better quality and more coherent (though docstrings and tests tend to accumulate garbage, so cleanups are necessary eventually).

It means something different when John Nagle says it than when some random person says it, but also I think your memory may be faulty.