Coding agents have only been really working for about 12-18 mmonths at most. It's easy to forget!

And even in that short time range, the difference in capabilities 18 months versus 3 months ago is so massive that the expectation of what it means for them to 'work' has changed.

You honestly don't notice any improvements in terms of model capabilities?

Not the same person, but I don't. I have noticed a slight improvement from GPT-3.5 until now, but not a marked improvement like people say. And I didn't notice any of the claimed massive improvements along the way either. Every model I use, even the very latest, is still fundamentally very stupid and cannot be trusted to do its own thing.

Same here. At certain tasks, you can see a marked improvement in code quality. I think it’s just focus during training that produces this. In terms of fundamentals, they’re just as likely to fumble things at random, over abstract, duplicate, overfit and fail in the same ways. A lot of people who spend a lot of time poking at AI slowly stop thinking critically about it too.

I see it like someone walking through a minefield safely after not noticing the signs.

“What do you mean it’s a minefield? MY legs didn’t get blown off.”

Sounds like it's getting better with each release then