On the other hand, if the models can be Atkinson-like and produce -2000 lines of code [0], that is a day to which I would look forward.
0. https://www.folklore.org/StoryView.py?story=Negative_2000_Li...
Could that be iterated? Can a half baked release ever become whole or does progress just slow down and stretch out exponentially? 1/2 + 1/4 + 1/8 + ...
Will be grateful and relieved when proven wrong.
You want to increase churn? Are you insane?
I feel like if you poll computer / phone users (ie., the vast majority of the population) one of the most annoying things users hate about modern devices are mandatory updates, like the infamous meme video where someone screams at their window machine updating. Now, you want to make it easier for middle managers to push new unneeded updates so they can pad their resumes, how absurd.
For most big software projects the bottleneck is not writing code and test boilerplate that runs correctly locally. Which of course LLMs can do reasonably well. Most time is spent on design, on integrations with other internal services, scaling concerns, and reasoning about other things that these models can't really reason about yet.
I'm obvious impressed that these models can generate simple client/server websites and generate code to solve competitive programming problems. But I don't think they've "solved" software engineering; they've only solved "what you boss thinks software engineering is".
Which is why bosses are excited but engineers are not, because subpar LLMs will be used to justify firing actual engineers and replace them with fresh college grads tasked to work with a code bot, because the bosses fundamentally think that is enough.
If a new grad + LLM can do task X about as well as an experienced engineer can, then the experienced engineer will have to find other harder problems to solve. Personally I think labor efficiency is good; why waste your labor on easy/solved problems?
The experienced engineer also gets an LLM though
Say experienced engineer Edna was working on a project and the new grad Nelson replaced her because he was cheaper and about as good if both have LLMs, i.e.,
Edna + LLM ≈ Nelson + LLM
even though
Edna >> Nelson.
So Edna needs to find a project to work on where LLMs don't equalize her arbitrage, i.e.,
Edna + LLM >> Nelson + LLM
and Nelson needs one where
Nelson + LLM >> LLM.
Project fails = bad.
If a bunch of projects given to NG + LLM fail, the company will say at some point "this is no good, let's not do that anymore". If they mostly succeed, they'll do it more.
From a middle manager's perspective, where they're funding 10-20 projects, this kind of measurement is feasible and carries at least some signal. So I don't agree it's difficult to measure.
I actually don't relate to this "bosses don't know what software engineering is" thing at all. All my bosses are older-school engineers-turned-management who are more skeptical of this than the young'uns.
That's still bad, though, because your boss makes hiring and salary decisions.
Maybe in the end it's a kind of "great filter" - companies and bosses that can't tell the difference don't survive. But a lot of good programmers can get fired before the clueless bosses die off...
The cause of the inefficiency and "great filter" you're describing is the labor protectionism currently in the U.S.
Theoretically a good programmer has arbitrage via their skills and expertise. If you can get things done people will pay you for it, assuming you're allowed to work in the U.S. (or EU, etc.).
Not a meritocracy, a market. "Meritocracy" implies some kind of value judgement (ex: "they are paid fairly" or "paid what they're worth"). The fundamental question is about the labor market and its efficiency.
I'll argue: your value in the market -- as an experienced engineer worth $230K or whatever ridiculous thing it is now -- is if you can go somewhere else that's willing to pay you $230K+ for your labor. The reason salaries are this number (despite antitrust things like [1]) is that some people actually exercise this option and move to higher paying jobs. And most employment is at-will -- you can leave whenever you want. So the market is at least somewhat efficient in this sense.
But efficiency works both ways. If you think your labor is worth more than what your employer is paying you, then you have to prove it by asking the market.
[1]: https://en.m.wikipedia.org/wiki/High-Tech_Employee_Antitrust...