My human co-workers generally have good faith. Even the developer who was clearly on the verge of getting a role elsewhere without his heart in it-- he tried to solve the problems assigned to him, not some random delusion that the words happened to echo. I don't have that level of trust with AI.
If there's a misunderstanding the problem or the context, it's probably still the product of a recognizable logic flow that you can use to discuss what went wrong. I can ask Claude "Why are you converting this amount from Serbian Dinars to Poppyseed Bagels in line 476?" but will its answer be meaningful?
Human code review often involves a bit of a shared background. We've been working with the same codebases for several years, so we're going to use existing conventions. In this situation, the "AI knows all and sees all" becomes an anti-feature-- it may optimize for "this is how most people solve this task from a blank slate" rather than "it's less of a cognitive burden for the overall process if your single change is consistent with 500 other similar structures which have been in place since the Clinton administration."
There may be ways to try to force-feed AI this behaviour, but the more effort you devote to priming and pre-configuring the machine, the less you're actually saving over doing the actual work in the first place.
I have been messing around with getting AI to implement novel (to me) data structures from papers. They're not rocket science or anything but there's a lot of detail. Often I do not understand the complex edge cases in the algorithms myself so I can't even "review my way out of it". I'm also working in go which is usually not a very good fit for implementing these things because it doesn't have sum types; lack of sum types oten adds so much interface{} bloat it would render the data structure pointless. Am working around with codegen for now.
What I've had to do is demote "human review" a bit; it's a critical control but it's expensive. Rather, think more holistically about "guard rails" to put where and what the acceptance criteria should be. This means that when I'm reviewing the code I am reasonably confident it's functionally correct, leaving me to focus on whether I like how that is being achieved. This won't work for every domain, but if it's possible to automate controls, it feels like this is the way to go wherever possible.
The "principled" way to do this would be to use provers etc, but being more of an engineer I have resorted to ruthless guard rails. Bench tests that automatically fail if the runtime doesn't meet requirements (e.g. is O(n) instead of O(log n)) or overall memory efficiency is too low - and enforcing 100% code coverage from both unit tests AND fuzzing. Sometimes the cli agent is running for hours chasing indexes or weird bugs; the two main tasks are preventing it from giving up, and stopping it from "punting" (wait, this isn't working, let me first create a 100% correct O(n) version...) or cheating. Also reminding it to check AGAIN for slice sharing bugs which crop up a surprising % of the time.
The other "interesting" part of my workflow right now is that I have to manually shuffle a lot between "deep research" (which goes and reads all the papers and blogs about the data structure) and the cli agent which finds the practical bugs etc but often doesn't have the "firepower" to recognise when it's stuck in a local maximum or going around in circles. Have been thinking about an MCP that lets the cli agent call out to "deep research" when it gets really stuck.
I haven't had to review too much AI code yet, but from what I've seen it tends to be the kind of code review that really requires you to think hard and so seems likely to lead to mistakes even with decent code reviewers. (I wouldn't say that I'm a brilliant code reviewer, but I have been doing open source maintenance full-time for around a decade at this point so I would say I have some experience with code reviews.)
If the engineer, doing the implementation is top-shelf, you can get very good results from a “flawed” process (in quotes, because it’s not actually “bad.” It’s just a process that depends on the engineer being that particular one).
Silicon Valley is obsessed with process over people, manifesting “magical thinking” that a “perfect” process eliminates the need for good people.
I have found the truth to be in-between. I worked for a company that had overwhelming Process, but that process depended on good people, so it hired top graduates, and invested huge amounts of money and time into training and retention.
Its the content that matters, not the process.
The first is phenomenal until someone makes a mistake and brings in a manager or supervisor from the C category that talks the talk but doesn't walk the walk.
If you accidentally end up in one that turns out to be the later. It's maddening trying to get anything accomplished if the task involves anyone else.
Hire slow, fire fast.
This idea is called "evolution"...
> as long as you have good quality control
...and it's QA is death on every single level of the systems: cell, organism, species, and ecosystem. You must consider that those devs or companies with not-good-enough QA will end up dead (from a business perspective).
Sounds like a stupid path forward to me
Like, evolution is not _good_ at ‘designing’ things.
For the core parts you cannot let go of the reins. You have to keep steering it. You have to take short breaks and reload the code into the agent as it starts acting confused. But once you get the hang of it, things that would take you months of convincing yourself and picking yourself back up to continue becomes a day's work.
Once you have a decent amount of work done, you can have the agent read your code as documentation and use it to develop further.
I'm not sure how this translates to programming, code review is too expensive, but for short code you can try https://en.wikipedia.org/wiki/Superoptimization
The way team leads seem to get used is people who are good at code get a little more productive as more people are told to report to them. What is happening now is the senior-level engineers all automatically get the same option: a team of 1-2 mid-level engineers on the cheap thanks to AI which is entirely manageable. And anyone less capable gets a small team, a rubber duck or a mentor depending on where they fall vs LLM use.
Of course, the real question is what will happen as the AIs get into the territory traditionally associated with 130+ IQ ranges and the engineers start to sort out how to give them a bit more object persistence.
Well you've just described an EKF on a noisy sensor.
I mean the average person surely knows at least what a Jacobian is, right? Right? /s
Quality needs to come from the process, not the people.
Choosing to use a process known to be flawed, then hoping that people will catch the mistakes, doesn't seem like a great idea if the goal is quality.
The trouble is that LLMs can be used in many ways, but only some of those ways play to their strengths. Management have fantasies of using AI for everything, having either failed to understand what it is good for, or failed to learn the lessons of Japan/Deming.
Not sure which Japanese school of management you're following, but I think Toyota-style goes against that. The process gives more autonomy to workers than, say, Ford-style, where each tiny part of the process is pre-defined.
I got the impression that Toyota-style was considered to bring better quality to the product, even though it gives people more autonomy.
It's a bit like Warren Buffet saying he only wants to invest in companies that could be run by an idiot, because one day they will be.
Edward Deming actually worked with both Toyota and Ford, perhaps more foundationally at Toyota, bringing his process-based-quality ideas to both. Toyota's management style is based around continuous process improvement, combined with the employee empowerment that you refer to.
You're also describing the software development process prior to LLMs. Otherwise code reviews wouldn't exist.
Use of AI seems to be a regression in this regard, at least as currently used - "look ma, no hands! I've just vibe coded an autopiliot". The current focus seems to be on productivity - how many more lines of code or vibe-coded projects can you churn out - maybe because AI is still basically a novelty that people are still learning how to use.
If AI is to be used productively towards achieving business goals then the focus is going to need to mature and change to things like quality, safety, etc.
With humans and code reviews now two humans looked at it. With LLM and code review of the LLM output now one human looked at it, so its not the same. LLM are still far from as reliable as humans or you could just tell the LLM to do code reviews and then it builds the entire complex product itself.
0. The dawn of video games had many titles with 1 person responsible for programming. This remains the case many indy games and small software apps and services. It's a skill that requires expertise and/or dedication.
Deming’s process was about how to operate a business in a capital-intensive industry when you don’t have a lot of capital (with market-acceptable speed and quality). That you could continue to push it and raise quality as you increased the amount of capital you had was a side-effect, and the various Japanese automakers demonstrated widely different commitments to it.
And I’m sure you know that he started formulating his ideas during the Great Depression and refined them while working on defense manufacturing in the US during WWII.
Third option: they want to automate all jobs before the competition does. Think of it as AWS, but for labor.
Its the same with any "general" tech. I've seen it since genetic algorithms were all the rage. Everyone reaches for the most general tool, then assumes everything that tool might be used for is now a problem or domain they are an expert in, with zero context into that domain. AI is this times 100x, plus one layer more meta, as you can optimize over approaches with zero context.
Normally, if you want to achieve some goal, there is a whole pile of tasks you need to be able to complete to achieve it. If you don't have the ability complete any one of those tasks, you will be unable to complete the goal, even if you're easily able to accomplish all the other tasks involved.
AI raises your capability floor. It isn't very effective at letting you accomplish things that are meaningfully outside your capability/comprehension, but if there are straightforward knowledge/process blockers that don't involve deeper intuition it smooths those right out.
You're probably envisioning a more responsible use if it (floor raising, "meaningfully inside your comprehension"), that is actually not what I'm referring to at all ( "assumes everything that tool might be used for is now a problem or domain they are an expert in"). A meta tech can be used in many ways and yours is close to what I believe the right method is. But I'm asserting that the danger is massive over reliance and over confidence in the "transferability".
Nothing has changed. Few projects start with you knowing all the answers. In the same way AI can help you learn, you can learn from books, colleagues, and trial and error for tasks you do not know.
Before AI, if I had the knowledge/skill to do something on the large scale, but there were a bunch of minute/mundane details I had to figure out before solving the hard problems, I'd just lose steam from the boredom of it and go do something else. Now I delegate that stuff to AI. It isn't that I couldn't have learned how to do it, it's that I wouldn't have because it wouldn't be rewarding enough.
Supermarket vegetables.
Quite a bit of it, like Tomatoes and Strawberries, is just crap. Form over substance. Nice color and zero flavor. Selected for delivery/shelf-life/appearance rather actually being any good.
From an economics POV, that's the correct test.
I was also considering the way the US food standards allows a lot of insect parts in the products, but wasn't sure how to phrase it.
Maybe we could stop filtering everything through this bullshit economics race to the bottom then
A race to the bottom leaves you like Boeing or Intel.
Late stage capitalism is not a must.
Your list of winners are optimising for what the market cares about, exactly like the supermarkets (who are also mostly winners) are optimising for what the market cares about. For most people, for food specifically, that means "cheap". Unavoidably, because most people have less money than they'd like. Fancy food is rare treat for many.
Apple software currently has a reputation for buggy UI; Oracle has a reputation for being litigious; that just leaves Nvidia who are printing money selling shovels in two sucessive gold rushes, which is fine for a business and means my investment is way up, but also means for high end graphics cards consumer prices are WTF and availability is LOL.
See, the only way is not a race to the bottom like all late stage capitalist claim.
What is your point?
I don't know how the US compares to other countries in terms of "insects per pound" standards, but having some level of insects is going to be inevitable.
For example, how could you guarantee that your wheat, pre-milling, has zero insects in it, or that your honey has no bee parts in it (best you can do is strain it, then anything that gets through the straining process will be on your toast).
Even if the process weren’t technically bad, it would still be shit. Doing code review with a human has meaning in that the human will probably learn something, and it’s an investment in the future. Baby-sitting an LLM, however, is utterly meaningless.
2. Quality control is key to good processes as well. Code review is literally a best practice in the software industry. Especially in BigTech and high-performing organizations. That is, even for humans, including those that could be considered the cream of the industry, code review is a standard step of the delivery process.
3. People have posted their GitHub profiles and projects (including on this very forum) to show how AI is working out for them. Browse through some of them and see how much "endless broken nonsense" you find. And if that seems unscientific, well go back to point 1.
So right off the bat, I don't trust you. Anyway, I picked one study from the search to give you the benefit of the doubt. It compared leetcode in the browser to LLM generation. This tells us absolutely nothing about real world development.
What made the METR paper interesting was that they studied real projects, in the real world. We all know LLMs can solve well bounded problems in their data sets.
As for 3 I've seen a lot of broken nonsense. Let me know when someone vibe codes up a new mobile operating system or a competitor to KDE and Gnome lol
Alternatively, a search is a way to show that basing your opinions on limited personal experience or a single source is silly given the vast amount of other research out there that largely contradicts it. Worse if that single source itself happens to have flaws that are not sufficiently discussed, e.g. at least one of the 16 participants from the METR study deliberately filtered out large tasks that he strongly prefered do only with AI -- what does that mean for its results?
https://xcancel.com/ruben_bloom/status/1943536052037390531
> Many of the studies in that search don't have anything to do with programming at all.
That's fair, but unfortunately due to the limits of keyword search. For instance "medical coding" is not programming-related at all, but is being impacted by LLMs and gets caught in the keyword search ¯\_ (ツ)_/¯
Anyway if your preference is "real-world projects", here are a couple specific studies (including one that I had already separately mentioned in the linked comment) at much larger scales that show significant productivity boosts of LLM-assisted programming at doing their regular, day-job tasks:
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4945566 (4867 developers across 3 large companies including Microsoft)
https://www.bis.org/publ/work1208.pdf (1219 programmers at a Chinese BigTech)
There are many more, but I left them out as they are based on other methodologies such as the use of standardized tasks for better comparability, or empirical analysis of open-source commits, or developer surveys, or student projects, which may get dismissed as "not an RCT on real-world tasks." Interestingly they all show comparable, positive results, so consider that it's not as straightforward to dismiss other studies as being irrelevant.
This is exactly the point of corporate Agile. Management believes that the locus of competence in an organization should reside within management. Depending on competent programmers is thus a risk, and what is sought is a process that can simulate a highly competent programmer's output with a gang of mediocre programmers. Kinda like the myth that you can build one good speaker out of many crappy ones, or the principle of RAID which is to use many cheap, failure-prone drives to provide the reliability guarantees of one expensive, reliable drive (which also kinda doesn't work if the drives came from the same lot and are prone to fail at about the same time). Every team could use some sort of process, but usually if you want to retain good people, this takes the form of "disciplines regarding branching, merging, code review/approval, testing, CI, etc." Something as stifling as Scrum risks scaring your good people away, or driving them nuts.
So yes, people do expect it to work, all the time. And with AI in the mix, it now gains very nice "labor is more fungible with capital" properties. We're going to see some very nice, spectacular failures in the next few years as a result, a veritable Perseid meteor shower of critical systems going boom; and those companies that wish to remain going concerns will call in human programmers to clean up the mess (but probably lowball on pay and/or try to get away with outsourcing to places with dirt-cheap COL). But it'll still be a rough few years for us while management in many orgs gets high off their own farts.
I predict many disastrous "AI" failures because the designers somehow believed that "some humans capable of constant vigilant attention to detail" was an easy thing they could have.
The pattern I see over and over is a team aimlessly putting a long through tickets in sprints until an engineer who knows how to solve the problem gets it on track personally.
In a strange kind of analogy, flowing water can cause a lot of damage.. but a dam built to the right specification and turbines can harness that for something very useful… the art is to learn how to build that dam
Also, a fab is intended to make fully functioning chips, which sometimes it fails to achieve. An LLM is NOT designed to give correct output, just plausible output. Again, not the same goal.
LLMs are an implementation of bogosort. But less efficient.
Or more broadly, the existence of complex or any life.
Sure, it's not the way I would pick to do most things, but when your buzzword magical thinking so deep all that you have is a hammer, even if it doesn't look like a nail you will force your wage slaves to hammer it anyway until it works.
As to your other cases.. injection molded plastic parts for things like the spinning t bar spray arm in some dishwashers. Crap molds, pass to low wage or temp to razorblade fix by hand and box up. Personally worked such a temp job before, among others so yes that bad output manual qc and fix up abounds still.
And if we are talking high failure rates... see also chip binning and foundry yields in semiconductors.
Just have to look around to see the dubious seeming is more the norm.
Imagine that your boss came to you, the tech lead of a small team, and said “okay, instead of having five competent people, your team will now have 25 complete idiots. We expect that their random flailing will sometimes produce stuff that kinda works, and it will be your job to review it all.” Now, you would, of course, think that your boss had gone crazy. No-one would expect this to produce good results. But somehow, stick ‘AI’ on this scenario, and a lot of people start to think “hey, maybe that could work.”