For now. Not sure why you think that would last though. Obviously the big labs are going to make it easy to use AI both safely and correctly. It's already significantly easier to do that today than a year ago.
I no longer really check the code, but I won't let any of them touch the database schemas. I modify the schema, the machine codes around my database. Same policy with APIs, even private ones.
I second this. Even OpenAI's flagship models... the code I write by hand has much less entropy and is easier to follow than the equivalent AI-generated code. I think part of the software engineering skill now is knowing which pieces you can delegate to AI and which still need human input. I know many people who just YOLO it all and have AI do everything... but, this can hardly make for a stable system in my opinion...
And, I'll be happy to be proven wrong, because it will mean a significant amount of work is off of my shoulders. But I think we aren't quite there yet in terms of having AI code that lowers the complexity instead of raising it in a codebase.
> Why are you looking at the code? Complexity? What do you care? Let the agent handle that.
Because when I don't look at the code, bad things happen. Outages, customer data is deleted, security issues.
I can hear the reply: "Why not let agents review the code?" - that's true, but at some point my boss is going to hold me responsible, and I'm going to have to answer for it. Both ethically and practically speaking I have to make the responsible engineering decision and actually own my actions.
The failure modes of the LLM are still the same. Nothing has fundamentally changed, despite the UI and the harness showing significant improvements and outcomes.
More and more it would appear the failure modes are a training data issue
Of course you do. Just because you don’t see the LLM trying the ls command for the billionth time with the wrong arguments, just to be corrected by the harness, doesn’t mean it doesn’t happen. Of course I use the ls cli as an example. Could be any task. That’s what I mentioned about the harness and that’s why they shine in the fields with hard verification mechanisms and feedback loops.
(this has been my experience since ~DSv4 from April)
It’s not the harness that attempts to fix the broken tool calls. It’s the LLM.
> last time I saw a tool call loop
We are not talking about tool call loops
Then what did "Just to be corrected by the harness" mean?
> last time I saw a tool call loop
Reasoning loops, output loops, etc, I just don't tend to see them with the models I use.
All I'm saying is that I find LLMs to have been consistently getting more reliable, to the point that their failure modes don't really stand out.
If you actually start looking inspecting what the boxes do, it is a scary place. The Chinese models are open source open weights and are trained on lies. This much can be proven. But the frontier labs have made it impossible to determine the provenance of their models. They have been fed with copyrighted works, but good luck proving that your copyrighted work lives in their weights.
In short, the frontier labs are operating opaquely, and that's very good reason not to trust them.
There is no reason why AIs wouldn't be able to replace CEOs. It'd be a crime against humanity for us to stop just short of replacing executives.
They'll go, but they'll be the last ones.
The only thing that won't ever be automated is ownership of property, which is the only thing that's sacred to the wealthy.
You know, it occurs to me… it might be interesting (and funny) to setup some kind of LLM hive mind consisting of agents assigned various asinine corporate roles (“you are hr bot”, “you are cto”, etc…). Feed random problems into it and see what it attempts to do.
An architect friend just released a full game on Steam, and it’s making money.
He has never written code is his life, has no idea what a database schema is, or exceptions or even a shell script. Never heard of git, ssh or really anything else. AI wrote everything, even the textures, sound effects, music, etc.
Coding is done.
This however will put salary pressure on the people who were simply digging holes or welding beads without being as involved in the planning and design work. Time to remember that we can't forget the business part!