These models are actually extremely good but they are far from an intelligence unto themselves. Truth is if someone told you they could build these things 5 years ago, you d write them a check for a trillion dollars. Problem is once we got them, we realized they are not all that. Its like a mecha suit in a universe, where mecha suits are abundant and cheap. Someone has to climb into them everyday and put in the work for it to be effective.
So now the skeptics are saying this technology is overrated. And the optimists are accusing the skeptics of moving goal posts.
Humans only what they know, until they acquire more information about what's possible.
The goal post narrative is stupid to begin with.
Isn't this just the hype cycle? [1]
Fake edit: I know its not a perfect model.
1: https://www.gartner.com/en/research/methodologies/gartner-hy...
If they get to the point where they're smart enough to make tasteful code decisions based on stakeholder input... we're cooked as a profession.
Dramatically improved my static site generator to the point it's better than markdown and added Atom and RSS feeds, used it to write several articles about my language. Pace is so fast I actually need to write those articles by hand in order to crystalize the knowledge I learned. If I don't I'm afraid I'll just forget everything. No LLMs for the articles.
Also created my own invoicing system for $DAYJOB so I can invoice companies from my terminal. Started a decompilation project for my cherished childhood games and I've already almost finished decompiling one game's engine after just a few days. Been working on my cyberdeck project too, this one's a bit slow because I got to the point where I'll actually need to spend money on it to move forward. All this inside the rootless development virtual machine system built on top of QEMU and systemd that I developed together with Claude, whose network isolation I'm currently hardening. Started reverse engineering my laptop again! And I'm actually making progress! Made a color scheme app for the keyboard LEDs controller I made many years ago, with loads and loads of color schemes! Found some kind of bug in my keyboard while doing it, in less than an hour I had the root cause and a fix applied locally, sent the fix to systemd, it got merged. Planning to ramp up my free and open source software participation as well now that exploring codebases is a breeze. Already have some mesa patches ready for upstream. Have been playing with strace since I use it so much.
Better?
There is ZERO chance I would ever be able to complete it on my own.
I doubt it'll get traction, but if it doesn't, I am pretty confident a future language will take the ideas for polymorphic synchronization and profile-guided optimization.
It has an easy version/mode of compilation that makes Rust's affine ownership accessible like a high-level scripting language, and it can progressively become more strict, where the compiler does ~99% of the work for you, and you just pick options as it finds issues (that it explains to you like you're 5) along the way.
Along the way, I also built a suite of tools that helps identify complexity better than anything I've seen (which was necessary to get the LLMs to be able to unslop themselves and write something that actually works).
I doubt the Ruby community shrugs it off, but time will tell.
Rust had memory safety bugs well after release - IIUC all the way until after the 1.0 release.
So, it's highly unlikely to be perfect, but I think it'll be in better shape than Go or Rust were when they initially launched.
NLP as a field saw huge shifts. NLP tasks that used to be complex and inaccurate can now be setup very easily and quickly using structured outputs from LLMs, often with greater accuracy.
A small charity I help with has now been able to build their own website to manage their day-to-day operations. It saves them a lot of time, and it was vibe-coded using Manus. I don't think people appreciate how much room there is left for bespoke software to have big impacts on small organisations that can't afford to hire developers. The cost for software like the one they made has gone from 10s of thousands of dollars to $10/month and volunteer hours.
My brother has recently been setting up Cowork to do an automatic review of contracts before human review, and he said it is far more diligent than people when it comes to routine things to check. This is another huge breakthrough for not just efficiency, but the quality of work.
I really don't think we can discount AI finding bugs and vulnerabilities. If you care about code quality and keep up review standard, LLMs can help you write more robust software. AI has found a huge number of bugs for me before they hit production, including potential out-of-bounds memory accesses and segfaults.
ChatGPT has 1 billion MAU. People are now getting life advice, financial advice, and mental health help from chatbots at a scale and cost that no human support network could match.
Personally not the kind of breakthrough I'm psyched about
Also, they have done a good job shutting down the psychotic behavior you could get from 4o era models. If there are remaining issues like that they ought to fix them too.
That's terrifying.
You realize that's terrifying, right?
If the only breakthrough is automated coding with no outside consequence then it’s just masturbation
The rate of improvement has been fast. Maybe it’ll plateau soon, or maybe we’ll have LLMs improving themselves rapidly. At this point it’s too early to say.
I don’t remember where I heard it, but there’s a saying that people overestimate how much can be accomplished in a year and underestimate how much can be accomplished in 10 years.
If we get to 2030 and still people are wondering where the breakthrough is, then I think I’d be agreeing with your skepticism. But I just think it’s too early to judge that yet.
But the clock is ticking.
Built a bunch of software tools to streamline my small ecommerce business - while also running it - and things have turned around from "losing money and ready to pull the plug" to "looking at our best financial year on record" in the span of about 8 months.
I could imagine it wouldn't make a huge difference to the life of someone deeply entrenched in a traditional tech role, trying to get an extra 9 of reliability in a service or roll out a new carefully planned and QA'd feature.
But for tech-adjacent people, it gives us something "good enough", instantly, and basically for free.
That doesn't include the other things I've got it to do (gave Claude SSH access and got it to successfully debug a hang on my Ubuntu server, chucked Codex in a folder full of financial data and got it to find every piece of misclassified payroll transaction data)
Genuinely the biggest breakthrough for "casual" tech users since Excel.
edit: it looks like I was wrong and they're still hiring many software engineers. Not completely sure why that is just yet.
I don’t publish them - but they’re put into use in production and they provide a tangible benefit that would not exist otherwise.
I especially love how making a nicely styled website these days is a matter of describing what it looks like and waiting 10-15 minutes. There are other examples
But the OP is claiming 10x productivity improvements along some metrics. If that was even slightly true under even a generous interpretation of what it might mean, I’d expect an actual breakthrough, not the ability to churn out little things
- The first web browser
- the first web browser with images
- typescript
- react
- rust
- Fil-C
- doom
- quake
- the anamorphic VM, and its follow-ups like HotSpot, and even competitors/copycats like J9, V8, JSC, etc
- Fortnite battle royale
- Roblox
- thefacebook
- ChatGPT
- Claude code
I know that’s quite a range and that’s intentional.
Anyway, I think we’ll know it when we see it.
These are new products (generally) and that's a different class of problem.
It is possible that since LLM+harness helps with execution then we should see more experiments.
For example NPCs in games that have complexity that previously was not possible.
Good games often push the boundaries a bit, so should be a good example.
Of course now we can start arguing that there isn't a lot of investment into gaming currently, because it all goes into AI. Too bad.
- Complete GuileMacs, the Guile implementation of Emacs. As AI is supposedly much more capable than Humans, it would be great if the above mentioned implementations are even more efficient and feature rich than Emacs!
- Something like Android (maybe even a clone?) with the Java Layer removed and replaced with CL and with Linux kernel still intact. Basically CL over Linux as opposed to the Java over Linux in Android.
- For fun, an implementation of the Lisp machines' OS with Lisp all the way down though Assembly is allowed for critical pieces. It should be a full blown modern Desktop with equivalents of what users expect from a modern OS ...
To play devils advocate, computers didn’t translate to massive productivity gains until long after businesses adopted them. There was that quote from ’87: "you can see the computer age everywhere but in the productivity statistics"
Maybe we’re seeing something like that right now with AI?
Who knows man
Personally, I'm seeing massive improvements to my workflow and the quality of the product I'm shipping. I'm using AI to crank out far more tests than I used to be able to write, and I am using AI to analyze results with far more fidelity and speed than I could ever have done myself. That means I have more quality time.
But this will change, because the meaning of software development will change to expect, nay to require AI use. I've heard this is already happening at e.g. Google. The expectation of what can be achieved by tinkerers and by professionals will change. The expectation of what it means to interact with software via your own agents will change and will become commonplace. Apple still hasn't figured out the local agent on the iPhone, but they will. 2027 is not going to feel at all like 2025.
But is any of that a fundamental change? It sure feels fundamental to me, but maybe that's because my everyday has totally changed, but the product I am responsible for has not. Yet. The product I am responsible for operates in critical infrastructure where I personally hope AI never has deep roots, but maybe that's just me. I don't think using AI to build a system that is offline from any AI is the same as depending on an AI to make realtime decisions for critical infrastructure.
For now... the shareholders demand managers get the max out of every employee. Throw the force of competition etc into the mix and yeah labour isn't going to benefit all that much.
Its yet to be determined just how 'efficient' people are with LLM's as its not really a one-person thing - the true measure is based on an entire collection of people's output.
Startups being rapidly efficient doesn't mean much in relation to the overall economy.
If you get yourself to define it, maybe you'll find it achievable :)
We implemented that in about three days earlier this year, just by feeding the files to LLMs. And it's good enough to not need a human to check.
I get that this isn't a "Computer Science breakthrough" in the sense you mean, but it used to involve a lot of hard CS to try and solve, and now it doesn't.
Claude is amazing, that’s true.
But if it was as amazing as this article implies, I’d expect some breakthrough outside of AI itself.
Rewriting a Zig program in unsafe Rust? Not a breakthrough. Finding a bunch of security vulns? Maybe that’s sort of a breakthrough though it’s underwhelming and possibly just a net negative. But like if I rolled back to using software from 2023 then life would be ok.
Maybe we just need to give it time, and sometime real soon, we will all be amazed by such a breakthrough? Who knows