LLMs are a communication technology, with a huge trained context of conversation. They have a long way to go before becoming anything intelligent.
But LLMs aren't "only good at stuff that better languages don't require you to do." In fact they are very good at taking a bad function definition and turning it into an idiomatic one that does what I wanted to do. That's very intelligent, there is no language that can take a bad spec and make it specific and fit for the specified task. LLMs can. (not perfectly mind you, but faster and often better than I can.) The problem is they just can't always figure out when what they've written is off-spec. But "always" isn't "never" and I've yet to meet an intelligence that is perfect.
That is perhaps the biggest weakness I've noticed lately, too. When I let Claude Code carry out long, complex tasks in YOLO mode, it often fails because it has stopped paying attention to some key requirement or condition. And this happens long before it has reached its context limit.
It seems that it should be possible to avoid that through better agent design. I don't know how to do that, though.
AI can do 80% of the work. I can review it later. And I spend much less time reviewing than I would have typing up everything manually.
I recently used it to add some logging and exception handling. It had to be done in multiple places.
A simple 2 line prompt one shotted it. Why do I need to waste time writing boring code?
We will be focusing more higher level design - which database, where the data flows, which service is used where and so on. So you will just need different skills. Coding as a skill won't be that important.
It always turns out that when you have to make a computer execute a very precise set of operations, the absolute best way to communicate this is not English, not diagrams, but code.
It also turns out that the challenge is generally not writing that code, but figuring out that precise set of operations in the first place.
But you just said
> Coding as a skill won't be that important.
So coding is only important if you need to make your software actually work, I guess?
Just like I sometimes need to understand and dive into assembly but not always. Its a spectrum and the value in the spectrum changing has consequences for how we work.
I'm just going to leave this here...
On the foolishness of "natural language programming".
- Dijkstra
https://www.cs.utexas.edu/~EWD/transcriptions/EWD06xx/EWD667.htmlMany of these tools, have started to go one step further, and the traditional ways are supplanted with AI, reducing even further the amount of low code/no code scenarios that a human has to produce.
The only thing left are the few serverless deployments to call into some code that needs more complex integration logic, and that is about it.
I have seen enough of these projects, to realize it is going to happen, not everywhere maybe, but in specific industry sectors certainly, it already is.
Yes, all those ever-growing layers of intricate abstraction that you take for granted and “don’t have to worry about” are conceived of, designed, built, and maintained by developers. Who do you think wrote the compiler for that syntax you don’t want to learn?
No one in my company writes assembly. very few developers work at that level of abstraction - this means those who made the compilers are doing a good job.
That’s kind of my point: most people will work on higher abstractions but there will be some who maintain lower ones.
I write C# but I barely care about memory, gc nor microcontrollers nor assembly. Vast majority of people work on higher abstractions.
Individual lines of code can bring down an entire system.
> and syntax
I’ve never seen this be a real issue even for lower performing devs.
I don’t worry about assembly because the abstraction layer is reliable. I don’t worry about ISAs, ucode, transistors, etc. the abstraction layer is reliable.
The same is not true for LLMs today. Circumstantial evidence: people commit the lower layer. That’s not an abstraction any more than an IDEs tab complete or “new project template” is an abstraction.
When someone stops reading the output entirely and has a codebase that is only prompts, I’ll hear them out on skill obsolescence.
(Edit: changed to hypothetical)
Some people actually enjoy that, believe it or not.
honestly I've largely stopped coding for fun since Claude Code got popular. It's too expensive to use for personal projects, and it does the fun part. I don't want to pay but if I'm not using it, all I can think about is how inefficient doing everything manually is
..
I'm going to get into gardening or something :(
The better question is: should that boring code be written? Code should only be non-boring.
The boredom of writing the code is not the only problem. The subsequent continued indefinite existence of that code is also a problem.
In my experience, AI very often gets into a sort of sunk-cost fallacy (sunk prompt?) and then it is very hard to get it to make significant changes, especially architecturally.
I recently wrote an extension for a popular software product and gave AI the same task. It created a perfectly working version however it was 5x the lines of code of my version because it didn't know the extension API as well, even though I gave it the full documentation. It also hard coded some stuff/solutions to challenges that we totally don't want to be hard coded. A big reason why I arrived at a much better solution was that I used a debugger to step through the code and noted down just the API interactions I needed.
The AI also was convinced that some things were entirely impossible. By stepping through the code I saw that they would be possible by using parts of the internal API. I suggested a change to make the public API better for my use case in a GitHub issue and now it is totally not impossible.
At the end of the day I have to conclude that, the amount of time invested guiding and massaging the AI was too much and not really worth it. I would've been better off debugging the code right away and then creating my own version. The potential for AI to do the 80% is there. At this time though I personally can't accept its results yet but that may also be due to my personal flavour of perfectionism.
Are you sure you're actually reviewing the code? Deeply, properly reviewing and understanding it? Because from what I've seen people that say they do, don't. That's why they 'speed up' from using LLM-generated code.
The old adage that it's far harder to review code than to write it still holds true.
I think the argument being put forward here is that writing that boring code is part of the journey to mastery. If you haven't crossed the inflection point, a backwards slide in skills will result in less competence and more confidence, which is a scary thought given how software runs the world.
What happens when all pain and boredom is taken away? Why do we need to invent new frameworks, systems, design patterns in that scenario?
So maybe LLMs will win on a technicality by making us more stupid as a species.
What? So Your Jeep Compass is higher quality than a 458?
I have seen people bring in thousands of lines of opencv lut code in ai slop form because they didnt understand how to interpolate between two colors and didnt have the experience to know that is what they needed to do. This is the catch 20/20 of the ai expert narrative.
The other part is that improvement has massively stagnated in the space. It is painfully obvious too.
I think there is something to this line of thinking. I just finished a bigger project and without going into details, one person from team supposedly dedicated to providing viable data about data was producing odd results. Since the data was not making much sense, I asked for info on how the data was produced. I was given SQL script and 'and then we applied some regex' explanation.
Long story short, I dig in and find that applied regex apparently messed with dates in an unexpected way and I knew because I knew the 'shape' that data was expected to have. I corrected it, because we were right around the deadline, but.. I noted it.
Anyway, I still see llm as a tool, but I think there is some reckoning on the horizon as:
1. managers push for more use and speed given that new tool 2. getting there faster wronger, because people go with 1 and do not check the output ( or don't know how to check it or don't know when its wrong )
It won't end well, because the culture does not reward careful consideration.
To be clear here i give the spec to ai many times asking what was off and it never found the issue.
Once i did get it working, ai one shotted converting it from python to go with the exception of the above mistake being added back in again.
You dont know what you dont know. That final 25% or 5% or whatever is where the money is at, not the 80%. Almost doesnt count.
... and can now measure my productivity in dependencies added per day! :)
I mean I'm opposed to all work, but the transition to jobless society might be traumatic. I hope you didn't neglect to buy crypto because that's the only remaining way to spontaneously and easily tap into the wealth of the rich when the work is gone.
I highly doubt the best of the best folks are even touching LLMs (barely) because they can see the flaws and the tradeoffs that are not-so-visible to others until they are 50 levels deep, with sunk investments, unwilling to go back to how they used to do things.
> Engineers are stuck in the old paradigm of "perfect" algorithms.
Reminds me of a misinterpretation of Knuth. > Premature optimization is the root of all evil.
He was definitely knocking engineers for wanting to write "perfect" algorithms, but this quote also got bastardized to mean something different from what he said (happens to many clichés). All he said was "grab a fucking profiler before you optimize."But now, I'm not sure a lot of programmers even know what a profiler is. When was the last time you saw someone profile their code?
Problem is we've taken the idea of "minimum viable product" too far. People are saying "Doesn't have to be perfect, just has to work." I think most people agree. But with the current state of things? I disagree that things even work. We're so far away from the question of optimization. It's bad enough that there are apps that require several gigs to just edit a 30kb document but FFS we're living in a world where Windows Hello crashes Microsoft Outlook. It's not the programs are ugly babies that could be better, they are monstrosities begging to be put to death.
I WISH we could talk about optimization. I WISH our problem was perfectionism. But right now our problem is that everything is a steaming pile of garbage and most people are just shrugging their arms like "it is the way it is". Just because you don't clean up that steaming pile of garbage doesn't mean someone else doesn't. So stop passing the buck.
LOL. OMG that was beautiful. It almost feels like we are de-evolving software to a state where shit is going to stop working bad. I know this is not full of facts, but this take reminds me of Jonathan Blow's video "Preventing the Collapse of Civilization"[0] Where he talks about how code runs worse than it ever has and I think he was arguing that civilization is collapsing before our eyes in slow time.
[0]: https://youtu.be/pW-SOdj4Kkk?si=LToItJb1Cv-GgB4q&t=1089
Honestly, I think the problem is that it's a Lemon Market[0]. Lemon markets thrive when there is asymmetric information. When a customer cannot tell the difference between a good product (peach) and a bad product (lemon). All it takes is a bunch of tech illiterate people... not sure where we'll find those...
On your video, funny thing. When I was in my PhD I had a very hard time publishing because I was building models that were much smaller, required less data, but got similar performance. Reviewers just looked at the benchmark like "not SOTA? lol". I've seen tons of great papers solving similar problems constantly get rejected. As a reviewer I frequently defended works like that as well as works that had good ideas but just didn't have enough GPU power. It was really telling...
[0] https://en.wikipedia.org/wiki/The_Market_for_Lemons
[P.S.] A nice solution I found for the pasting problem he mentioned (and in various forms) is that I first paste the text into the url bad or search bar then copy that and then paste. {<c-k>,<c-l>}<c-v><c-a><c-c>. Works 98% of the time every time.
1. Every single measurement in a timeseries is encoded as a utf-8 string having (roughly) the following format:
"${name}:${value}|${type}|${tags}"
where name is like "my.long.namespace.and.metric.name", value is a string formatted number, god only knows what type is, and tags is some gigantic comma separated key:value monstrosity.2. Each and every one of these things is fired off into the ether in the form of a UDP datagram.
3. Whenever the server receives these presumably it gets around sometime to assigning them timestamps and inserts them into a database, not necessarily in that or any other particular order.
"it is the way it is[1]."
[1] https://github.com/statsd/statsd?tab=readme-ov-file#usage
A year ago. I heavily relied on one to optimize a complex data import that took an hour for a million line Excel file. The algorithm translated it to a graph according to a user-specified definition and would update an existing graph in neo4j, keeping the whole thing consistent.
The only other guy who understood the algorithm (a math PhD) thought it was as optimal as it could get. I used the profiler to find all the bottlenecks, which were all DB checks for the existence of nodes, and implemented custom indices to reduce import time from an hour to 3 minutes.
It did introduce a bunch of bugs that I had to fix, but I also discovered some bugs in the original algorithm.
It was one of my best programming experiences ever. Especially the payoff at the end when it went down from an hour to 3 minutes is a dopamine rush like never before. Now I want to optimize more code.
I don't think users cared, though; originally this work would take days by hand, so an hour was already pretty good. Now I made something fiendishly complex look trivial.
> from an hour to 3 minutes
I sure bet that the users cared. Yeah, starting from a few days an hour feels great but you also get accustomed to it. > It did introduce a bunch of bugs that I had to fix, but I also discovered some bugs in the original algorithm.
I find this is extremely common when I profile code. It is just so easy to miss bugs. People get lulled into a false sense of security because tests pass but test just aren't enough. But for some reason when I say "tests aren't enough" people hear "I don't write tests."Seeing those big improvements and knowing you did more than make it faster is always really rewarding. I hope you do do more optimization :) Just remember Knuth's advice. Because IO is a common problem and Big O isn't going to tell you about that one haha
> People get lulled into a false sense of security because tests pass but test just aren't enough.
Users weren't using a particular feature because they said they didn't understand it. So we explained it, again and again. Turns out that feature was incredibly buggy and basically worked the way we claimed it did, only when it was used in the specific configuration we tested for. Add another node somewhere and weird stuff starts happening.
The tests looked good, and code coverage was great, but the fact that the tests run through all the branches of the code doesn't mean you're really testing for all behaviour. So I added tests for all configurations I could think of. I think that revealed another bug.
So look at the actual behaviour you need to test, not merely the code and branch coverage.
> Yeah, I first want to know there's an actual performance issue to fix.
Honestly, I think profilers and debuggers can really help with this too. > So I added tests for all configurations I could think of.
I think that's the key part. You can only test what you know or expect. So your tests can only be complete if you're omniscient.I love how you put it: "grab a fucking profiler before you optimize". I get complaints sometimes about using FP because of performance, and I think a variant of "grab a fucking profiler before you optimize" is much better response than "avoid premature optimization". Introducing them to a magical thing called as "profiler" is a nice bonus too.
Or alternatively, we will build bigger and better things with the power of AI. Everyone talks about it replacing us, but we aren't running out of things to build. It's not like we're gonna run out of ways to improve the world, and compared to other things the digital world is unconstrained.
I think the image you post at the beginning basically sums it up for me: ChatGPT o3/5 Thinking can one-shot 75% of most reasonably sized tasks I give it without breaking a sweat, but struggles with tweaks to get it to 100%. So I make those tweaks myself and I have cut my code writing task in half or one third of the time.
ChatGPT also knows more idioms and useful libraries than I do so I generally end up with cleaner code this way.
Ferrari's are still hand assembled but Ford's assembly line and machines help save up human labor even if the quality of a mass-produced item is less than a hand-crafted one. But if everything was hand-crafted, we would have no computers at all to program.
Programming and writing will become niche and humans will still be used where a quality higher than what AI can produce is needed. But most code will be done by minotaur human-ai teams, where the human has a minimal but necessary contribution to keep the AI on track... I mean, it already is.