https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-a...
“The purpose of abstraction is not to be vague, but to create a new semantic level in which one can be absolutely precise.” — Edsger Dijkstra
The abstraction has never been this leaky or vague — Me
I think this is the "closing the loop" ( https://en.wikipedia.org/wiki/Control_loop#Open-loop_and_clo... ) moment for coding AI.
All pieces are there, we just need to decide to do it. Today's AI are able to produce an increasing tangled mess of code. But it's also able to reorganize the code. It's also capable of writing test code, and assess the quality of the code. It's also capable to make architectural decision.
Today's AI code, is more like a Frankenstein's composition. But with the right prompt OODA loop and quality assessment rigor, it boils down to just having to sort and clean the junk pile faster than you produce it.
Once you have a coherent unified codebase, things get fast quickly, capabilities grows exponentially with the number of lines of code. Think of things like Julia Language or Wolfram Language.
Once you have a well written library or package, you are more than 95% there and you almost don't need AI to do the things you want to do.
> All pieces are there, we just need to decide to do it.
Another silver bullet.
You've got to bite the bullet at one point and make the transition from open-loop to closed-loop. There is a compute cost associated to it, and there is also a tuning cost, so it's not all silver lining.
>Once you have a well written library or package, you are more than 95% there and you almost don't need AI to do the things you want to do.
That's an idealistic view. Packages are leaky abstractions that make assumptions for you. Even stuff like base language libraries - there are plenty of scenarios where people avoid them - they work for 9x% of cases but there are cases where they don't - and this is the most fundamental primitive in a language. Even languages are leaky abstractions with their own assumptions and implications.
And these are the abstractions we had decades of experience writing, across the entire industry, and for fairly fundamental stuff. Expecting that level of quality in higher level layers is just not realistic.
I mean just go look at ERP software (vomit warning) - and that industry is worth billions.
The software development is absolutely a fractal. In 1960s we were solving the complexity by using high level language that compile to machine code to enable more people write simple code. This has happened again and again and again.
But different generations face different problems, which requires another level of thinking, abstraction, and push both boundaries until we reach the next generation. All of this is not solved by a single solution, but the combination based on basic principles that never changes, and these things, at least for now, only human can do.
For example, the jump in productivity from adding an operating system to a computer is orders of magnitude larger than adding an LLM to a web development process despite the LLM requiring infrastructure that cost tens of billions to create.
It seems that while tools are getting more and more sophisticated, they aren’t really resulting in much greater productivity. It all still seems to be resulting in software that solves the same problems as before. Whereas when html came around it opened up use cases that has never been seen before despite being a very simple abstraction layer by today’s standards.
Perhaps the opportunities are greatest when you are abstracting the layer that the fewest understand when LLMs seem to assume the opposite.
Another funny thing is that we are using LLM to replace creative professionals, but the real creativity is from human experience, perception and our connections, which are exactly missing from LLM.
Not a robot to do my art so I can spend more time on dishes, laundry and cleaning.
What important problem do you have for which "AI generated art" is the answer?
You seriously want to claim you don't already have enough "content" to waste your free time consuming?
Curation of content is also a problem, but if we can come up with better solutions there, generative AI will absolutely result in more and better content for everyone while enabling a new generation of creators.
AI is more likely to contribute to the 1000s.
There is unlimited content online, that doesn't mean theres 100 movies worth watching in any given year. Maybe not even 10.
Content creation is not the problem
Content curation is
Would you rather write some code, or constantly code-review verbose, sneakily wrong junior level code?
https://www.physicalintelligence.company/blog/pi0
Software moves faster than hardware
Writing dumb scripts that can call out to sophisticated LLMs to automate parts of processes is utterly game changing. I saved at least 200 hours of mundane work this week and it was trivial.
I think you're also right about LLMs. I think path forward in programming is embracing more formal tools. Incidentally, search for method references is more formal than grepping - and that's probably why people prefer it.
The issue with LLMs is they enshrine the status quo. I don't want ossified crappy software that's hard to work with. Frameworks and libraries should have to fight to justify their existence in the marketplace of ideas. Subverting this mechanism is how you ruin software construction.
I think this analogy is more apt than you may realize. Just like a fractal, the iterated patterns get repeated on a much smaller scale. The jump to higher-level languages was probably a greater leap then the the rest of software innovation will provide. And with each iterative gain we approach some asymptote, but never get there. And this frustration of never reaching our desired outcome results in ever louder hype cycles.
I get your sentiment, I've been through a few hype cycles as well, but besides learning that history repeats itself, there is no saying how it will repeat itself.
I don't know why this is a bad thing. I don't think projects that you believe are nonsensical shouldn't exist just because of your opinion, especially if they're helping people survive in this world. I'm sure the people working on them don't think they're nonsensical.
Rise of antidepressants imply otherwise.
The more bullshit we stop doing, the more energy there is for the awesome things we could do instead.
The arts have a place in society. Tackling real problems like hunger or health do too, arguably more so - they create the space for society to tolerate, if not enjoy art.
But the down side is we have a huge smear of jobs that either don't really matter or only matter for the smallest of moments that exist in this middle ground. I like to think of a travel agent of yesteryear as the perfect example: someone who makes a professional experience of organising your leisure so you don't have to; using questionable industry deals. This individual does not have your consumer interests at heart, because being nice to you is not where the profit is generally. The only role they actually play is rent seeking.
Efficiency threatens the rent seeking models of right now, but at the same time leads to a Cambrian explosion of new ones.
Yet society and economy keeps going and nobody apart from some academic discussions really cares. I mean companies have 100% incentive to trim fat to raise income yet they only do the least minimum.
I don't know this reference, so I have to ask: Was "accidental" supposed to be "incidental"? Because I don't see how "accidental" makes any sense.
Chapter 16 is named "No Silver Bullet—Essence and Accident in Software Engineering."
I'll type out the beginning of the abstract at the beginning of the chapter here:
"All software construction involves essential tasks, the fashioning of the complex conceptual structures that compose the abstract software entity, and accidental tasks, the representation of these abstract entities in programming languages and the mapping of these onto machine languages within space and speed constraints. Most of the big past gains in software productivity have come from removing artificial barriers that have made the accidental tasks inordinately hard, such as severe hardware constraints, awkward programming languages, lack of machine time. How much of what software engineers now do is still devoted to the accidental, as opposed to the essential? Unless it is more than 9/10 of all effort, shrinking all the accidental activities to zero time will not give an order of magnitude improvement."
By accidental, he means "non-fundamental complexity". If you express a simple idea in a complex way, the accidental complexity of what you said will be high, because what you said was complex. But the essential complexity is low, because the idea is simple.
Anniversary edition, p182.
"... let us examine its difficulties. Following Aristotle, I divide them into essence - the difficulties inherent in the nature of the software - and accidents - those difficulties that today attends its production but that are not inherent"
I have almost a dozen viz books. Some written over 50 years ago.
While they impart knowledge, I want the knowledge but also the application. I'm going to go out and paint that bike shed. You can go read Tufte or "Show me the Numbers" but I will show you how to get the results.
Right there is your problem. Read the Mythical Man-Month and Design of Design. They are not long books and it's material that's hard to find elsewhere. Old rat tacit knowledge.
In an era when UIs become ever more Hieroglyphic(tm), Aesthetical(tm), and Nouveau(tm), "AI" revolutionizing and redefining the whole concept of interacting with computers as "Just speak Human." is a wild breath of fresh air.
It's akin to how everyone can build a shelter, but building a house requires a more specialized knowledge. The cost of the later is training time to understand stuff. The cost of programming is also training time to understand how stuff works and how to manipulate them.
Most people can't use mice or keyboards with speed, touchscreens are marginally better except all the "gestures" are unnatural as hell, and programming is pig latin.
Mice and keyboards and programming languages and all the esoteric ways of communicating with computers came about simply because we couldn't just talk Human to them. Democratizing access to computers is a very good and very productive thing.
Generative AI can be thought as an interface to the tool, but it's been proven that they are unreliable. And as the article outlines, if it can get to 70% of the task, but you don't have the knowledge requires to complete it, that's pretty much the same as 0%. And if you have the knowledge, more often than not you realize that it just go faster on a zigzag instead of the straight route you would have taken with more conventional tools.
If you aim for the stars and fail, you'll end up on the moon.
Most of the "you'll never need programmers again!" things have ended up more "cars-showered-with-chunks-of-flaming-HTPB" than "accidentally-land-on-moon", tbh. 4GLs had an anomaly, and now we don't talk about them anymore.
(It's a terrible adage, really. "Oops, the obviously impossible thing didn't happen, but an unrelated good thing did" just doesn't happen that often, and when it does there's rarely a causal relation between A and B.)
Programming today is literally hundreds of times more productive than in 1950. It doesn’t feel that way because of scope creep, but imagine someone trying to create a modern AAA game using only assembly and nothing else. C didn’t show up until the 70’s, and even Fortran was a late 50’s invention. Go far enough back and people would set toggle switches and insert commands that way no keyboards whatsoever.
Move forward to the 1960’s and people coded on stacks of punch cards and would need to wait for access to a compiler overnight. So just imagine the productivity boost of a text editor and a compiler. I’m not taking an IDE with syntax checks etc, just a simple text editor was a huge step up.
And so forth.
You're missing things like LISP and Forth, which allowed for lot of productivity early on. It usually had a performance cost, though.
And PHP (the programming language) just before that, that was a huge change in "democratising" programming and making it easier, we wouldn't have had the web of the last 20-25 years without PHP.
The comparisons are lacking and are almost at whataboutism level.
The amount of actual 'work' that AI does versus the tools of yesterday are an order of magnitude away
In that view I'd say the productivity boost by LLMs is somewhat disappointing, especially with respect to how amazing they are.
From memory, they took some old java projects, and had some LLM driven "agents" update the codebase to recent java. I don't know java enough to know how "hard" this task is, but asking around I've heard that "analog" tools for this exist, but aren't that good, bork often, are hardcoded and so on.
Amz reported ~70% of code that came out passed code review, presumably the rest had to be tweaked by humans. I don't know if there are any "classical" tools that can do that ootb. So yeah, that's already imrpessive and "available today" so to speak.
Really it's not about just using technology, but how you use it. Lots of adults expected kids with smartphones to be generally good with technology, but that's not what we're witnessing now. It turns out browsing TikTok and Snapchat doesn't teach you much about things like file system, text editing, spreadsheets, skills that you actually need as a typical office worker.
That said, it’s a really nice tool. AI will probably be part of most developer’s toolkits moving forward the way LSP and basic IDE features are.
Just like we did at every earlier stage.
AI is great at getting you started or setting up scaffolds that are common to all tasks of a similar kind. Essentially anything with an identifiable pattern. It’s yet another abstraction layer sitting on an abstraction layer.
I suspect this is the reason we are really only seeing AI agents being used in call centers, essentially providing stand ins for chatbots- because chatbots are designed to automate highly repetitive, predictable tasks like changing an address or initiating a dispute. But for things like “I have a question about why I was charged $24.38 on my last statement” you will still be escalated to an agent because inquiries like that require a human to investigate and interpret an unpredictable pattern.
But creative tasks are designed to model the real world which is inherently analog and ever changing and closing that gap of identifying what’s missing between what you have and the real world and coming up with creative solutions is what humans excel at.
Self driving, writing emails, generating applications- AI gets you a decent starting point. It doesn’t solve problems fully, even with extensive training. Being able to fill that gap is true AI imo and probably still quite a ways off.
Wishful thinking? You'll just get kicked out of the chat because all the agents have been fired.
None of these tools hurt, but you still need to comprehend the problem domain and the tools -- not least because you have to validate proposed solutions -- and AI cannot (yet) do that for you. In my experience, generating code is a relatively small part of the process.
You know what is even cheaper, more scalable, more efficient, and more user-friendly than a chatbot for those use cases?
A run of the mill form on a web page. Oh, and it's also more reliable.
> A run of the mill form on a web page. Oh, and it's also more reliable.
Web-accessible forms are great for asynchronous communication and queries but are not as effective in situations where the reporter doesn't have a firm grasp on the problem domain.
For example, a user may know printing does not work but may be unable to determine if the issue is caused by networking, drivers, firmware, printing hardware, etc.
A decision tree built from the combinations of even a few models of printer and their supported computers could be massive.
In such cases, hiring people might be more effective, efficient, and scalable than creating and maintaining a web form.
Hum... Your point is that LLMs are more effective?
Because, of course people are, but that's not the point. Oh, and if you do create that decision tree, do you know how you communicate it better than with a chatbot? You do that by writing it down, as static text, with text-anchors on each step.
Are they?
If the LLMs could talk to grandma for 40 minutes until it figures out what her problem actually is as opposed to what she thinks it is and then transfer her over to a person with the correct context to resolve it, I think that's probably better than most humans in a customer service role. Chatting to grandma being random for an extended amount of time is not something that very many customer service people can put up with day in and day out.
The problem is that companies will use the LLMs to eliminate customer service roles rather than make them better.
Ok, but it can't. If we had superhuman AGI, it would beat people in customer service, yes.
that aged like LeCun
If you can code, and you understand the problem (or are well on your way to understanding it), but you're not familiar with the exact syntax of Go or whatever, then working with AI will save you hundreds of hours.
If you can't code, or do not (yet) understand the problem, AI won't save you. It will probably hurt.
I used GitHub Copilot in a project I started mostly to learn Go. It was amazing. I spent not so much time fiddling around with syntax, and much more time thinking about design.
I used Claude recently to refresh my knowledge on the browser history API and it said that it gets cleared when the user navigates to a new page because the “JavaScript context has changed”
I have the experience and know how to verify this stuff, but a new engineer may not, and that would be awful.
Things like these made me cancel all my AI subscriptions and just wait for whatever comes after transformers.
Good knowledge of facts, willing to give anything a try, but not much wisdom.
I haven’t seen this demonstrated in gpt-4 or Claude sonnet when asking anything beyond the most extreme basics.
I consistently get subtly wrong answers and whenever I ask “oh okay, so it works like this” I always get “Yes! Exactly. You show a deep understanding of…” even though I was wrong based on the wrong info from the LLM.
Useless for knowledge work beyond RAG, it seems.
Search engines that I need to double check are worse than documentation. It’s why so many of us moved beyond stack overflow. Documentation has gotten so good.
"Of course not," replied Bohr, "but I understand it brings you luck whether you believe it or not."
s/luck/lower negative log likelihood. Besides, a participant can still think about and glean truths from their reflections about a conversation they had which contained only false statements.
That it’s better to be superstitious than not, because maybe it matters?
That was GP's point. Steve Yegge has also talked about this.
Someone who does not have that experience would be much more harmed by the false information LLMs can and do spit out.
…which sometimes feels like it is more work than just fucking doing it yourself. So yeah. I dunno!
Terraform? Hah. 4o and even o1 both absolutely sucked at it. You could copy & paste the documentation for a resource provider, examples and all, and it would still produce almost unusable code. Which was not at all helpful given I didn’t know the language or its design patterns and best practices at all. Sonnet 3.5 did significantly better but still required a little hand holding. And while I got my cloud architecture up and running now I question if I followed “best practices” at all. (Note: I don’t really care if I did though… I have other more important parts of my project to work on, like the actual product itself).
To me one of the big issues with these LLM’s is they have zero ability to do reflection and explain their “thought process”. And even if they could you cannot trust what it says because it could be spouting off whatever random training data it hovered up or it could be “aligned” to agree with whatever you tell it.
And that is the thing about LLM’s. They are remarkably good bullshitters. They’ll say exactly what you want them to and be right just enough that they fool you into thinking they are something more than an incredibly sophisticated next token generator.
They are both incredibly overrated and underrated at the same time. And it will take us humans a little while to fully map out what they are actually good at and what they only pretend to be good at.
Of course everything needs double-checking but just asking the LLM: "how do I do X" will usually at least output all the names of terrraform resources and most configuration attributes I need to look up.
They are great for any kind of work that requires "magical incantations" as I like to call them.
Could code assistants be used to help actually learn a programming language?
- Absolutely.
Will the majority of people that use an LLM to write a Swift app actually do this?
- Probably not, they'll hammer the LLM until it produces code that hobbles along and call it a day.
Also, LEARNING is aided by being more active, but relying on an LLM inherently encourages you to adopt a significantly more passive behavior (reading rather than writing).
What I personally would like AI to do would be to refactor the program so it would be shorter/clearer, without changing its semantics. Then, I (human) could easily review what it does, whether it conforms to the specification. (For example, rewrite the C program to give exactly the same output, but as a Python code.)
In cases where there is a peculiar difference between the desired semantics and real semantics, this would become apparent as additional complexity in the refactored program. For example, there might be a subtle semantic differences between C and Python library functions. If the refactored program would use a custom reimplementation of C function instead of the Python function, it would indicate that the difference matters for the program semantics, and needs to be somehow further specified, or it can be a bug in one of the implementations.
LLMs absolutely can "detect intent" and correct buggy code. e.g., "this code appears to be trying to foo a bar, but it has a bug..."
It’s not either/or of course, and AI can help
But sometimes it takes another leap beyond the current set of test cases
Once you write sufficiently detailed unit tests, the AI writes the implementation.
Mind you, when you treat the implementation as the documentation, it questions what you need testing for?
It may not be tab-tab-tab all the way, but a whole lot more tabs will sneak in.
Humankind will just develop a lot more software, faster and further into industries.
Availability will create new demand.
I design top-down, component by component, and sometimes the parts don't fit together as I envisioned, so I have to write adapters or - worst case - return to the drawing board. If the AI could predict these mismatches, that would also be helpful.
Unfortunately, I don't think AI is great with only the accidental tasks either.
AI will only get better.
And AI has proven a lot of unproven maths problems as far back as 2019: https://mathscholar.org/2019/04/google-ai-system-proves-over...
Like, for example, all the big-data stuff we do today was unthinkable 10 years ago, today every mid-sized company has a data team. 15 years ago all data in a single monolithic relational database was the norm, all you needed to know was SQL and some Java/C#/PHP and some HTML to get some data wired up into queries.
How many of those things were envisioned by futurists or great authors? This AI stuff is the stuff of dreams, and I think it’s unwise to consider it another go around the sun.
Yes, a powerful tool, and as powerful tools go, they can re-shape how things get done, but a tool none the less and therefore we must consider what its limits are, which is all OP is getting at and the current and known near future state suggests we aren’t evolving passed the tool state
This AI stuff? No, not really. The stuff of dreams is an AI that you can talk to and interact infinitely and trust that it doesn’t make mistakes. LLMs ain’t it.
I don't think that's really the problem with using LLMs for coding, although it depends on how you define "accidental". I suppose if we take the opposite of "essential" (the core architecture, planned to solve the problem) to be boilerplate (stuff that needs to be done as part of a solution, but doesn't itself really define the solution), then it does apply.
It's interesting/amusing that on the surface a coding assistant is one of the things that LLMs appear better suited for, and they are suited for, as far as boilerplate generation goes (essentially automated stack overflow, and similar-project, cut and pasting)... But, in reality, it is one of the things LLMs are LEAST suited for, given that once you move beyond boilerplate/accidental code, the key skills needed for software design/development are reasoning/planning, as well as experienced-based ("inference time") learning to progress at the craft, which are two of the most fundamental shortcomings of LLMs that no amount of scale can fix.
So, yeah, maybe they can sometimes generate 70% of the code, but it's the easy/boilerplate 70% of the code, not the 30% that defines the architecture of the solution.
Of course it's trendy to call LLMs "AI" at the moment, just as previous GOFAI attempts at AI (e.g. symbolic problem solvers like SOAR, expert systems like CYC) were called "AI" until their limitations became more apparent. You'll know we're one step closer to AI/AGI when LLMs are in the rear view mirror and back to just being called LLMs again!
Packages were supposed to replace programming. They got you 70% of the way there as well.
Same with 4GLs, Visual Coding, CASE tools, even Rails and the rest of the opinionated web tools.
Every generation has to learn “There is no silver bullet”.
Even though Fred Brooks explained why in 1986. There are essential tasks and there are accidental tasks. The tools really only help with the accidental tasks.
AI is a fabulous tool that is way more flexible than previous attempts because I can just talk to it in English and it covers every accidental issue you can imagine. But it can’t do the essential work of complexity management for the same reason it can’t prove an unproven maths problem.
As it stands we still need human brains to do those things.