The original "law of leaky abstractions" talked about how the challenge with abstractions is that when they break you now have to develop a mental model of what they were hiding from you in order to fix the problem.

(Absolutely classic Joel Spolsky essay from 22 years ago which still feels relevant today: https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-a... )

Having LLMs write code for you has a similar effect: the moment you run into problems, you're going to have to make sure you deeply understand exactly what they have done.

I'm finding that, If I don't have solid mastery of at least one aspect of generated code, I won't know that I have problems until they touch a domain I understand.

Using abstractions to trade power for simplicity is a perfectly fine trade-off... but you have to bare in mind that at some point you'll run into a problem that requires you to break through that abstraction.

I read that essay 22 years ago and ever since then I've always looked out for opportunities to learn little extra details about the abstractions I'm using. It pays off all the time.

No, of course not. I’m saying that the article is bad. Because he’s not saying anything that the term itself isn’t already describing or implying.

No adjectives needed. And certainly no laws.

Agree, specially useful when you join a new company and you have to navegate a large codebase (or bad-maintained codebase, which is even worse by several orders of magnitude). I had no luck asking LLM to fix this or that, but it did mostly OK when I asked how it works and what the code is trying to code (it includes mistakes but that's fine, I can see them, which is different if it was just code that I copy and paste).

…but I haven’t joined a new company since LLMs were a thing. How often is this use case necessary to justify $Ts in investment?

It's something I would cosnider paying for the first months when joining a new company (specially with a good salary), but not more than that, to be honest.

You can generally trust that transistors won't randomly malfunction and give you wrong results. You can generally trust that your compiler won't generate the wrong assembly, or that your interpreter won't interpret your code incorrectly. You can generally trust that your language's automatic memory management won't corrupt memory. It might be useful to understand how those layers work anyway, but it's usually not a hard requirement.

But once you reach a certain level of abstraction (usually 1 level above programming language), you'll start running into more and more issues resulting from abstraction leaks that require understanding the layer below to properly fix. Probably the most blatant example of this nowadays are "React developers" who don't know JS/CSS/HTML and WILL constantly be running into issues that they can't properly solve as a result, and are forced to either give up or write the most deranged workarounds imaginable that consist of hundreds of lines of unintelligible spaghetti.

AI is the highest level of abstraction so far, and as a result, it's also the leakiest abstraction so far. You CANNOT write proper functional and maintainable code using an LLM without having at least a decent understanding of what it's outputting, unless you're writing baby's first todo app or something.

I want to frame this. I am sick to death of every other website and application needing a gig of RAM and making my damn phone hot in my hand.

Llms currently have tiny holes and we don't know if we can fix them. Established abstractions are more like cups that may leak but only in certain conditions (when it's hot)

This is the weakest point which breaks your whole argument.

I see it happening ALL the time: newer web developers enter the field from an angle of high abstraction, whenever these abstractions don't work well, then they are completelly unable to proceed. They wouldn't be in that place if they knew the low-level and it DOES prevent them from delivering "value" to their customers.

What is even worse than that, since these developers don't understand exactly why some problen manifests, and the don't even understand exactly what their abstraction trully solves, they wrongly proceed to solve a problem using the wrong (high level) tools.

That has some amount to do with the level abstraction but almost everything to do with inexperience. The lower level you get, the harder the impact of inexperience.

New web developers are still sorting themselves out and they are at a stage where they’ll suck no matter what the level of abstraction.

This creates a barier in gaining experience on the topic that matters in order to solve your problem. Let me try to give an example (please don't neat pick, it is just an example):

A front-end dev who has never bundled an app like we did in the old days (manual scripts or manual gulp, grunt, webpack pipelines) but use a off-the-shelf webpack config (lets say CRA) has great trouble understanding why some webpack plugin doesn't work as expected, or they might not even understand how importing an svg into their jsx actually works. Yes this is due to lack of experience, but their current level of working isn't exposing those details directly so they can't gain experience, the waters are way too deep.

"Abstractions" in the abstract neither do nor do not do this, as it is orthogonal to what an abstraction is; concrete implementations of abstractions can either swallow or wrap information about the underlying cause when they break, both are valid approaches.

> but their current level of working isn't exposing those details directly so they can't gain experience

This isn't really true. They absolutely can gain experience; they don't pay a (very high) extra effort with the side benefit of maybe gaining some experience on routine tasks where nothing breaks, and tend to gain experience only if they expend extra effort in situations where things fail or their are unusual challenges to resolve.

Not sure where you are getting with this. Yes, literally speaking they can, but what matters is if they actually do, in general, in the real world. My admitedly biassed opinion, based on personaly observating other colleagues, shows that in general, they don't learn and they give up.

Have you noticed an other trend in your circles?

And, despite the fairly consistent bias people used to working at lower levels of abstraction have against people who usually work at higher levels, we all work with some level of abstraction, and most people gain experience at lower levels by dealing with problems that emerge, and those that don't don't fail to do so because they can't but because they choose not to bother either because they have other people that they can rely on for lower-level problems, or because they aren't interested and are doing things where they can afford to simply change what they are doing in response to those problems.

It’s not the same as the other times. The naysayers might be the same elitists as the last time. But that’s irrelevant because the moment is different.

It’s not even an abstraction. An abstraction of what? It’s English/Farsi/etc. text input which gets translated into something that no one can vouch for. What does that abstract?

You say that they can learn about the lower layers. But what’s the skill transfer from the prompt engineering to the programming?

People who program in memory-managed languages are programming. There’s no paradigm shift when they start doing manual memory management. It’s more things to manage. That’s it.

People who write spreadsheet logic are programming.

But what are prompt engineers doing? ... I guess they are hoping for the best. Optimism is what they have in common with programming.

But do LLMs provide a higher level of abstraction? Is this really one of those transition points in computing history?

If they do, it's a different kind to compilers, third-party APIs or any other form of higher level abstraction we've seen so far. It allows programmers to focus on a different level of detail to some extent but they still need to be able to assemble the "right enough" pieces into a meaningful whole.

Personally, I don't see this as a higher level of abstraction. I can't offload the cognitive load of understanding, just the work of constructing and typing out the solution. I can't fully trust the output and I can't really assemble the input without some knowledge of what I'm putting together.

LLMs might speed up development and lower the bar for developing complex applications but I don't think they raise the problem-solving task to one focused solely on the problem domain. That would be the point where you no longer need to know about the lower layers.

Oddly enough, using an AI assistant, despite it guessing incorrectly as often as it did, helped me learn and write code faster!

Source? A quote? Or are we just making up historical strawmen to win arguments against?

https://vivekhaldar.com/articles/when-compilers-were-the--ai...

He provides some sources (7) at the bottom of the article.

To pick one, the following has a video interview with one of the founders of Fortran:

https://www.ibm.com/history/fortran#:~:text=Fortran%20was%20...

So we're not making up stuff, this perspective was ubiquitous among assembly programmers of the 1950s. In 1958 (as the first article I link to mentions), half of programs were written in Fortran. Which means half of people still thought writing assembly by hand was the way to go.

I've personally written assembly by hand for money on an obscure architecture, and I've also written a non-optimizing compiler for a subset of Rust to avoid the assembly. There is great joy in playing stack tetris, but changing code requires a lot of effort. Imagine if there weren't great alternatives, you'd just get good at it.

(just to save some face: I learned Prolog in my second year).

I also had BlueJay on my first semester. But fortunately I had machine architecture, compilers and operating systems later.

  I had been hired to write a FORTRAN compiler
  for this new marvel and Mel was my guide to its wonders.
  Mel didn't approve of compilers.

  ``If a program can't rewrite its own code'',
  he asked, ``what good is it?''

Though to be fair The Story of Mel supports rather than refute the argument against compilers.

The joke story is mocking the common arguments/beliefs at the time.

If you expect me to source you a collection of comments about "real programmers" from over 30 years ago though that is too much of an ask but I was there, I read it often and I started fairly late on the scene in the 90s.

But i agree, it’s unrelated to devs being dumb.

Everything has some imperfection, so LLMs are just fine... Completely missing the totally valid criticisms people have of the systems.

Tools being bad currently doesn't matter just like compilers being bad in the past doesn't matter.

Every 5 years or so of my career something new has come out to make coding easier and every time a stack of people like yourself come out to argue it's making devs dumb or worse.

LLMs still generate terrible output a lot of the time but they are improving. Early compilers generated terrible ASM a lot of the time to the point that it was common to use inline assembly in your code or rewrite parts later. Tools can improve, the point is that neither make the dev worse they just add to productivity.

Writing code isn't my job it's a task I do to make the systems I design functional.

I would consider it as a tool to teach and learn code if used appropriately. However LLMs are bullshit if you ask it to write something, pieces yes, whole code... yeah good luck having it maintain consistency and comprehension of what the end goal is. The reason it works great for reading existing code is that the input results into a context it can refer back to but because LLMs are weighted values it has no way to visualize the final output without significant input.

The point is LLMs may allow developers to write code for problems they may not fully understand at the current level or under the hood.

In a similar way using a high level web framework may allow a developer to work on a problem they don’t fully understand at the current level or under the hood.

There will always be new tools to “make developers faster” usually at a trade off of the developer understanding less of what specifically they’re instructing the computer to do.

Sometimes it’s valuable to dig and better understand, but sometimes not. And always responding to new developer tooling (whether LLMs or Web Frameworks or anything else) by saying they make developers dumber, can be naive.

It's simply not the same thing as a high level web framework.

If you have an intern, or a junior engineer - you give them work and check the work. You can give them work that you aren't an expert in, where you don't know all the required pieces in detail, and you won't get out of it the same as doing the work yourself. An intern is not a layer of abstraction. Not all divisions of labor are via layers of abstraction. If you treat them all that way it's dumb and you'll have problems.

I won’t waste my time too much reacting to this nonsensical comment but I’ll just give this example, LLMs can hallucinate, where they generate code that’s not real, LLMs don’t work off straight rules, they’re influenced by a seed. Normal abstraction layers aren’t.

I dearly hope you’re arguing in bad faith, otherwise you are really deluded with either programming terms or reality.

Criticisms of each change are not somehow invalid just because the change is inevitable, like all the changes before it.

Don't confuse low-level tedium with CS basics, if you're arguing that knowing how computers work is not relevant to working as a SWE then sure, but why would a company want a software dev that doesn't seem to know software? Usually your irreplaceable value as a developer is knowing and mitigating the leaks so they don't wind up threatening the business.

This is where the industry most suffers from not having a standardized-ish hierarchy, you're right that most shops don't need a trauma surgeon on-call for treating headaches but there's still many medical options before resorting to random grifter who simply "watched some Grey's Anatomy" as "medschool was a barrier for providing value to customers".

If you are writing SQL, it’s helpful to understand how database engines manage storage and optimize queries. If you write database engine code, it’s helpful to understand (among many other things of course) how memory is managed by the operating system. If you write OS code, it’s helpful to understand how memory hardware works. And so on. But you can write great SQL without knowing much of anything about memory hardware.

The reverse is also true in that it’s good to know what is going on one level above you as well.

Anyway my experience has been that knowledge of the adjacent stack layers is highly beneficial and I don’t think exaggerated.