Unfortunately Eric Sink probably feels now how the formal methods people did 4-5 decades ago? (on the other hand, deep understanding ought to provide some sort of comparative advantage — but will it do so in industrial as well as academic settings?)

From 2021 @dang

Relevant fiction: "Pump Six" by Paolo Bacigalupi.

Progress is made by most people not needing deep understanding. There's only so much attention to go around. If people really needed to understand most of the things they dealt with, little would get done.

He doesn't mean deep understanding of the subject matter. He means detailed knowledge about the inner workings. Those are different. For his database example, deep understanding would involve understanding third normal form, the history of databases from ISAM to CODASYL through relational databases to large key-value stores, with a side trip into eventually-consistent systems. Understanding the inner workings is knowing how an InnoDB table in MySQL is represented on disk.

Progress in a purely capitalistic sense, maybe—but in terms of human knowledge and innovation in technology, absolutely not.

Besides, in the same way that being an engineer with no people skills isn’t a desirable thing, there are plenty of problems with knowing one’s business domain only at the surface-level:

1. It’s always these types of people who end up pushing for unrealistic, and many times, senseless engineering goals from management positions so that they waste a lot of the company’s time and money just by existing in it.

2. It’s these types of people who prioritize profits at all costs, even if it could endanger a customer’s life or data security.

3. These types of people are very easily replaceable, because people are by default not inclined to learn about the world around them, and hence they’re dime a dozen.

I thought everyone understood Progress® as a marketing term and a type of theater with the only real progress being the net worth of the actors.

I didn't think people actually confused the theatrical performance with the progress of Boltzmann or Newton. Come on!

> Progress in a purely capitalistic sense, maybe—but in terms of human knowledge and innovation in technology, absolutely not.

No this is still the case, because even folks who understand deeply can only understand a few things deeply. I recently spent several years working in battery tech. The field could be years ahead of where it is now if so many of the field's most pressing problems didn't require researchers to deeply understand programming as well as their field.

Deep understanding of man-made systems? Probably not as much.

Deep understanding of Nature? As valuable as ever.

So the 'Foundation' timeline?

I don't understand that. So we won't (or shouldn't?) have deep understanding of man-made systems, which causes all of our problems? The history of mankind is fraught with issues stemming from not knowing what we're doing.

Humans are kind of like termites. Deep understanding of termite-made systems is also deep understanding of nature.

AI generated code is average code, because most human made code is also. So AI generate code is fine for MVP or some basic CRUD thing but when you need optimize that last 1 ms from complex SQL query human will be still be better.

IMO it comes down to essential vs incidental complexity. Understanding fundamental realities is valuable. Memorizing trivia isn't. Most software complexity is still the latter rather than the former.

Totally agree. AI tools devalue esoteric knowledge, and increase the value of conceptual understanding.

AI tools devalue common trivia. Esoteric knowledge remains valuable, because it's not common enough in the training data that the AI can be expected to get it right.

That hasn't been my experience. Purely anecdotal, but it seems to me that if a topic has been written about with a high degree of accuracy a small number of times, AI is more likely to be factual than a topic for which there is a lot of low quality information available.

A few situations I have encountered:

1. Sufficiently obscure technical details in some field of research. There are several high-quality sources, but each of them uses its own terminology with subtly different definitions. Many immediate consequences are also left unsaid, because you can't say everything if you want to keep the paper readable. Anyone with sufficient background should see them anyway.

2. Word etymologies. There are often many competing explanations with varying degrees of plausibility and popularity, and many sources with varying degrees of credibility. The things the LLM hallucinates are often quite interesting.

3. Questions with well-defined answers that are overshadowed by something far more popular. Like, for example, what did Caesar do on the night before crossing Rubicon.

I still think specific, esoteric knowledge gets devalued.

The way I experience AI is that it's like a super-powered search. It's like having an undergrad with limited knowledge read everything and come back to you with an answer.

Based on their own limited knowledge on complex topics, they're not always going to come back to you with the correct answer, but it gives you a heck of a better starting point than if you were to search for it yourself from scratch.

So what that means regarding esoteric knowledge is that the specific detail you were looking for is no longer hidden beneath 5 layers of obscura - you at least get pointed to the general area of the solution, and then maybe you have to read a paper or two to get the details you need. And instead of having to read 5 other papers to understand that one, you can ask questions and get decent answers about the terms you don't understand etc.

So imo it becomes less valuable to yourself be an encyclopedia of esoteric knowledge, since that knowledge becomes more accessible. It's like how being an expert on any road system become much less valuable with the advent of GPS.

You'd think that, but when I asked ChatGPT for a stack canary (a fairly esoteric thing ) it knew what it was, how it was supposed to work and produced code that needed very little work to change.

Stack canaries are a good example of what I called common trivia. I had never heard of them before, but I could guess the overall idea from the term itself. A straightforward Google search returned several sources that gave further details. ChatGPT simply gave you a more user-friendly way of accessing information that was already easy to find if you knew to look for it.

So what would you consider esoteric knowledge then?

I have to be honest I never heard of Stack Canaries before - it seems like a fairly specific term of art and I would have a hard time thinking of it as "common trivia"

Esoteric knowledge is something accessible to a limited number of people. As an academic researcher, many details of my work are esoteric in this sense, as there are not that many people in the world working on anything closely related. Similarly, the internal workings of an established organization often involve esoteric knowledge. Traditions emerge over time, many things are never fully documented, and people outside the organization generally have no idea what's going on.

In contrast, stack canaries are potentially relevant to a large number of developers. They are also accessible: many developers should be able to figure out the details on their own once they have heard of the idea.

Idk to be honest, it seems like you are defining esoteric in a very narrow sense, which borders on "no true scotsman", such that anything AI would be capable of producing would no longer fall within the definition.

Imo stack canaries is esoteric, at least relative to myself. On reading the first google result, I can understand the jist of the intent, but it would certainly take me time to understand the details of how it works, and to come up with an implementation. And if I didn't see it mentioned in this thread I never would have even known to search for it, and probably would have had to dig through a lot of domain knowledge to even know it's something that exists. That's the same way something can be esoteric, even if it's written about and available to everyone with access to a university library, because anyone without domain knowledge would not even know it exists or why it's relevant.

And I'm sure you can agree that if Google did not exist, probably stack canaries would be unknown and inscrutable to a vast, vast majority of developers. So if we go with your definition, the invention of Google certainly greatly reduced the amount of esoteric knowledge in existence, by making information easier to access in general. So I would argue that generative AI further reduces the scope of which information could be considered generally inaccessible,

Esoteric knowledge in the sense I described is very common in everyday life. In my work as a researcher, some of the topics are esoteric. Many of the codebases I use in my work are esoteric, because the ideas behind them only exist inside a few people's heads. The administrative processes at the university are esoteric. Many details about the gaming convention I've organized as a hobby for 20+ years are esoteric.

Other topics may be difficult and advanced, but they are not esoteric, because they are more universal. You don't need a specific context to understand them, only the basic expertise.

The internet turned much esoteric knowledge into common trivia by making information more accessible. I don't think AI will have a similar effect. At least not the public models that are mostly trained using information that is already accessible. Large organizations may achieve something with internal models, but in many cases, esoteric knowledge is esoteric because it's implicit.

I don't think separating "trivia" from the "fundamental realities" is as easy as we hope it is.

I have a very hard time convincing myself I could be as effective a programmer (and learner) without all the "trivia" I accumulated. And this is coming from someone that dislikes trivia with an - at best - average memory.

I think this could be a fundamental problem with intelligence in general. You cannot just build towers of abstraction and live on the highest plane only. All layers co-operate and mingle chaoticly. Every little seemingly random patch of pixels in your field of vision could correlate highly with the highest of abstract concepts you are capable of mentating. Knowing what is "trivia" and what is not is itself too a very hard problem.

Doesn’t even stop to consider that perhaps the AI pair programmer will actually helpfully answer your questions about how something actually works, thus enriching one’s deep understanding.

Human-out-of-loop self-programming systems are coming. Philosophical opinions of craftsmanship and understanding are orthogonal and thus moot to the impending replacement of human labor with automation. Beyond prompts, NAI applied to capturing and apply requirements, feedback, feature requests, and bug reports will be the tipping point.

It doesn't matter if someone thinks they're special when an algorithm can and does take their job by spitting out disgusting Javascript that happens to accomplish business needs well enough to not hire the human. Business value is all businesses care about.

Human-out-of-loop clothing has been a thing for awhile now. And if business only wants to wear one-size-fits-all t-shirts and tube socks, then they are in for a treat.

Those that need something more bespoke, that fits them better, that isn't disposable because the problems it's trying to solve aren't disposable... they might still need the tailor.

I know a few human programmers who would make really good LLMs. And they seem unable to come up with non-absurd code at all. If you define the specs with enough details that they haven't screwed it up, then you're at a point where you've basically already written the software anyway. Should I be worried that robot programmers will do any better?

Hard disagree.

I think we primarily make progress as engineers (and a species) by making better abstractions.

Yes there are some leaky abstractions, SQL is a great example. But there are plenty of systems that don't leak (e.g. harddrives with constantly dying sectors), and the more you think about all the abstractions the more you have to admit that nobody could get their job done if they had to understand every bit of the computer and the cloud platform and the software.

I think progress is made by standardizing on the right abstractions. What the purest and best form of progress is is something like half of the internet being wordpress. Progress is building tremendous things with the least amount of learning expense necessary.

AI will help get us there. Some day soon AI will write queries and it'll do it better than most companies need before they're profitable.

Civilization advances by extending the number of operations we can perform without thinking about them.

– Alfred North Whitehead

The counterpoint is that these extensions are systematically produced by people who do think about them.

> Hard disagree. I think we primarily make progress as engineers (and a species) by making better abstractions.

Abstractions are produced and consumed. You can make your abstractions better, but that doesn't mean you won't have to deal with leaky abstractions done by others.

Also, having depth in understanding is not only good in cases of leaky abstractions. It's also good to have a better breadth of options for what's out there. If layer B is built on layer A, but you just know B, you might end up building layer C on top of B that's very much like A, but worse in bloat and convolution. Like making a server that serves requests passed as files via NFS because you didn't know TCP sockets existed.

Dunno if what we're making by building ever higher jenga towers of abstractions is actually progress. Most of the software that is built that way hasn't significantly increased in its capacities in the last few decades (despite running on a thousand times more powerful hardware).

The software that has brought new things to the table has mostly been built close to the hardware, far away from the apex of the software stack. Stuff like redis, the AI boom, none of that is built in wasm or electron.

> But there are plenty of systems that don't leak (e.g. harddrives

Until they do. Had to deal with this recently. Not easy.

I got physically unwell when I read "the purest and best form of progress is is something like half of the internet being wordpress". That's like saying standardizing on crystal meth is progress.

I get the general sentiment, but I wouldn't confuse "popularity" with "progress".

An old debate! Here's a counterpoint from a Steve Yegge piece written nearly two decades ago: https://sites.google.com/site/steveyegge2/practical-magic

> I wrestle with the whole abstraction question, and I'm still not sure where a good engineer these days should draw the line — the one below which it's all just magic.

> We all have that line somewhere; I encountered mine in school, back in my semiconductors course. We were (ostensibly) learning how semiconductors work, but it was perfectly clear to me that this was just magic, and I was never going to understand it — in part, because I didn't want to make what was clearly going to be a HUGE effort to get it. There was all this crap about K-spaces and lattices and who knows what else. Maybe they'd listed the prerequisites wrong, but I did not have the math foundations to understand what was going on.

> It didn't help that my professor was 100% senile, either. Once a week he'd break into his senile rant about what an Amazing Device the Human Eye is, totally forgetting that he'd shared this insight with us every week since the quarter started. We'd all recite the story along with him, sometimes jumping ahead, which he senile-ly mistook for us being just as big fans of the Human Eye as an Amazing Instrument as he was. It was really damned hard to concentrate in that class, and at the end of the quarter, my Human Eye was Amazed to see I'd almost failed it, which signalled clearly that it was Time to Change Majors. I'd found that I love programming and detest hardware, so staying a Computer Engineering major probably wouldn't be too wise.

> So I transferred into the UW's Computer Science department, which meant switching from Engineering to Arts & Sciences, which meant taking a bunch of extra foreign-language and froo-froo social anthropology classes, where I got to watch films of dolphins counting to five through their blow-holes. Which is about how smart I felt after that semiconductors class. I felt like I could have made it as a dolphin.

> It took me an extra year, but I finished in CS instead of CE, which allowed me to conveniently pretend that computer hardware, starting somewhere down at the level of the doping barrier in a silicon semiconductor, is just pure magic. Parts of the hardware above that level are still a little murky, but I have a pretty good idea how computers can be made by assembling successively larger components together from smaller ones. Given time and a gun to my head, I could probably start with any set of smaller components and derive how to build higher-level abstractions: transistors into logic gates, logic gates into larger assemblies that do arithmetic and boolean computing, those into state machines and ALUs, and so on. I'd suck at it, but I get the general idea, so it's not really "magic".

> It's magic when you have absolutely no frigging clue how it works, not the first clue. It just works. That's how it is for me and semiconductors.

> But I'm OK with that. I'm OK with treating semiconductors as "atoms" (or axioms, anyway) in our little programming universe. Chips are like molecules, CPUs are compounds, and Von Neumann machines can be bricks and mortar. Well, I'm sure you could come up with a way better metaphor. Whatever.

> The point is, the computers and networks and power supplies and so on are all amazingly complex under the hood, but usually I can just pretend they work a certain way, and not worry about how, any more than I have to worry about my Amazing Eye in order to pour myself another glass of fine wine, which I will doubtless use to polish my keyboard if I don't finish this blog entry soon.

Without deep understanding, technological process stops. This has happened before, there are periods in human history when people said 'this is good enough, it works' and that went on for hundreds of years. Might even have been fairly pleasant, until it wasn't.

Deep understanding is valuable for the areas that will benefit you for having deep understandings. Time is a limited resource. It is impossible to anyone to learn all the detail inner workings of everything on the way down. A thousand years would not be enough as knowledge changes all the time. You need to choose which level of deep understanding you need to have on any subject. And it is more art than science.

Totally agree on gaining understanding. I think everyone should be able to get a university level education if they want to.

But an AI buddy should make it way easier to get to deep understanding imo. People are naturally curious and will dive into topics that pique interest/joy.

> It seems like most people in our industry care far more about "how to do" rather than "how does it work".

It might come to you as a shocker: businesses rarely care about the "how does it work" - illusion is all that matters for most businesses, unfortunately.

The developer is the abstraction maintaining that illusion - be it by juggling a legacy codebase or a broken implementation, or a great one.

If it does actually work, I'd hardly call that an illusion. You're right that businesses (and the end users!) are typically not interested in implementation details though.

One thing I've notices over the decades of improving as a software engineer that that the deeper I could understand things, the better abstractions I could produce where I didn't need to understand things -- which actually was the goal.

That is, the deep understanding of an abstraction is required so that you can confidently use it without having to worry about the possible side-effects that those with shallow understanding inevitably introduce.

It's not even valuable now, it's why we use abstractions and rely on interfaces.

"How does it work" rarely brings value.

I don't understand why this is downvoted.

It's true. Business doesn't need to care or know about how btree indices works in RDBMs, they just care whether there reports are generated fast enough.

Most of the older software engineers are tinkerers. Like when I was a kid I tried to take apart all my toys and see how it worked. When a computer magazine came out with a game listing I typed it in and learned from it. Then moved and changed things around to see how it worked. This is how I went into software engineering.

These days I just do it for a living mostly. CRUD code, arbitrary complex systems because of governance/compliance. It is all a bit boring. It is a how I make a good bundle of money.

On the side I build custom motorcycles. Older ones... I buy an engine that is as mechanical as possible. Easy to fix/maintain etc. Then I do metal fabrication and build a bike around it. For hours I can tinkering with this. Just as I did over 25+ years ago with my computers.

what I wat to say is you still need people to do this so they can dive deep into the abstractions. Bit how people are able to drive their car, maybe know highlevel how the engine works. But are not able to build or maintain one :)

I think it is probably worth having deep understanding of some things. The reason is that getting that deep understanding will be hard, and doing so will make you smarter, not just at that thing but in general. A bit like the enlarged hippocampus of London Cab drivers who have to remember all the streets (being a London Cab driver might be as good as learning networking to a deep level?).

Then when you are given something hard to quickly learn, e.g. say you need to go from zero to Kubernetes pro in a week, doing so should be easier. Or even learning the bits that are worth learning to optimize that week so you can be most productive after it, will be easier.

It is a different argument than the one about avoiding leaky abstractions.

On a positive note: at least you can iteratively question the assistant, while the traditional 'cookbook' coding recipies are dead letter.

There’s a kind of bathtub curve between generalism and specialism.

On the far left you have master generalists with incredible breadth of understanding and the ability to summarize and strategize. These people are highly valued and often land in architecture or leadership roles.

Then in the middle you have the specialists. The Java developers and the Ops people. They’re less valuable because there are so many of them, and they’re somewhat interchangeable.

But then on the far right you’ve got super-specialists. They are masters of their niche - world-class experts with deep understanding who are sought after as the authority in their domain. These folk are super-valuable too.

AI will probably eat the ones in the middle first.

It’s very clear that Copilot/GPT-4 give superpowers to the generalist—-rapid prototyping—-and to the ultra-specialist—-no more yak-shaving in other domains.

Most roles that are run-of-the-mill “knows framework X” just enough to support naive clients or actually high/value teams, will indeed go to zero.

Which is amusing because 90% of the “AI” companies being started are run by people with no actual expertise, who probably are the most replaceable parts of deploying actual high/end systems.

The penny dropped at

> Last week I was trying to confirm that the CIL castclass

This person is working in the Microsoft .NET ecosystem, in which the "you don't need to know that" attitude is prevalent. The whole ecosystem is built around it, and I'm not for a second saying that is a bad thing. .NET technologies are powerful and effective.

However, like the author I didn't find myself comfortable in that world. I tend to gravitate towards a certain kind of problem and my problem solving approach relies on fully understanding what is going on. While .NET Core (now just .NET) is open source, the don't-look-under-the-hood mentality is built into it in many ways, and I found my way of thinking to be at odds with it. After a few years of Windows .NET work I eventually moved to a C++/Python/Linux role and am a lot happier. I don't think the employer I was with for those years should have gone with the latter stack (but I emphatically don't think my new employer should have gone with the former - it's a different problem domain). It's not a criticism. I just feel more at home doing this kind of development.

In short, if you find yourself reading the 30,000 line C++ file containing the .NET garbage collector implementation more than once, you may not be working in a technical environment best suited to your mental wellbeing.

Yes, you can succeed and make a living just relying on abstraction and getting the job done without fully understanding how it works. But the really good engineers that deliver better quality software are the ones that understand how things work inside (at least down to a reasonable level) and also have the big picture of how their code works with the rest of the systems/services in their company.

(2021)

AI will make the https://en.wikipedia.org/wiki/Competent_man even more competent.