> In the same way high-level languages are a layer on top of writing machine code manually.

Another layer, yes, but NOT "in the same way." It is a new, different kind of layer. The other layers were deterministic, not probabilistic.

Probabilistic algorithms are not new. If the distribution is known you can bend surprisingly many things to your will in the long run - and the most interesting part is the long run can be unexpectedly short.

LLMs are quite interesting in that regard because you have a lot of levers to influence the shape of the distribution; the ways are different for each model, so it’s a very experimental science, but applying the scientific method is exactly how you get good results.

The scientific method is not about running experiments and then trying to work out the gist of something. It's about understanding and theory building and then doing experiments to verify your model. The end result is not an experimental result but a model that you understand and can use to build solutions etc. LLMs are bringing us back to pre- science correlations and superstitions. That's why all the conversations on these threads about AI are so boring. It's all anecdotes and tips and no understanding.

It almost feels like alchemy.

Vibecoding == digital dowsing?

Sir you haven’t seen evals done properly if you’re saying this.

I meant that no-one writes assembly anymore. I wasn't around when that transition happened, but I bet there were the same "this high-level language is compiling to inefficient code, it's useless" comments.

Yes. They know what you meant. You missed the point of their response.

A compiler is semantically deterministic. It has a spec. Give it an input and you can know exactly what the output will do. And if there's edge cases where behaviour is undefined, thats also well known apriori.

(And yes, to get ahead of the nitpickers, I know the specific sequence of bytes the compiler will emit can vary (hence the difficulty of reproducible builds), hence why I said "semantically").

LLMs do not share this property. At all. I mean not even close

They are a completely new and different class of tool. They are not just a step up the abstraction layer. They're not compilers for English. To call them that is to fundamentally not grok what a compiler is or does.

That doesn't make them bad. It just makes that analogy, and any conclusions you might draw from it, bad.

Interesting point about "letting go". I hear that quite a lot, which I find surprising. In IT, when new technologies like cloud computing emerge people always need to adapt their workflows, and for some that is challenging e.g. going from manually maintained servers to virtual machines or containers that are just spun up and down on demand. Maybe it is similar with AI, we gain new capabilities along one dimension like speed of development and we lose some capabilities along the way e.g. manual control of quality. And of course there are large financial incentives here as well, maybe they are larger than we have ever seen before and the change also happens faster. Cloud computing took probably multiple decades to be fully adopted (I think AWS became available in 2006 and we still see large corporations migrating to the cloud from their on premise setups today, though the adoption curve is flattening), LLMs have significantly higher adoption after only around 2-3 years of them becoming "production-ready". So naturally people struggle with how to adopt them and we need to figure out where they make sense and where not. That said it's precisely an engineers' job to figure that out, people that just see the upsides of this technology seem quite naive to me.

I can see this struggle it in my organization as well, in the last year there was a big push to adopt AI everywhere and tons of initiatives to automate processes and produce code and text and other artefacts with LLMs. Now it seems the pendulum is swinging back a little as people see that all of the LLM generated stuff shows all of these subtle quality degradations, and people get tired of managing it as they are suddenly confronted with tons of additional information they need to manage.

Given that models are still evolving and becoming better at a rapid pace I think that we will solve most of these issues in the near future, but for now I don't think the age of hand crafted code is over yet.

I have been thinking about that machine code analogy before as well, I don't think it really holds. Machine code is written in an automated way but following mostly deterministic rules that have been crafted through decades of manual optimizations and testing. AI generated code has nowhere near this level of scrutiny, testing and optimization behind itself. The fault rate of compilers and optimizers is incredibly small (I can't find any numbers but it must be on the order of ppm or ppb), AI generated code has fault rates that are even in the best case on the order of 99-99.99 % maybe (i.e. between one error per hundred lines and one error per ten thousand lines in the best case), try building anything complex using such a fault rate without manual correction and review. It's impossible. I have done it, I dabbled with writing a compiler, a database and even a simple web framework from first principles. I didn't get far, even though I had good mental models of these things and I carefully wrote RFCs and documents for the LLM, specified test cases etc... If you're lucky it will regurgitate some existing code or follow documented guidelines, but when you're on new territory these models won't be able to produce anything good. I would really like to see a single example of someone vibe coding a high quality library or tool with LLMs, I haven't found anything and no one can point me to a complex codebase (say 10,000 lines or more) that was generated using high level prompts that looks decent and doesn't have multiple glaring issues that appear when looking at it in detail.

There's practical considerations with AI like it being a probabilistic layer and not deterministic in the way that compilers and deterministic, yes.

But the "letting go" point is I think about what people enjoy. Personally, I enjoy code. I've had jobs where I really enjoyed what I was doing because they gave me the opportunity to write interesting code. The end product wasn't interesting to me. I had a great job building tools for companies that build a very popular consumer product that I've never owned or been interested in. So the software we were building to assist in creating those products wasn't interesting to me in terms of what it is, but the job was motivating and interesting because of the technical problems I got to solve.

I think that's one of the big divides we're seeing in comments now. There are people who want to build things for the sake of the final product. They're happy because LLMs speed that up by a lot, at least initially. Then there are people who enjoy the process and craft of writing code. For them it's hard to let go of manual coding because that's the enjoyable part of the work.

  > They're happy because LLMs speed that up by a lot, at least initially. Then there are people who enjoy the process and craft of writing code. For them it's hard to let go of manual coding because that's the enjoyable part of the work.

coding enjoyer here, but for me i want it to be an assistant to speed up what im doing (in the "bicycle for the mind" kind of way) or a problem solver for when i got really stuck; i feel like "agentic" workflows miss a lot of this cause its like talking to someone through slack... thats not enjoyable and also too far removed from the actual code/problem im working on; i think there is an opportunity for much better ux here...

Anything built this way will have a hard limit on the ability to scale in terms of functionality and load. Eventually it will get to a point where not even AI can work with it.

It is hard to me to fathom why anyone would find those limits acceptable, especially with years of experience writing code. It seems something like an experienced painter giving up on their art to doodle with crayons, and maybe that could be cathartic. Maybe you get experience with other aspects of your craft.

We’ve already had those limits for years on any corporate project with average (or below) developers.

I work with a lot of AI code, and it’s not any worse than what big teams were already producing.

A small team of 2-3 really good humans is in a whole different league, but that has always been a very rare case.

The ai does what you ask it to do. One thing you can ask is for it to refactor the code base to achieve some goal, for example to make new features easier to add , or to just make it more readable (though that’s a bad goal, no one agrees what is readable, and what AI finds readable may be different from what you think).

I am saying this because I haven’t seen latest gen AI unable to implement something no matter how convoluted the code base. On the other hand, people easily get overwhelmed. These days I have more faith in Claude implementing something for me than for a person, including me , doing it thoroughly enough on unfamiliar code bases (and for any big enough code base, large swathes of it will be unfamiliar to any individual developer).

In order for you to properly guide it through a refactor, you have to understand the code, which becomes less possible as it grows into a mess of spaghetti. I think that's just setting yourself up for a massive amount of frustration. When there are multiple systems that need a refactor, each individual refactor becomes that much more complicated. And trusting the tool that created the mess to sort it out seems a tad delusional.

It's true that AI has an incredible ability to take in a massive amount of information and do useful work with it. But it has no direct ability to exercise judgment, and with each iteration of reasoning and applied changes, its ability to hold the original intent degrades very differently from a human intelligence.

I’ve done many successful refactors without needing to guide Fable or even look at the code beyond glancing when it’s done to confirm the new architecture is sound, and that last step is becoming optional.

Have you heard about the concept of refactoring? AI is also good at that! ;)