Frontier models can easily one shot what I ask for, and make adjustments as ask to quickly make an app that meets what I wanted.

But often when I programmed outside of work (and at work too) I learnt something.

Quite often that my favourite pattern is an antipattern in some use cases.

That might never happen again at the rate of development.

But paraphrasing

> I can produce in a month what took years for teams of hundreds of people

Is completely insane.

If you can really do that Anthropic would like a word with you.

To be fair, in the game dev world there are a lot of things you can do now that would have been impossible, or required 10,000x the number of devs that it does now, e.g. making No Man's Sky without the proc-gen tools that can exist now would have been physically impossible. Point being, it's perfectly normal to be able to say, "using the tools available now, I can build what would have taken tens or hundreds of devs years of work in the past", in game dev.

The project I'm building rn is a black-box rebuild of a client that has been being developed for ~22 years, and I've rebuilt significant portions of it over the last 2 weeks. That was, quite literally, equivalent to tens or hundreds of peoples' labor.

Taking the end result of a 22 year old dev process and telling Claude to rewrite it, is in no way equivalent to doing 22 years worth of the work put in by the developers working on it.

That’s like saying replicating a paper that was the result of 22 years of research is the equivalent of 22 years of the author’s labor.

No Man’s Sky isn’t the equivalent of 10,000x the devs who made it. Because the procedurally generated content isn’t equivalent to handcrafted content. I’m also not aware of any procedural generation techniques in No Man’s Sky that weren’t possible 30 years ago.

As far as Tamriel rebuilt goes, if you’re talking just about just reimplementing Daggerfall in the Morrowind engine, you might be able to do that in a month of vibecoding.

But that’s not hundreds of dev years worth of work.

And no, the procgen techniques and capabilities utilized by NMS were not available 30 years before NMS (important distinction, because NMS is already 10 years old, and 2006 vs 1996 is quite a big jump in game complexity as well).

Saying, "isn't equivalent to handcrafted content" is a fallacy, because hand authoring doesn't denote any particular level of quality or outcome. You can absolutely hand craft the NMS assets and game systems, but not at the scale that NMS did using procgen.

Wrt TR, it literally has tens of years (if not hundreds) of labor hours that have gone into it.

I'm not sure where you got the idea it's about recreating Daggerfall- it's not, it's about implementing the rest of the Morrowind province beyond just the island of Vvardenfell- but it's a massive, multi-decade project with hundreds of contributors.

In contrast, the value of a program is typically just the "black box" service it provides. Indeed, the program itself has always been opaque to most business stakeholders. Before, they paid human programmers and got back a program they didn't understand but could use; now, they pay an LLM provider and get back a program they don't understand but can use. In the pre-LLM era, the economic value of "understanding" and "shared knowledge" about the programs themselves was only that it made maintenance and changes cheaper. It never had intrinsic value, except as intellectual satisfaction to programmers (I was... am... one of those who found that value).

They come up with the features and the implementation is an unimportant detail in realizing their designs.

In reality implementation details frequently drive design and in most cases they are at least highly intertwined.

I can’t count how many products and features I’ve built that ended working completely differently than the initial design, or how many times I was able to simplify a workflow after spending a days thinking about the implementation details.

You can kinda do something similar by spending days or weeks going back and forth with an LLM, but at least in my experience that doesn’t really save much time and I don’t understand the problem quite as well by the end of it.

The economic value is still the product you delivered. Your understanding of the implementation meant you participated in the work of the "product manager". I think most good devs do. That might be an argument for devs to transition into designer/PM roles. That is effectively happening already, whether or not role titles have caught up.

But in any case the value isn't in the understanding itself. It just happens that the understanding still helps create better block box products. Also, I wouldn't count on this "human advantage" lasting long. I hope it does, but ime LLMs are now good enough to do this "co-design" too, and will only get better.

> In the pre-LLM era, the economic value of "understanding" and "shared knowledge" about the programs themselves was only that it made maintenance and changes cheaper.

This only holds if there is a bright line separation between the problem, the solution, and the implementation of the solution. And that is not the case.

Understanding the implementation frequently improves your understanding of the solution, which improves your understanding of the problem, which leads to better solutions. Sometimes it leads to realizing that there is a completely different problem you should be solving.

So what I’m refuting is that there is no value to understanding the program.

But beyond that black box products that users know how to use but don’t understand is also a vast oversimplification.

Frequently software is useful specifically because it forces users to adopt a process not just because it allows them to do something they were previously doing faster. In those cases the process is usually informed by the implementation, so the user needs to understand parts of the implementation even if it’s indirectly.

Practically speaking, I don't think the "black box" metaphor is as broken as you're saying, though I like the point you made. There is still a ton of software (the majority, I think -- crud apps of all flavors) where an unthinking, untechnical PM wants something they can specify, and an LLM can give it to them, and it will more or less solve their problem. It won't be the best possible thing, but it will be good enough.

Learning is not an output-efficient process but is a fantastic outcome. AI lowers some portions of the development workflow but not yet every part, and, efficiently produced outputs aren't always the major outcome desired. In the real world / outside the benchmark lab, you care about robustness, reusability, composability, security, and more. Targeting speed to acceptable average output inferred from a prompt or similar benchmark metrics miss a big part of the game.

A lot of progress comes from taking assumptions and breaking them. There was a time when everyone assumed that locked checkouts were absolutely necessary for version control, for instance. You could argue that AI could learn how to do this well but humans have context that machines won't. This is just an extension of the fact that things with independent existence will have different impressions of the world. If they didn't they would functionally be one thing.

> The result may be impressive, but if the only beings that can move math forward in future are machines grinding away on billions of scenarios that no human would have the patience to undertake

Seems like the same thing as neural networks. You have something like a few thousand parameters for computer that works well, but no underlying algorithms and formulas, like the shaders.

No one from OpenAI is willing to give a talk on any of their 300+ results.

There is a fixed landscape of logically provable mathematical statements. Normally, the process of exploring that landscape simultaneously improves human understanding. Now, formal proofs and actual knowledge have become decoupled.

You can ignore the AI, but that doesn't mean the AI ignores you. Suddenly you become a little isolated society like the Amish and everyone else thinks you're kind of quaint and backwards.

Also AI Boosters: These entities are too intelligent to explain what they are doing to people...

I mean, Terry Tao actually used ChatGPT to explain one of the earlier OpenAI results. Humans working with AI can explain this stuff. If nothing else, OpenAI could be releasing things at the rate that it can get human mathematicians to explain the results. Which is to say, "yes, I can blame them", I don't expect them to behave differently but I can certainly blame them.

It'd be some sort of... epistemic step backwards by civilization.

It's still of course useful to have some of this distilled understanding be written down, just like I had it generate high level descriptions of our code at work so it doesn't have to rediscover everything every time, but that's more of a cache/optimization.

Indeed, as we speak mathematicians are taking the time and effort to (successfully) digest and explain these results.

It's the same in the software engineering, there is a shift from software engineering to managing SWE agents, with the associated loss of expertise.

It's a hard challenge, but it is not unique or impossible to solve. Aviation industry faced it long time ago with autopilots. Autopilots took over nearly all aspects of routine flight and left pilots without time/ability to practice their skills. While these skills are needed in an emergency. Well, the industry solves it through certification, training and simulation.

AI used in a framework of human augmentation generally gets that for free. Used under the mistaken notion that humans can be easily replaced by AI gets you to the big complaints here.

(These are just my personal read on the topic -- the market is in AI frenzy where AI valuations are disconnected from productive capacity -- heavy on my mind and looking to get some writings out on the topic soon).

I argue this also applies with beauty. We got rid of that, also saying it was unnecessary. The result is sad.

If you care about results, and AI doesn't cap out soon, the best humans can do is stand back and do nothing. Useless lumps that aren't able to keep up. Mathematicians wouldn't insist on teaching your dog their new theorem before calling their job done; why should an AI insist on teaching you?

If humans care about human creativity, and don't want to be mere passive observers, they need to make a choice.

Professional mathematics is whatever gets you funding so you can keep doing it. Today this is spamming papers and applying for project grants. Turns out the machine can do that quite well, oops.