tjr

One possibility may be that we normalize making bigger, more complex things.

In pre-LLM days, if I whipped up an application in something like 8 hours, it would be a pretty safe assumption that someone else could easily copy it. If it took me more like 40 hours, I still have no serious moat, but fewer people would bother spending 40 hours to copy an existing application. If it took me 100 hours, or 200 hours, fewer and fewer people would bother trying to copy it.

Now, with LLMs... what still takes 40+ hours to build?

But in any case, the question really refers to, can the LLM-generated software be copyrighted? If not, it can’t be put under any particular license.

If you keep some for yourself, there’s a possibility that you might not churn out as much code as quickly as someone delegating all programming to AI. But maybe shipping 45,000 lines a day instead of 50,000 isn’t that bad.

I would further think that the same practices could be applied to any software, whether if it was safety-critical or not. If software development isn't engineering, it's not because it can't be, but because not every project is critical enough to warrant the extra time and expense.

I think a similar train of thought applies here. As the article points out, skipping reading the code is probably not a good idea for safety-critical software, but for less critical things, it may be fine.

If someone told me that they applied avionics-level rigor to an iOS puzzle game, I would think that (a) it's probably very solid software, but also (b) they were probably wasting their time. But on the flip side, if someone wanted to spend their time making their puzzle game rock-solid, I don't think that's necessarily a bad thing to do. It's not harmful to have especially robust puzzle games.

Is it worth it to review LLM-generated code? For some projects, maybe not. Even for many projects, maybe not. But I'm not sure that it should be frowned upon either. It might turn up something interesting. Put in whatever level of rigor matches your project needs, personal interest, and schedule!

I personally disagree with the rulings thus far that AI training on copyrighted information is "fair use", not because it's not true for human training, but because I think that the laws were neither written nor wielded with anyone but humans in mind.

As a comment upstream a bit said, some people are now rethinking even releasing some material into the public, out of not wanting it to be trained by AI. Prior to a couple of years or so ago, nearly nobody was even remotely thinking about that; we could have decades of copyrighted material out there that, had the authors understood present-day AI, they wouldn't have even released it.

In your example scenario, it sounds like Bob is vibe-coding a project in assembly language, and then using an existing assembler to create the final binary?

I have not tried it, but I would imagine that current LLMs are relatively weak on generating assembly language, just due to less thorough training compared to higher-level languages, but even if so, that's surmountable.

As for, what I think you are suggesting, having the LLM also do the assembly step? Again, in theory, sure, but I would think just using the existing known-good assembler would be preferable to training an LLM to convert assembly language to binary. I'm not sure what you would gain in terms of either speed or overall productivity to have the LLM itself do the assembler step?

Which also makes me refute the idea that AI coding is just another rung up on the programming abstraction ladder. Depending on how much you delegate to AI, I don't think it's really programming at all. It's project management. That's not a bad thing! But it's not really still programming.

Even just in the context of my human team, I feel less mentally engaged with the code. I don't know what everything does. (In principle, I could know, but I don't.) I see some code written in a way that differs from how I would have done it. But I'm not the one working day-in, day-out with the code. I'll ask questions, make suggestions, but I'm not going to force something unless I think it's really super important.

That said, I don't 100% like this. I enjoy programming. I enjoy computer science. I especially enjoy things more down the paths of algorithm design, Lisp, and the intersection of programming with mathematics. On my team, I do still do some programming. I could delegate it entirely, but I indulge myself and do a little bit.

I personally think that's a good path with AI too. I think we're at the point where, for many software application tasks, the programming could be entirely hands-off. Let AI do it all. But if I wish to, why not indulge in doing some myself also? Yeah, I know, I know, I'll get "left behind in the dust" and all of that. I'm not sure that I'm in that much of a hurry to churn out 50,000 lines of code a day; I'm cool with 45,100.

I installed some drywall a few years ago. I plan to install a room of drywall exactly never again. Not worth it for me to buy the best drywall tools.

But I have installed multiple wood floors, replacing old carpet, and would do so again if needed. I’d rather get higher quality tools there so I can keep them and reuse them for years.

https://mitpress.mit.edu/9780262526401/artificial-intelligen...

envisions a future of AI assistance that looks not too far off from today.

So I think this question needs to be asked in the context of particular projects, not as an industry-wide yes or no answer. Does your particular project still need humans involved at the code level? Even just for review? If so, then you probably ought to retain human-oriented software design and coding techniques. If not, then, whatever. Doesn't matter. Aim for whatever efficiency metric you like.

Not entirely unlike other code generation mechanisms, such as tools for generating HTML based on a graphical design. A human could edit that, but it may not have been the intent. The intent was that, if you want a change, go back to the GUI editor and regenerate the HTML.

I do not see LLM coding as another step up on the ladder of programming abstraction.

If your project is in, say, Python, then by using LLMs, you are not writing software in English; you are having an LLM write software for you in Python.

This is much more like delegation of work to someone else, than it is another layer in the machine-code/assembly/C/Python sort of hierarchy.

In my regular day job, I am a project manager. I find LLM coding to be effectively project management. As a project manager, I am free to dive down to whatever level of technical detail I want, but by and large, it is others on the team who actually write the software. If I assign a task, I don't say "I wrote that code", because I didn't; someone else did, even if I directed it.

And then, project management, delegating to the team, is most certainly nondeterministic behavior. Any programmer on the team might come up with a different solution, each of which works. The same programmer might come up with more than one solutions, all of which work.

I don't expect the programmers to be deterministic. I do expect the compiler to be deterministic.

I've been a project manager for years. I still work on some code myself, but most of it is done by the rest of the team.

On one hand, I have more bandwidth to think about how the overall application is serving the users, how the various pieces of the application fit together, overall consistency, etc. I think this is a useful role.

On the other hand, I definitely have felt mental atrophy from not working in the code. I still think; I still do things and write things and make decisions. But I feel mentally out of shape; I lack a certain sharpness that I perceived when I was more directly in tune with the code.

And I'm talking, all orthogonal to AI. This is just me as a project manager with other humans on the project.

I think there is truth to, well, operate at a higher level! Be more systems-minded, architecture-minded, etc. I think that's true. And there are surely interesting new problems to solve if we can work not on the level of writing programs, but wielding tools that write programs for us.

But I think there's also truth to the risk of losing something by giving up coding. Whether if that which might be lost is important to you or not, is your own decision, but I think the risk is real.

Per corporate policy -- in the name of safety and legal reasons -- the most extensive public uses of AI, like vibe coding and agentic systems, aren't really options. The most common usage I have seen is more like consulting AI as a fancy StackOverflow.

Will this change? I personally don't expect to ever see pure vibe coding, with code unseen and unreviewed, but I imagine AI coding uses will expand.

Are there any software products that you think will survive?

Aerospace here. Roughly this would be typical:

- comprehensive requirements on the software behavior, with tests to verify those requirements. Tests are automated as much as possible (e.g., scripts rather than manual testing)

- tests are generally run first in a test suite in a completely virtual software environment

- structural coverage analysis (depending on level of criticality) to show that all code in the subsystem was executed by the testing (or adequately explain why the testing can't hit that code)

- then once that passes, run the same tests in a hardware lab environment, testing the software as it runs on the the actual physical component that will be installed on the plane

- then test that actually on a plane, through a series of flight tests. (The flight testing would likely not be as entirely comprehensive as the previous steps)

A full round of testing is very time-consuming and expensive, and as much as possible should be caught and fixed in the virtual software tests before it even gets to the hardware lab, much less to the plane.