IME it is much, much more difficult to review code that is the result of a LLM regurgitation than to review human-written code made by someone even a bit experienced in programming - even more so if the human in question is a domain expert.

For one, LLMs tend to waffle and produce code that works but is much more extensive and/or unintuitively organized than code that a human writes. The bigger thing however is testing. LLM code tends to test itself, whereas a domain expert human tends to have many more domain-specific edge cases in mind.

Maybe, yeah. But isn't possible for a human to write "bad code", too?

Does it matter if it comes from a human you don't know or an AI you don't know? If it's difficult to review for the maintainers, then the maintainers should not merge it.

Let me put it this way: if a random contributor opens a PR as a human, then the maintainers will do what they were doing before AI, right? Now if a random contributor opens a PR pretending to be written manually, but actually written with the help of an AI, what will the maintainers do? Exactly the same, right? Try to read and understand the contribution, and merge it if they want to take responsibility of it.

The difference, really, is that everyone and their dog can now open a PR that looks evolved, and for maintainers it's difficult because it adds noise.

> But isn't possible for a human to write "bad code", too?

Sure. But I'd trust someone whom I know to be a domain expert and a good coder much more than some random github account. And once I see anything like "co-authored-by: claude"... oh hell no I'll go over that with a fine toothed comb because I have zero idea what the user prompted into the LLM, which in turn means I don't know what edge cases the author (and by extension the LLM) even knew about.

Yeah I agree. At the end of the day, the question boils down to whether or not the contribution is helping the maintainer, right?

As a maintainer, if it's easier for me to ask an AI to do it, then I will do that. Now depends on what "easier" is. Maybe a question of trust (if I don't trust a random contributor, I may as well do it myself), maybe a question of testing it (if a contributor can show me that it works on their 5 devices I don't own, it may have value).

But the fact that it was written by AI is not the biggest factor, IMO. Proven by the fact that if the AI writes exactly what I wanted, then I will be happy to merge it.

good balanced view. Regardless of how much effort is put into AI writing correctly, humans should review and understand.

the last part is tricky on these kinds of projects.

Just look at the bugs coming out in Linux or other kernels. Often they are very subtle errors based in very subtle misunderstandings.

This is why you'd want a lot of different people involved who in their combined knowledge have a good chance to understand enough to make such a project a success, even IF all they do is review the AI code.

> Often they are very subtle errors based in very subtle misunderstandings.

I'd like to address this specifically. Introducing a very subtle bug can be an adversarial move, right? Say I want to introduce an exploit into a project, I can try to do that. And it is a risk in any open source project where contributions come from "random people on the Internet".

Now my question: as a maintainer, is it better for me to try to convince myself that all those contributions I receive from random people are well intentioned (and maybe miss subtle bugs that were introduced on purpose), or is it better to have an AI replace those contributors?

I don't have an answer, but I think it is an interesting question. As a maintainer, my bottleneck has generally been more around reproducing, understanding the bug and testing. Less around producing the code. And an AI can produce the code I ask it to produce, copying my style, and I can ask it to start over as often as I want.

Most contributors nowadays probably use LLM. And those who don't, often produce code worse than LLM.

If you're not OK with random code, you gotta rewrite everything yourself. So treat their contributions just as inspiration and work from there.

And if you're OK with random code, maybe it's safer to just use LLM and ask it to rewrite the contributed code? At least you know that LLM is not ill-intentioned.

Yeah, that's pretty much exactly what I meant, but you expressed it better.

> At least you know that LLM is not ill-intentioned.

It's more complicated than that, but yeah... I see what you mean here and I agree :-). Probably the LLM is not trained to insert bugs into my repositories, I guess? But anyway, in the future I could totally imagine that some open weight models are "mainstream" and "audited", at which point it sounds harder to target my repo from the LLM than from a PR by a random account.

I played with the idea of starting a new Linux distro called "Slopware", which accepts basically everything from either humans or AI but requires rigorous testing and development processes, raising the quality to levels realistically unreachable by single human developers.

Not sure if that naming choice would have been a good idea though. :)

Isn't that what Omarchy is trying to do? Sloparchy maybe, for the name? :-)

GNU/Slop