If I get a review at work from a bot that is annoying, misunderstands, etc I just ignore their reviews, or tell the bot to leave me alone. The bot is there to help me, and if it doesn't - that's not my problem. There's also always a human reviewer. For any automated feedback for PR's I think there should be "manual review" requests. If you disagree and the bot can decide that there is real human effort involved, then just let real humans also make the effort to review.
This is the key idea: to spend the human effort from maintainers, where there has been human effort from contributors. The AI should be able to determine the amount of human effort in a contribution. The bot should be an aid to make sure the human maintainer effort is spent only where there is human contributor effort.
If the maintainer wants a LLM to review my PRs, let the maintainer use a LLM. Don't make me deal with abrasive chatbots when you want to deal with abrasive chatbots. I don't want an LLM to pre-screen my work to decide if a human should review it; a human should decide if my contribution is worth it. Otherwise, that project is not worthy of my modest contribution.
But that's the problem at hand: if there are 999 low-effort automated AI contributions for every human contribution, either all 1000 contributions must be pre-screened to gauge "human effort" and the human efforts get human review, or all PR's get 1/1000 the human attention.
That's a super weird way of putting it, which makes me hesitate to agree with you.
buuuut I agree that the LLM bot review experience is usually miserable, as the persona these things use is insufferable. It's a know-it-all only-speaks-in-imperatives thing, where the speaker lacks the actual merit that would grant them the right to that style.
LLMs can be very useful for code review, but the existing product implementations took a bit too many pages out of Kafka's works and are a bit too real in simulating a dysfunctional bureaucratic org.
Yes that's weird sorry. What i meant is it belongs in the uncanny valley. I don't have any good/bad feelings about test suites and CI feedback, it's very binary. Receiving a mail about a comment on my PR raises my hopes that it reached the maintainers (or interest from other contributors); receiving LLM text destroys my expectations.
Then, about the content itself, it has a very patronizing tone. Saying out loud every single thing that's in my +25/-3 PR is really infantilizing. I would probably have slightly less negative reactions if it only commented to suggest improvements, instead of patting me on the back. See this example: https://github.com/TwiN/gatus/pull/1712#pullrequestreview-46...
Bots don't understand how their words can hurt people's feelings, and can even less understand that talking to a LLM is even more sad/infuriating for actual humans. I can handle CI/tests just fine, but LLMs playing on my human emotions will have me swearing in no time.
I contribute to free software projects because i want user control over the machine (that was the idea, long before LLMs). Slopware might technically be FLOSS, but it's certainly not in the spirit of FLOSS.