So I could imagine debating a point while being in the right.
I absolutely can. Every single time I've gotten detailed rejection note it turns out to to be over pedantry (or rather, the fact that you didn't follow the letter of the instructions perfectly, god forbid you show some creativity).
It's really hard to not send a point-by-point rebuttal of their nonsense, but it never leads to anything.
Some people are fantastic on the happy path but horrible on the sad path.
Instead I sent something akin to “we decided on another candidate”
So the candidate gets feedback along the lines of: "Thank you for participating in our interview process. Unfortunately, our panel decided you weren't the best fit for position X at this time, because ...reasons.... Under company policy, we won't accept further applications from you for one year from today, but we would encourage you to apply for a role with us in the future".
There is a chance they will reply back to HR arguing, but it is their job to be polite but firm that the decision is already made, and that they can apply again in one year (and not pass anything back to the hiring manager).
The key is to think long term and about the company as a whole - the candidate who gets helpful feedback and is treated fairly is more likely to apply again in the future (after the mandatory cooling off period), when they might have more skills and experience working somewhere else. There is a finite qualified labour pool no matter where you are based, and having the good will even of rejected candidates is a competitive advantage. The message should be "not now", rather than "not ever" (although of course, if they do go on some kind of rampage, they could turn the not now into not ever - that's a bridge burning move). If a tiny percentage go on a rampage, but the company protects the individuals from it, and has lots of counteracting positive sentiment from prospective and actual staff, then it's still a net positive.
I didn't last very long. Candidates would see the feedback as an invitation to prove me wrong or argue with my assessment. I got a few very angry e-mails from people who took their rejection very personally and made it clear that I was their enemy. One person (who was actually very unqualified) even went on a mini rampage across the internet, trying to "name and shame" my company and even my personally for the rejection. There were even threats of a discrimination lawsuit.
So I stopped. It's back to something like "We've decided to proceed with other candidates"