That's right. The other reason to not give feedback is the rejectee will inevitably start arguing with you about it, demand more detail, try to reopen negotiations, tell you how unfair/wrong you are, etc.

I've tried it both ways. Not giving feedback works much, much better.