I sympathize with your view very much, but there's also the agony that results from unconstrained breeding of cats and dogs to consider.
In developed nations feral cats and dogs are often hunted down and exterminated. Even while they live their lives in are often no picnic, whether they're in a shelter or in the wild.
Also, if you have, say, a cat and you don't get them neutered and you let them out what are you going to do with all the kittens? If you don't let them out, the endless confinement could be argued as a form of misery itself.
Then you could argue about having pets at all, but then what about all the cats and dogs that live miserable lives in shelters or are unwanted or abused in other homes? Wouldn't they be better off with you?
There are two different situations, though: preventing reproduction and preventing potentially undesirable hormone-fueled behavior. You don't need to castrate just to prevent reproduction.
I don't think there'd be anything wrong with performing a vasectomy, for example. I'm not sure if there's a similar analog for female animals, though.
Since we are the ones who introduced cats in the first place, we are responsible for the consequences. Violating the bodily autonomy of individual cats is a small price to pay to avoid a population explosion that would ultimately result in the large scale killing of countless birds and other small animals.
That's my general take on the subject. To your point about anthropomorphism, I disagree, I think assessing one's health and bodily integrity is a primal instinct that is most likely shared by most animals. I can't say for sure but I have experienced a lot of body horror myself (torn & deformed limbs) and it was never higher thoughts that prevailed, even in situations without pain (like after surgery).