I don't mind it that much for things, but I hate it used towards people.

I think few people are that bad that you need to isolate yourself from them. In most cases, you can learn not to take their criticism, not to give into their games...