Accessibility permissions allow you to do a lot of really great stuff with Apple Events that is otherwise just completely impossible. I use it very heavily on my own machine.

It's certainly a powerful ability, and I can completely understand not wanting to grant it, but realize that the need is legitimate (and non-malicious) in a lot of cases.