You might be turning away people who are just sensitive to being manipulated. I can't find a reference, but it's a common trick for manipulative people to "task" someone with a trivial thing. Once they've done that, it can become more difficult to refuse to do slightly less trivial things, because, hey, you didn't have a problem doing that other thing... so what's the big deal now?