It's fine to say no to requests. I don't like how organized the charitable industry is in the US and the high-pressure tactics used to solicit donations. I prefer to donate anonymously or tell charitable solicitors that the more often and elaborately they contact me the less likely I am to contribute to their cause. Frankly I prefer just getting cash directly into the hands of people in need and don't burden myself with worrying about which cause should have priority over others.
I don't enjoy having my empathy tugged at by people who only care about me because they want money from me. This experience does not leave me feeling like I have contributed to my community and helped a person. It leaves me feeling used.