In my opinion, saying just "no" it's quite harsh. That can create distance instead of building team (respect, trust, good manners etc.).

In such situations I usually say something like "I'm very sorry but I'm extremely busy. Could you send me email with your issue, please? I'll come back to you as soon as I solve the problem".