Hmm but wouldn't I have those people in my phone? I'm just trying to think of someone I care about who I wouldn't have in my phone. Or someone who would actually be upset with me. If a friend had to borrow someone else's phone they'd probably either call twice or send me a text. Either way they wouldn't be insulted. I guess we associate ourselves with different people.

a potential customer or partner.

Customers never call (they can't) and a partner, at least one of my partners, would never be upset. I see what you're saying though. Just different businesses & people associations.

I wouldn't be interested in doing business with people like these. Odds are I'd insult them in some way I didn't realize down the line anyway.

Aside: these people are always the 20%, IMHO.

Sure; don't form a joint venture with someone like this. But refuse a customer/client over it? Good luck in business.

Some customers cost more than they are worth to you. That's not to say that you should be an asshole, but I'm not their indentured servant either.