If your customers are somewhere, you could be there too, and that often has the least amount of friction or is the easiest choice, so it's understandable why so many people elect to do it. But another thing you could do is bring them to where you are.
Most people who run businesses opt for meeting people where they already are though.
It's almost every line of business. You probably need a mildly active twitter/facebook/instagram presence if you have a nice nail salon, or toy store. Anything not in a huge popular franchise.
edit: this sentiment puts me in mind of a watch or shoe repair shop guy in the 1960s who refuses to install a telephone because he wouldn't want customers who think that they could just loudly barge into whatever he was doing at will. Telephone culture brings a sense of unearned entitlement that attracts bad people. But you miss out on business worth far more than the irritation without a telephone. Keep the telephone outside the actual shop and hire someone else to answer it. They used to call answering machines "phone butlers."
I really wouldn't want those people as customers. Really. You get enough toxic customers through organic acquisition.
Hint: I am not alone in this.
Wow, that's a depressing thought.
I think I'd prefer to find a different line of business, if it were me.