On Android (at least with Samsung Galaxy 6+) you can put your phone into do not disturb mode (settings, sounds & vibrations, do not disturb), then set up Custom Exceptions to allow calls and texts from Contacts Only.

I believe you can also do the above on an iPhone, but not 100% positive. My GF has it setup and I'll have to ask her how she did it.

As an added measure, under phone settings, you can block calls from unknown callers as well.