That's what I do. If a call comes into my phone from a number that isn't in my address book, it just gets sent straight to voicemail. My phone won't even ring.

The only reason I don't just drop the call completely is to cover situations where a someone not in my address book might be calling me for something important.

That has never actually happened yet, though.