Kind of ironic that a doctor would say that to you. Just try getting a doctor's direct line or cell. It's almost impossible, they have receptionists because they'd be overwhelmed with patients calling them with stupid questions all day if they didn't.

Of course, you could also just hire a remote assistant to field phone calls for you if you wanted.