Of course it is. Doctors work for you and offer advice. They can't force you to take medicine unless you're involuntarily commited.

Do your own research. Get a second opinion. Etc. I rarely end up taking medication that's prescribed to me and overall I think I'm the better for it.