You might change your mind if you'd ever met an arrogant doctor whose biggest concern was their own ego and you didn't have a second option. I'm met a few doctors who are so rude and arrogant they wouldn't even pass the HR screen at anywhere I've worked. Depriving people of choice just gives assholes even more of a chance to be assholes, because people have no other alternatives than to go to the assholes.

You're right - depriving people of choice can cause bad doctors to stick around longer because patients are forced to go to them.

This is why we need something like single-payer - no one should be forced to go to a bad doctor because their private insurance decided to only have one or two options in-network locally.