I really want to agree, but there are some situations where it does need to be that one doctor. The below anecdote notwithstanding, I do agree that any patient who demands your personal number is the last patient you should be giving it to.

My aunt had a condition that causes fluid build up in her brain. She has a slightly unusual shunt arrangement, and over Christmas (we now know) it got blocked up. She started developing neurological symptoms, and the only person who knew enough about her treatment to be able to sort it out was her doctor.

The actually had his home landline number, but he was away, and it was only due to luck of a call coinciding with the neighbour feeding the cat that they were able to get the number of where he was for Christmas. By that point, my aunt with unconscious in hospital, and might well have died if treatment had been delayed any more.