I tried switching to Dvorak, and encountered exactly those problems. Perhaps if every keyboard instantly switched to Dvorak layout the moment I touched it, then it might have been worth it.

Having to think about which keyboard layout I was using added more time and stress than was ever going to be repaid by the supposed benefits of using Dvorak.

I gave up after about 3 months, reverting purely to using Qwerty.