The most obvious intermediate stage is designated sections of highways in which self-driving cars can operate without active human drivers. The question though is whether that's an interesting enough use case to push through all the legal/regulatory/etc. changes that would be required.
For day to day driving, the cars should either be fully autonomous or have the automated systems limited to intervening in dangerous situations (like current brake priming systems and the like). If the driver is able to take (primary) control, they should be required to be in control.
Given this "stop, your turn" option, IMHO it would be horrible to also deny the driver from taking over during normal driving conditions, seeing as though computers will certainly make mistakes. Allowing that option to exist, however, should not equate to the human 'driver' being liable for not taking control when the computer failed.
Basically, you should give passengers an emergency stop option and not make them liable for not using it.
The stop to change control thing makes sense to me.
This computer drives much more conservatively than a human driver, so much so that I'm not sure a human failsafe driver would grok what is going on.
Waiting for a railway crossing to be completely clear before proceeding is not something most drivers do (ehem, especially at that particular intersection), and may seem like odd behavior to a failsafe driver behind the wheel. Similarly, waiting for a bicyclist coming from behind may also seem odd. For one, he may not have that information since he is less attentive in 'failsafe' mode... or many human drivers would see the cyclist and proceed a bit more quickly to make the turn to get out of traffic and free the intersection for the oncoming cyclist (and cars behind him).
On the other hand I would have probably pulled the plug on the computer driver when Mr. Indecisive-Cyclist was in the road. The computer handled it fine, but as the failsafe, am I sure that the computer will handle it well?
This intermediary stage is going to get really weird, I don't even know how you'd manage liability in this world.