Agreed, this is annoying. Little things like this make a big difference in voice UIs.

But not quite as bad as the old Hertz Neverlost (r) system, which would tell you, in a huffy voice, "Please return to the highlighted route!" Rather than just recalculating the route from where you are.