I would say safety dictates that the car would still have to be safe when such external signals are either missing, wrong or flat out hostile. At that point I don't really see the value they provide.

Make it self driving on specifically marked highways and switch to manual on normal roads. That would free up hours for many people and might even improve traffic congestion.

And where is this going to come from? Our public infrastructure is crumbling after decades of insufficient spending and maintenance as it is. It will take a significant social shift to reverse this trend and stop robbing the future to support short-term political agendas of neoliberal austerity and privatization that merely enrich the rich further.

If we can accomplish such a shift, there are already much more worthy ends to put the money and political will towards: increased public transportation, rebuilding of streets to safely support non-vehicle traffic, public ev infrastructure, etc. Spending massive amounts to subsidize the few rich who could afford autonomous cars would just be further regressive policy.