* Will people, on mass, accept being driven around by a machines, on public roads.
* How will they deal with "trolley problem" type ethical scenarios? How will these decisions be made.
* How will self-driving cars cope with rural roads (where they are arguably MORE useful than in Urban settings better served by public transport)?
* How will they cope with signs/marking intentionally designed to fool machine vision systems. Who will be liable for accidents caused in these cases?
There are other issues too. However, overall, I think with some infrastructure investment we could have had self-driving cars since the late 90s early 2000s and there have been many prototypes. It's the ethical/social issues that are the bigger barrier in my mind.
They won't, but neither can humans. Signs aren't secure. Anyone with a wrench can take down a stop sign or put up a fake speed limit.