The "every direction" bit is actually something autonomous cars handle far better than humans since they can have eyes pointing in every direction. The hard part is noticing that someone waiting at a crosswalk wants you to stop so they can cross.