if a street changes into a one-way one day (signaled by a sign) relying on a map will lead to a big unhappy problem.

sure, if you consider everything selfdriving that works on a NASCAR track, then yes, a map is sufficient, but if we are talking about driving on public roads then recognizing and "obeying" signs visually seems like a hard dependency.