It’s (not) self-driving in the same way planes do (not) fly. Just because something doesn’t do things the way we (or our or a bird) does doesn’t mean it doesn’t achieve its goal.

if a street changes into a one-way one day (signaled by a sign) relying on a map will lead to a big unhappy problem.

sure, if you consider everything selfdriving that works on a NASCAR track, then yes, a map is sufficient, but if we are talking about driving on public roads then recognizing and "obeying" signs visually seems like a hard dependency.

Then how about an AI misclassifying a firetruck for a road?