A great way to explain this to people would be to ask them if they've ever seen a panoramic photo glitch, and tell them that's what the car would be seeing most of the time (well, assuming anything is moving, like a car or a person or a piece of trash).
Visual slam is still linear-algebra/geometric/keyframe based traditional computer vision (including variants that incorporate GPS/accelerometer info). I think the state of the art is stereo LSD-SLAM, but I could be wrong.