It has more to do with training a neural network. The environmental cues for driving on a road are optimized for human vision. This becomes important when you think about it from a machine learning perspective. Fewer inputs are better for many reasons. Non-visual inputs will sometimes be in disagreement when the visual inputs during training which leads to a worse model.

If everyone could take some deep breathes and press pause on their emotional response to Elon Musk (and not assume everyone who happens to agree with him has a Musk tattoo) then they would fine plenty of rational arguments from an engineering perspective.