> I just find it very hard to understand the perspective of those that are opposed to self driving cars

Consider it in terms of human agency: People expect the use of "consumer electronics" and "appliances" to be mostly unpleasant, frustrating, buggy, full of arbitrary restrictions, and overmonetized.