Well, that point is most likely lost on someone who dies in a crash of these cars. IE, the difference is big from some viewpoints but irrelevant from others.

I think we can see the point where objections to self-driving cars start to become strong. It's not some cooked-up version of the "trolley car problem" but a situation where a self-driving car does something a human observer naturally interprets as "wrong" and "dumb", does it in a way that kills someone and the designers give the answer "sorry, not a mistake, just a necessary design compromise. This is still, on-average safer than a human". I think this would produce a strong emotional effect regardless of any "generally safer" assertion.