If that's the case then I'm not sure how it is a benefit to society.
As with automatic escalators, there are two primary benefits to society: greater aggregate safety[1], and freeing up people's time. You point to the lack of a human operator to hold responsible as a detriment, but the statistics point the other way around: human drivers are horribly irresponsible and even if we can punish them when they do wrong that doesn't substantially change their behavior. Freeing up people's time is self-explanatory.
That's my point. I 100% agree, if self-driving cars did make things better, than the safety aspect is a good thing.
I think the freeing up people's time is okay too but only if all of society has access to AV's. However, like many innovations of the past, advancements which are not accessible to all people just free up the time of some people while offsetting burdens onto others (ie., those who can't afford AV's). BUT, that is a separate issue.
[0] It is just one data point, but it's a bad sign to be so quick to kill a pedestrian so soon out of the gate.
That's a strange way to put it. Surely if someone is not driving their car, they can't be responsible if the car kills people (assuming of course the car was well-maintained and the model was legal). I don't see how that's a bad thing, since they have zero control over the situation.
Do you mean that it could make car users feel less responsible for deaths, even though cars will presumably still kill people?