Even if overall self-driving cars are safer, they won't feel safer to the individual if the situations in which it will kill them are situations they could avoid if they were driving themselves.

The self-driving car might be able to avoid that t-bone situation more often than the driver can, but if the death scenario for the self-driving car is it gets confused and drives into a river, that will feel unacceptable to a human because it seems so simple to avoid to them.

Most people aren't concerned how many other people survive, but are generally primarily interested in their own survival over population-level statistics.