In utilitarian terms, self-driving cars just need to be better than the drivers they replace. (Or even stronger: not much worse, because the added convenience is worth something.) Ie we should start to replace the worst drivers already.

In practice, the target seems to be not just the average driver, but the best human driver (or at least the 99%th percentile.) Given the trajectory of improvement we often see with AI, surpassing the average human and surpassing all humans shouldn't be too far apart.