I think the better question is why we try to replace cars 1 to 1, even if it is in terms of mileage rather than number of vehicles. The problem isn't just drivers, the problem is cars. For most trips there are superior technological alternatives, there's just no political motivation to implement them because infrastructure is boring and it's easier to justify throwing more money at roads in the short term than investing in long term urban renewal projects to build and expand tram networks, underground transit, dedicated bus lanes and bicycle-first road networks. Especially now that the idea of "walkable neighborhoods" has cynically become a culture war issue rather than something that's objectively improving quality of life while reducing emissions and resource waste.

I like reducing car use too, but that's not what autonomous driving tech in particular is useful for.

Also I expect that a single autonomous taxi can replace more than one private car.