Not everyone drives equally safely. For many drivers, switching from manual to a self-driving car will increase their probability of a collision. Maybe in the future this will not be the case, but until self driving cars are proven safer than the most cautionary drivers, some people will increase their risk by switching.