I also don't agree it's exclusively a matter of luck whether human error results in an accident. It's usually experience and good reactions whether you're able to recover from human error fast and safely enough to prevent an accident. Much like how cyclists have to learn to read the road and pedestrians have to learn how to safely cross busy roads, motorists should learn how to handle shitty situations. Please remember that sometimes those situations happen for reasons outside of the motorists control such as jaywalkers, animals in the road, other motorists making mistakes or even cyclists not following the rules of the road. This absolutely means motorists have a significant responsibility to drive safely but also means that it's not just a matter of luck whether someone forgetting to indicate results in fatality; that's simply not the reality of things.
I agree there is a possibility but it's not even common likelihood (for the reasons I described above). I also agree (as I've also said above) that drivers do have a significantly greater responsibility to drive safely and I'm absolutely in favour of penalising people who have caused accidents....however penalising everyone for every minor mishap doesn't help improve road safety. It effectively just becomes another road tax. If you want to penalise bad drivers then go after bad drivers; but let's not throw the baby out with the bath water.
I don't have a principled argument why someone else shouldn't regard all riders the same way. It's arbitrary where you draw the line between risks that are reasonable to take because you feel like it and risks that aren't.
Just because you don’t kill someone every time you drift through a lane on a corner or don’t stop at a light doesn’t mean that you never will.