I was a cyclist for many years and stayed to the sidewalk as much as possible. As you said , must fatalities occur when a car traveling at a high rate of speed rear ends a bicyclist. I'd rather deal with the non fatal accidents that happen on the sidewalk then potentially fatal accidents on the street.

People fly by you in their car, inches away, while on the street on a regular basis, and you have no control over this. On a sidewalk, I have to be more observant, but more accidents there will be avoidable if I am paying attention.