Riding on the sidewalk is dangerous despite the sidewalks being usually empty. There's another factor: when sidewalks become intersections. Very few drivers check for cyclists on the sidewalk before going through an intersection. From what I understand this is shown quite clearly in the crash statistics.

Many dedicated cycletracks have similar problems. Here in Austin there are several that I refuse to use because drivers far too frequently turn into my path without looking. I ride in the lane so that I can be seen. It's counterintuitive to most drivers, but visibility is often the deciding factor in bike safety. This would be more obvious to drivers if they spent more time cycling. If a cyclist is stuck using a path with poor visibility, I find paying attention to the cars and using light touches of an air horn when approaching an intersection to help. But I still prefer being visible to that.

More generally, intersections are relatively more dangerous (crashes per person mile or something like that) than straight segments of road for any mode of transportation using the road.

It's not a matter of principal, it's a matter of statistics. Even empty sidewalks are more dangerous to ride on than riding with traffic.