People routinely break their own bones when mountain bike riding. There is plenty of momentum available to do serious damage.

It's a shared trail. Would it be safe for hikers if someone rode a bike at 100mph? Of course not. To paraphrase Churchill, we've established the need for a speed limit, so the only thing left to discuss is which speed to set the limit at.