As someone who did competitive weightlifting early in my life I don't think it's true.

First of all, a healthy body is very good at adaptation, i.e. if properly trained, your bones, muscles and other tissues get thicker and less prone to tearing, breakage etc.

Second, you will be surprised at how much pressure could daily activities put on your tissues, e.g. your vertabrae disks get higher pressure when you are wobbly sitting than standing even with some additional load. And in my experience people who train their body for extreme conditions have less problems in their elderly than people who never trained much at all.

And as you've noticed, most sportspeople have troubles with their serious traumas, not due to routine trainings. Plus of course this traumatization level varies from sport to sport.