Even if the structure remains intact - and storm surges have no problem washing concrete and stone away - the house has to be ripped apart. The flooding ruins sheetrock, furniture, carpet, etc and mold growth is a huge issue.
The wind is usually not that big a deal unless you have a tornado during the hurricane. I have a 100 year old wood framed house that has been though many hurricanes, including Katrina. Generally the worst case wind-wise for most people is that you see some minor roof damage and little else unless a tree falls on the house. Wooden houses certainly do not "disintegrate" during storms.
Debris doesn't cause a lot of damage, particularly if you've prepped for the hurricane (by boarding up windows, garaging cars, etc.) The biggest factors are definitely trees and storm surge.
Pitched roofs do well against storms (30° roof slope is best).
Luckily, we don't have deadly hurricanes.
Whenever I see a report about a tornado or a hurricane in the US I wonder what the damage would be like if people had stone houses instead of wooden houses that disintegrate during a storm and then in turn damage other houses. It reminds me of the Kessler Syndrome.