I've been riding on carbon rims on both hard tail XC bikes and my full suspension trail bike for six years. I've also had aluminum rims on my single speed and hard tail trail bike. I can't count the number of times I've hit the carbon rim and had it be just fine, but all of my aluminum rims have some degree of denting from much lesser riding.
Between the bikes with carbon wheels there's about 12,000 miles of trail riding and they've been just fine.
After I dented the rim on the I9s and had a spoke break I decided a good winter project would be rebuilding the wheels to carbon rims because my previous experiences with them have been so good.
I think the primary benefits of carbon are stronger sidewalls (less likely to dent) and that they cannot be bent and thus can't go out of true (unless the wheel is built wrong). Yes, they can be broken, but it takes a LOT more than it would for a comparable aluminum rim.
(For reference, the rims I've beat on so hard are Light Bicycle RM29C07 and I'm replacing the stock rims on the I9 wheels with AM930.)
You can watch Danny macAskill try to destroy bare carbon wheels (no tires) on concrete here: https://www.youtube.com/watch?v=VfjjiHGuHoc
I assume some of the horror stories come from people buying cheap carbon wheels or early models before they had layups and molds figured out.
If you're really concerned, add something like a cushcore to the inside of the tire for additional damping.
Your complaints are common, though. I've all but given on up trying to discuss carbon wheels with people in person because there are so many preconceived notions out there. It was the same way 5-10 years ago with everyone avoiding carbon handlebars for one reason or another.
Look for "Danny MacAskill Tests Santa Cruz Reserve Carbon Wheels": he really tries to destroy those rims, and it takes him doing ridiculous things to do so. Aluminum rims would have cracked/ bent much earlier.