Can't we build our own propensity for imperfect decision making into a calculation of safety? Can I not realize that while I consider myself more rational and cautious than my neighbors, they likely feel the same, and X% of them die when they try to fly?

I would consider this "second derivative caution". It got cemented into me when I was taking hang gliding lessons and saw one of the most stone cold cautious members of my class become exuberant and act rashly on a landing, and break his arm quite badly.