In corporate finance something like a ROI is seen as telling the future. I can’t tell you how common it is to revisit something on an “let’s understand why the ROI is off” basis. And them not understand when I point out not a single assumption held true. More often than not aggressive assumptions were used to validate the strategy and gain approvals and it was always going to fall short. In my role, CYA is a huge deal.
> like reporting confidence intervals
Then there is the problem of how insanely innumerate society at large is to the point where something like this doesn't register for huge swaths of people. might work in the context of engineers or people with some higher education but not all contexts.
Re:Boring Problems Need Attention Too
by 6gvONxR4sf7o on Thursday June 17, 2021 @02:42PM
(#27542984)
I don't know, it feels like many people understand ranges (but still not all). They don't need to understand what a CI is to get "somewhere between 1 and 3." The barrier I face is that while they understand ranges, they can't take uncertainty.
It's amazingly hard to communicate "variability" in an answer. Whenever I communicate experimental results, my business partners always want to hone in on the simplest explanation (e.g. "treatment effect somewhere between 1 and 3" becomes "treatment effect = 2"). And that's just the uncertainty part of variability. Then there's the kinds of variability you're talking about.
There are a ton of tricks to mitigating this, like reporting confidence intervals instead of point estimates, or giving the right number of digits to communicate the key message, or making super sharable/linkable results to minimize the game of telephone. I'd love someone to write a book on how to communicate this sort of "variability" in a business setting.