Re:Mental Models I Find Repeatedly Useful
by nhaliday on Tuesday July 05, 2016 @11:42PM
(#12041067)
I'm not sure about expanding on Bayes' theorem, but some other notions from ML/stats that would be good to know are overfitting/the bias-variance tradeoff and base rates.
One instance where I've seen the former applied to society is the idea of research benchmarks getting stale from "overfitting". Even when researchers do cross-validation, we might still expect our exploration of the space of ML models to be skewed towards models that perform unusually well on well-known benchmarks. This was described in http://www.deeplearningbook.org/ with reference to ImageNet (of course).
As for the latter, pretty much every time I've seen a discussion of statistics on social or old media, 90% of the participants seem unaware that base rates matter.
One instance where I've seen the former applied to society is the idea of research benchmarks getting stale from "overfitting". Even when researchers do cross-validation, we might still expect our exploration of the space of ML models to be skewed towards models that perform unusually well on well-known benchmarks. This was described in http://www.deeplearningbook.org/ with reference to ImageNet (of course).
As for the latter, pretty much every time I've seen a discussion of statistics on social or old media, 90% of the participants seem unaware that base rates matter.