The pneumonia-asthma example seems to be an example of a Simpson's paradox [1]. The doctors acted on a strong (accurate) belief about asthma sufferers contracting pneumonia and acted in such a way that the data obscured an actual causal link (asthma as an aggravating factor to pneumonia). This is opposed to the canonical Simpson's paradox where doctors acted on a strong (inaccurate) belief about severe kidney stones [1a] and again produced lopsided data that hid the best treatment option until the paradox was identified.

Humans have a very hard time uncovering so-called "lurking variables" [2] and identifying such paradoxes. I don't see how a neural network (or other machine learning tool) could do so on their own, but I don't know that much about machine learning. So, I guess I have two questions for the experts out there:

* If all training data is affected by a confounding variable, can a machine learning algorithm identify its existence, or is it limited by only knowing a tainted world?

* Once we have identified such lopsided data and understood its cause, how do you feed that back into your algorithm to correct for it?

---

[1] https://en.wikipedia.org/wiki/Simpson%27s_paradox

[1a] https://en.wikipedia.org/wiki/Simpson%27s_paradox#Kidney_sto...

[2] https://en.wikipedia.org/wiki/Confounding