News for hackers. Stuff that matters.
Re:Understanding Causality Is the Next Challenge for Machine... by wenc on Thursday October 29, 2020 @02:28PM (#24933685)
(First-order) causality actually isn't hard to determine in environments where first-order effects dominate. The way to determine causality here is through a combination of physical laws and controlled experimentation [1]. In fact, we have plenty of causal models (e.g. 1st principles physics-based models, or design-of-experiments models). Without these models, machines/control systems/etc would not work.

The trouble is, outside of these 1st-order effect dominant, deterministic environments, causality becomes much harder. In complex systems, stochasticity, nonlinearity, feedback loops and higher-order effects dominate. There's also emergent behavior -- properties that are true in the small are not true in the large.

Consider a complex system like human society -- can we truly determine causality of broad interventions? Likely not in a first-order way like in the physical sciences. We can do it imperfectly through tools like causal inference (Rubin) which makes much more modest claims about the "strength" of effects (average causal effect). Randomized Controlled Tests (RCT) is another tool for making causal claims.

But in a complex world, 2nd, 3rd and higher order effects dominate and so the notion of root causes itself becomes ambiguous. Richard I. Cook once said "post-accident attribution to a 'root cause' is fundamentally wrong". Though humans are attracted to the idea of a chain of simple causes (which is why we have the myth of Mrs O'Leary's cow kick over a lantern and starting the Great Chicago Fire of 1871), there's typically no easily-identified root cause. First-order causal thinking assumes a Directed-Acyclic-Graph (DAG) idea of a causality chain which converge into a set of effects, but the reality is that such a DAG, if it can be represented, is likely to be infinitely complex in a complex environment. First-order causal thinking is an insufficient mental model in a complex environments.

Instead, I think instead of aiming for a deep understanding of epistemic causality (where we try to know and represent causality), it's probably more useful to focus on instrumental causality (where we aim to know the main points of leverage that are effective in changing the system). I think we'll likely get very far just by finding the knobs that have the most effect on the variables we would like to change (that don't also simultaneously change variables that we wouldn't want to change).

[1] to determine causality, we typically have to perturb the system -- determining causality through observational data is possible, e.g. via natural experiments, but there are many epistemic restrictions which limit the claims that can be made.

Re:Understanding Causality Is the Next Challenge for Machine... by lambdatronics on Saturday October 31, 2020 @03:22AM (#24949678)
>complex systems, stochasticity, nonlinearity, feedback loops and higher-order effects ... emergent behavior

Plasma physics & fusion energy (my field) is challenging for exactly these reasons (despite being 95% classical physics). It's very rare that we can do a nice controlled experiment where only a single variable is changed at a time. I joke that it's really a subset of biology, not physics.

> probably more useful to focus on instrumental causality

I partly agree -- we humans do seem to get by on rudimentary reasoning. On the other hand, the issue of back-progagation is quite similar to identifying 'root causes.' There's also an issue of combinatorial explosion of the number of possible sets of variables that interact with each other, coupled with the fact that data becomes exponentially sparse as the dimension of the space grows. The human ability to detect causal relations is really stunning when you realize how tough it is -- I wouldn't want to bet that we can reproduce it by trial and error. Evolution had plenty of time to get it right, but we don't.

Re:Understanding Causality Is the Next Challenge for Machine... by wenc on Saturday October 31, 2020 @04:57PM (#24955134)
Very much so. If we could quantitatively characterize every little effect, we might be able to model complex systems accurately, but it isn't always possible (exceptions exist, like meteorological modeling). For the longest time in biology, the modeling approach has tended to be more qualitative than quantitative, though this is changing.

Yes, identifying root-causes from the outcomes is a model inversion problem (here's the data, find the generating model), and model inversion problems tend to be ill-conditioned. In complex systems, there's so many possible combinations of causes that could have led to a single outcome that characterizing the entire set is difficult.

Re:Understanding Causality Is the Next Challenge for Machine... by 6gvONxR4sf7o on Thursday October 29, 2020 @02:52PM (#24934086)
Where is a conditional average treatment effect insufficient? We want to say that, seeing X right now, if we do a1, we'll see a change of d (+/- D), compared to doing a0. Or being able to predict y1 and y0 corresponding to actions a1 and a0. That would be a huge step, and something some of ML's most useful methods can't do.
Re:Understanding Causality Is the Next Challenge for Machine... by zwaps on Thursday October 29, 2020 @05:24PM (#24936094)
DAGs are not the only valid approach to causality, nor do they subsume other approaches (I have previously commented on this during the Gelman&Pearl debates).

As you suggest, the treatment effect and econometrics literature currently is on a semi-parametric trend: Given that they don't believe that one can actually produce a believable complete causal model (or DAG), one tries to estimate a treatment effect that does not depend on parametric or functional assumptions.

Re:Understanding Causality Is the Next Challenge for Machine... by a9h74j on Friday October 30, 2020 @12:06AM (#24939044)
> DAGs are not the only valid approach to causality

It has been a truism to some that "most efforts in policy are responses to previous efforts in policy." In SPC you can measure noise and overcontrol. With a long-lived policy-making institution [unusual?] honesty might admit to facing effects of previous well-intentioned policies.

Re:Understanding Causality Is the Next Challenge for Machine... by jostmey on Thursday October 29, 2020 @02:35PM (#24933815)
> to determine causality, we typically have to perturb the system

Couldn't agree more! For computer models to understand causality, it must be able to interact with the environment and probe it. I think understanding causality is one and the same as reinforcement learning, where a computer model learns to interact with its environment

Re:Understanding Causality Is the Next Challenge for Machine... by zwaps on Thursday October 29, 2020 @05:20PM (#24936062)
The majority of the books listed in the first link in these comments are about whether and when we can determine causality given observational data - data where the system is perturbed, just not by us (and not as we would ideally like).
Re:Understanding Causality Is the Next Challenge for Machine... by a9h74j on Friday October 30, 2020 @12:13AM (#24939086)
I have not looked at the links, but aren't there people who use Bode plots to estimate direction-of-causality, given time-series data? IIRC there are basic relationships between wide-frequency phase behavior and, e.g., impulse-response functions. There is apparent phase information beyond correlation and ANOVA.
Re:Understanding Causality Is the Next Challenge for Machine... by jbay808 on Thursday October 29, 2020 @02:43PM (#24933955)
But many datasets will be full of recorded perturbations already. Can't you usually hunt for the evidence you need in the data you have?

The Michelson-Morley experiment was enough data to get special relativity, for example.

Re:Understanding Causality Is the Next Challenge for Machine... by goatlover on Thursday October 29, 2020 @03:03PM (#24934242)
Einstein was inspired to come up with special relativity from that experiment because he had the relevant concepts to draw upon to think through various thought experiments. That and the math to back up his intuition.
Re:Understanding Causality Is the Next Challenge for Machine... by jostmey on Thursday October 29, 2020 @04:05PM (#24935068)
Yes, but Einstein was able to learn cause and effect by interacting with the physical world as a child. Without being able to interact with an environment, the I believe the ability to learn causality is limited if not impossible
Re:Understanding Causality Is the Next Challenge for Machine... by 6gvONxR4sf7o on Thursday October 29, 2020 @03:10PM (#24934339)
Alternatively, we need to extend our typical notion of a dataset to include where it came from/how it was perturbed. If you give your best statistician a dataset without column names and a description of the experiment, they aren't going to be able to do causal inference. We need to make those things more available to the machines too, in a structured way.