News for hackers. Stuff that matters.
Re:“The Book of Why” by Pearl and Mackenzie by nkurz on Friday January 11, 2019 @10:42AM (#18884244)
I have much more sympathy for Pearl. He's constructed "toy" problems that yield different answers depending on the hidden causal structure chosen. He then shows that unless one takes account of the causal structure, it's impossible to correctly answer the problem. The logical conclusion of this is that any technique that doesn't use the causal structure as an explicit assumption is instead using it as an hidden implicit assumption. His hope is that anyone who goes through his exercise will come to the same conclusion. Starting with real-world problems is tricky, because one doesn't know the correct answer in advance, thus it's much harder show the reliance on the assumptions.

Consider a parallel with computer programming. A user complains that they fear a program is giving them the wrong answer on a complex real world problem. They report this, and get back the unhelpful answer "Works for me, will not fix". Unable to shake the feeling that the answer is unreliable, they reduce the problem down to a proof of concept that serves as a simple self-contained test case. Two different inputs produce the same answer, but only one of them can be right! But now they are unable to convince the maintainer to even look at the test case, because now the maintainer says "I need to focus on the real world, and don't want to waste my time on toy examples".

It's a discouraging position to find oneself in.

Re:“The Book of Why” by Pearl and Mackenzie by bachmeier on Friday January 11, 2019 @01:26PM (#18885602)
Oh, I'm completely on board with being explicit about any assumptions you're making, and for thinking deeply about the causal relations, and for thinking about the reliability of the analysis. I've been critical of economists for not thinking things through. And I'm definitely on board with teaching using toy problems.

The problem is that, as far as I can tell, Pearl doesn't go beyond making points with toy problems. He hasn't done much empirical work (has he published a single empirical paper?) or even read much of the empirical literature he's criticizing as worthless. Ultimately, the question is whether policy and other decisions will be better using a particular framework. The fact that Pearl writes with the aggressive confidence of a Hacker News commenter does not mean he's right.

Re:“The Book of Why” by Pearl and Mackenzie by harperlee on Friday January 11, 2019 @02:42PM (#18886220)
But from a mathematical point of view this insistence of focusing on empirical work is a little bit disheartening isn’t it?

And frankly Judea Pearl is hardly a random internet tinfoil hat guy, nor is he asking people to invest a massive effort checking his looong work (as we eg often see with random NP-completeness “proofs”, or was discussed with the time investment needed to check Mochizuki’s ABC proof - no, he just asks Gellman to apply his own familiar techniques to a toy problem. It does not sound unreasonable at all.

Re:“The Book of Why” by Pearl and Mackenzie by bachmeier on Friday January 11, 2019 @03:09PM (#18886461)
> But from a mathematical point of view this insistence of focusing on empirical work is a little bit disheartening isn’t it?

Why? The only thing that matters is the quality of the empirical work. You can get caught up in philosophical debates about the best way to do research. If it has no impact on empirical work, it's useless.

That's not to say Pearl's arguments are wrong or that his work is useless. The problem is the incompleteness of his arguments. You can't arrogantly dismiss empirical work just because it's imperfect. There's no reason a priori to expect that Pearl's approach will lead to better decisions.

There are many self-proclaimed experts who can point out the flaws in programming language designs, but that doesn't mean they can design a better language, and it doesn't mean existing programming languages are useless. Pearl's approach is not some kind of magic pixie dust that suddenly guarantees your empirical work is more trustworthy. It's unfortunate that Pearl thinks it is, and it prevents him from having a reasonable conversation about the topic.

Re:“The Book of Why” by Pearl and Mackenzie by harperlee on Friday January 11, 2019 @03:40PM (#18886747)
Well from my point of view when I read the link, I see (with a little bit of characterization) two academics, one of them points at an issue on the standard statistical approach and tools, provides a reproduceable way to showcase that answer, and the other one answers “not interesting - i’ll go now to do real work”. And I found that avoidance of a discussion of a shared example disheartening.

And Pearl isnt just complaining - the provided an alternative. So it seems that one side identified a problem, curated examples, delivered a solution, and now the other side doesnt care about the problem - and causality is not a trivial problem. As academic discussion, not a business meeting, to me thats subpar.

That’s only my view from the outside, it may well be the case that it is wrong and I just need to review more in depth Pearl to see that he does not bring anything useful.

Re:“The Book of Why” by Pearl and Mackenzie by bachmeier on Friday January 11, 2019 @05:31PM (#18887823)
They've been debating this for many years, and Gelman is a coauthor and student of Rubin, who has been debating with Pearl for decades. Gelman is saying he doesn't want to debate the same issues yet again.

The primary message of Pearl is "all other methods are trash and all empirical work done using those methods is trash." My interpretation of the post is that Gelman is arguing other methods are not trash.

I probably agree more with Pearl than Gelman on the details, but Pearl's approach is just not appropriate for an academic setting.

Re:“The Book of Why” by Pearl and Mackenzie by evrydayhustling on Friday January 11, 2019 @06:40PM (#18888287)
What they are really fighting about is who gets to reference colloquial notions of causality when discussing their work. I don't think it's fair to say that Pearl treats the rest of stats as trash, but he is saying that they are misleading folks by describing their inference as evidence of cause and effect - which is about as bad for most academics and especially for the Rubin crew who have often policed integrity of other inference regimes (like less rigorous ML).

Debating who gets to use which words is a great way to make sure your debate only matters to other academics. I'd love to see the causality camp make their point by unlocking some great new applied results instead!

Re:“The Book of Why” by Pearl and Mackenzie by srean on Saturday January 12, 2019 @02:50AM (#18890232)
Refusing to provide a demonstration of the statistical technique on a toy example constructed for pedagogic purpose, hardly constitutes as a "debate". I would file that under: using a lot of words to avoid answering a direct and fundamental question
Re:“The Book of Why” by Pearl and Mackenzie by rbkillea on Friday January 11, 2019 @07:16PM (#18888500)
http://causality.cs.ucla.edu/blog/index.php/2016/02/01/works...

Is this sufficiently applied for you? Pearl may not apply his work on causality often, but there are a lot of people who are on the applied side who do.