News for hackers. Stuff that matters.
Re:Statisticians want to abandon science’s standard measure ... by btilly on Friday September 06, 2019 @12:21PM (#20896989)
I disbelieve.

The problem is that people want an answer to the question, "Here is a pile of data, what should I believe?" But mathematically, the proper role of data is to modify existing beliefs, and not to dictate beliefs.

Statisticians can spend forever explaining this. But instead have gone with the cop-out of asking a question that is confusingly similar to the one that people want to ask. It is popular exactly because it is so easily misunderstood. You can give any number of lectures on what it actually means - I guarantee that it will be misunderstood.

Worse yet, p-values are sensitive to particulars of experimental design that logically should never matter to inferences. My aunt and uncle provide a classic example. They wanted a son and a daughter. They had 6 sons then a daughter. Are they biased towards one gender? The p-value for this question is .03125 (we'd see this much evidence against with all boys, all girls, last girl, last boy, so 4/2^7). But if I instead said that they planned to have 7 children, and had 6 sons then a daughter, the p-value changes to 0.125 because the one child out could have been anywhere in succession. But in Bayes' formula, their plans if something else had happened cannot ever affect how we adjust our inferences.

(This example is not made up. When Lorna found that they had a daughter, she told Bill to get a vasectomy from the delivery table. By another crazy coincidence, their 7 children were also born on all the days of the week.)

Re:Statisticians want to abandon science’s standard measure ... by ghkbrew on Friday September 06, 2019 @12:55PM (#20897338)
> But in Bayes' formula, their plans if something else had happened cannot ever affect how we adjust our inferences

That's not how statistics works. Frequentist p-values and Bayesian inferences are both entirely dependent on the mathematical model in question. "Their plans if something else had happened" are a key part of the model. What you're describing at 2 entirely different experiments:

1) A couple has children until they have 1 of each gender. they have 7 children. How likely is a result this extreme (or greater)?

2) A couple has 7 children. They have a gender ratio of 6:1. How likely is a result this extreme (or greater)?

Re:Statisticians want to abandon science’s standard measure ... by btilly on Friday September 06, 2019 @03:10PM (#20898808)
No, that is EXACTLY how probability theory works.

The probability of event A given that we observed B is the probability of A and B happening divided by the probability that B happened. Mighta, coulda, shoulda but didn't doesn't enter into it and can't affect the result. The fact that frequentist statistics does care and shouldn't is one of the major criticisms that Bayesians offer.

If you think you understand statistics and don't understand this fact, then you do not understand statistics as well as you think you do. But you can be pardoned. Most statistics classes are too busy cramming statistical tests into student's heads to bother them with bothersome facts about where the cracks in the foundations are.

Re:Statisticians want to abandon science’s standard measure ... by RosanaAnaDana on Friday September 06, 2019 @12:33PM (#20897109)
Well, sort-of.

Its not just about p-values or the likelyhood of making a particular type of error by agreeing with a thesis.

There is a fundamental misunderstanding that most people, and even many scientists make with regards to the philosophical interpretation of 'truth' using the scientific method.

By definition, scientific 'truth' is always fallible, and the vast majority of people have a very hard time dealing with this concept. A better explanation or additional data, or better data is always possible and needs to be possible for science to not be dogma. This is fundamentally different than how people interpret truth and understand what the word means. All scientific truths are replaceable.

Re:Statisticians want to abandon science’s standard measure ... by srpguy on Friday September 06, 2019 @01:07PM (#20897454)
"You can give any number of lectures on what it actually means - I guarantee that it will be misunderstood."

So true. That said, here's a great interactive explaining the p-value: https://www.jwilber.me/permutationtest/