News for hackers. Stuff that matters.
Re:Bayesians Moving from Defense to Offense by naasking on Saturday December 23, 2023 @06:30PM (#38749356)
> This advantage can't be understated. Researchers, at least in psychology, almost never seem to care about the magnitude of an effect. It's simply enough to show that some effect happens in some direction

That's not an advantage, that's one of the primary reasons psychology is among the worst subjects hit by the replication crisis.

Re:Bayesians Moving from Defense to Offense by SubiculumCode on Sunday December 24, 2023 @01:10AM (#38751538)
A cheap shot that misunderstands the so called "crisis" in psychology, which almost comes entirely from the fact that brains are non randomly exchangeable, unique, samples differ, the brain is complex, multiple ways to solve problems, and sussing it all out is complex as hell.
Re:Bayesians Moving from Defense to Offense by naasking on Sunday December 24, 2023 @01:38AM (#38751640)
If they are at least as knowledgeable as you, who correctly describes how messy you can expect the data to be, then you should be more concerned with effect size, because small effects are almost certainly just "noise". Instead, sensationalism about miniscule effects that vanish on replication attempts abound.

So not a cheap shot at all IMO.

Re:Bayesians Moving from Defense to Offense by bogtog on Saturday December 30, 2023 @01:50PM (#38817722)
Sorry for this very slow response, but I actually disagree. I don’t think effect sizes are usually meaningful beyond the context of a specific experiment.

For example, suppose you are researching the effect of an emotional/negative picture on how a participant makes decision in an economic game. Here, you may think “a big effect size implies this study will have a large practical relevance.” However, following typical statistical designs, an effect size may be largely determined by just how many trials the experiment used for each participant. This is also not the only factor that’s relevant. For instance, it’s debatable how the emotion induced in the task compares to the intensity of real life emotional situations. With these factors in mind, the effect size really doesn’t tell you much that can be applied outside the laboratory.

Maybe in some non-experimental sub-fields (e.g. personality psychology) effect sizes are more meaningful…