News for hackers. Stuff that matters.
Re:New academic journal only publishes 'unsurprising' resear... by onurcel on Thursday August 20, 2020 @06:04PM (#24229568)
I like the idea but I don't really get why they accept "statistically insignificant" results. What I expect from a scientific paper is to prove (even emprically) something. For example, if a paper claims something like "we show that using method X instead of Y doesn't improve the results", it won't get published on most journals... except this one, which is awesome, but the paper still has to prove that claim.
Re:New academic journal only publishes 'unsurprising' resear... by bboy13 on Thursday August 20, 2020 @06:10PM (#24229611)
When you open the field to reward research itself rather than p-hacking, everyone benefits.

It's like a flea market. Bargain bin discounts, junk to many, valuable to some, but it shifts the economics of the whole system and lets specialty niche researchers not have to reinvent the wheel.

Also, knowing that a method or study was statistically insignificant is statistically significant for future study design.

Thanks!

Re:New academic journal only publishes 'unsurprising' resear... by acchow on Thursday August 20, 2020 @06:07PM (#24229589)
If the only goal is p 0.05 and there's many groups around the world trying the same experiment, we should hope that they all publish their non-results otherwise by chance someone is going to have a significant result just by luck.
Re:New academic journal only publishes 'unsurprising' resear... by progval on Thursday August 20, 2020 @06:16PM (#24229666)
Doesn't "statistically insignificant" mean papers which conclude something like "we find no correlation between X and Y"?
Re:New academic journal only publishes 'unsurprising' resear... by BenoitEssiambre on Thursday August 20, 2020 @07:14PM (#24230119)
No! It means the data is too noisy to make any conclusion.
Re:New academic journal only publishes 'unsurprising' resear... by ellis-bell on Thursday August 20, 2020 @08:23PM (#24230572)
not necessarily. it could. but if x and y are truly uncorrelated variables that are observed with unlimited precision, then you still would not reject the null that they are uncorelated
Re:New academic journal only publishes 'unsurprising' resear... by BenoitEssiambre on Friday August 21, 2020 @07:29AM (#24233952)
"Truly uncorrelated" is a problematic concept. Most of the time, any two things within each other's light cones, have at least tiny direct or indirect effects on each other. These effects are usually not that tiny in experimental settings because of imperfect experimental tools and procedures.
Re:New academic journal only publishes 'unsurprising' resear... by gus_massa on Friday August 21, 2020 @10:03AM (#24234966)
Like the speed of sound and the height of the table where the experiment is.

Note that the pressure of the air changes with height. For a few inches the change is negligible and very difficult (impossible?) to measure, but the effect must be out there.

I had once a problem with the temperature of the room, you usually ignore it, but it has a bigger effect like a 5% variation.

Re:New academic journal only publishes 'unsurprising' resear... by kelnos on Thursday August 20, 2020 @06:26PM (#24229750)
I think information like that is perhaps useful, at least to avoid wasting time repeating the same research, but publishing that does not prove that there's no correlation between X and Y, only that the things tested didn't show one.
Re:New academic journal only publishes 'unsurprising' resear... by dguest on Thursday August 20, 2020 @06:26PM (#24229742)
You can't ever prove that switching to X doesn't change the result. All you can really hope for is putting smaller limits on the amount by which something could be improved.

For one dumb example: if you wanted to see if rats can fly, you might observe 100 rats and see that none of them fly away. But this doesn't prove anything: in reality even if 3% of rats could fly you'd get this result about 5% of the time. You've really shown with 95% confidence is that less than 3% of rats fly.

Re:New academic journal only publishes 'unsurprising' resear... by walty on Thursday August 20, 2020 @06:13PM (#24229644)
If you only publish the positive results, you miss the context of the unsurprising results. For example, in https://xkcd.com/882/ , if the results from all the experiments in the comic were published, the public would know that the "Green Jellybeans cause acne" result is probably a fluke.
Re:New academic journal only publishes 'unsurprising' resear... by collyw on Friday August 21, 2020 @02:32PM (#24237569)
It should stop other groups wasting their time on the same research, no?