Re:MIT Study Finds AI Use Reprograms the Brain, Leading to C...
by sarchertech on Wednesday September 03, 2025 @11:01PM
(#45123036)
With a greater than 99% probability that this is a real effect, i wouldn’t expect this to be difficult to reproduce.
But it turns out I misread the paper. It was actually an 80% effect size so greater than 99.9% chance of being a real effect.
Of course it could be the case that there is something different about young college students that makes them react very; very differently to LLM usage, but I wouldn’t bet on it.
0.05: 11 people per cohort
0.01: 16 people per cohort
0.001: 48 people per cohort
So they do clear the effect size bar for that particular finding at the 99% level, though not quite the 99.9% level. Further, selection effects matter -- are there any school-cohort effects? Is there a student bias (i.e. would a working person at the same age, or someone from a difficult culture or background see the same effect?). Was the control and test truly random? etc. -- all of which would need a larger N to overcome.
So for students from the handful of colleges they surveyed, they identified the effect, but again, it's not bulletproof yet.