News for hackers. Stuff that matters.
Re:LSD microdosing: a randomized, blind self-experiment by javajosh on Thursday October 17, 2013 @02:14PM (#6567135)
Given the fact that he had a null result, I don't think this criticism could possibly be valid.
Re:LSD microdosing: a randomized, blind self-experiment by Someone on Thursday October 17, 2013 @05:24PM (#6568226)
Most importantly, the criticism of a _design_ is independent of the results of the experiment.

Secondly, he may have wanted a null result.

Re:LSD microdosing: a randomized, blind self-experiment by gwern on Thursday October 17, 2013 @11:31PM (#6569623)
> Most importantly, the criticism of a _design_ is independent of the results of the experiment.

Flaws can go in any direction, and make a design conservative or optimistic. In this case, the identified flaw does not undermine the actual result of the experiment - though it would have undermined the opposite (hypothetical) result of positive effects.

Re:LSD microdosing: a randomized, blind self-experiment by Someone on Friday October 18, 2013 @07:01PM (#6574526)
If you design an experiment in such a way that outcome A can be explained away by "yes, but the design is flawed", while outcome B cannot, I don't think it is fair to do the experiment.
Re:LSD microdosing: a randomized, blind self-experiment by gwern on Friday October 18, 2013 @08:25PM (#6574793)
Experiments don't have to be 'fair'; your 'fairness' is not required for tests to be informative. Imagine an IQ test which spits out a binary yes/no result based on whether you're in the top 10% or not. This is a useful, meaningful test, which is 'unfair' in your naive sense. Issues of information content, discrimination, false positives and negatives - it's more complex than saying 'it is fair to do the experiment'. In this case, I got a reliable negative, rather than a less reliable positive.