No, that's not what P=0.05 means. This misconception is exactly why p values are so problematic. The question we want to answer is unanswerable, so instead we subtly substitute a different question in its place and don't even realize we've swapped concepts.
There is absolutely nothing valid about your "flip" (called transposing the conditional) and the damage done to the human species by people doing the "flip" is too huge for most to comprehend. Sorry, but I really cannot overstate the problems with this "flip".
Perhaps people who get paid by grants should get three chances regarding this flip. If they strike out they are banned from ever being funded by NIH, etc for life. The result (either 99+% of current researchers would be culled, or p-values would become appropriately rare) would do more to advance science than any other idea I have seen.
Sure there are debates about priors. These are unavoidable. The hidden assumption in my example is a 50/50 prior on too noisy, not too noisy hypotheses. IMO a reasonable uniform (maximum entropy) prior to start a discussion from. Of course, one should be more precise when doing this type of analysis in a paper. The reason a good prior is hard to define here is not because you shouldn't do the flip, it is because the question that the t-test is trying to answer doesn't make much sense.
How can saying false things make your science "about the the real world"? The only thing I can think of is that by "real world" you are referring to using the publication of massive amounts of incorrect/questionable information as a metric for success. Yes, that is indeed the sham going on.
>"There is a reason humans naturally and instinctively do it when they talk about experiments. This is how you describe things in real world human language."
I have no problem talking about data without transposing the conditional, if you do that is some issue with your training.
>"Sure there are debates about priors."
The error you advocate (of transposing the conditional) has nothing to do with priors.
Also P(A|B) = P(B|A) if the prior probabilities are equal, often a reasonable prior.
Ok, but I fail to see the relevance of this to anything.
>"Also P(A|B) = P(B|A) if the prior probabilities are equal, often a reasonable prior."
Here, A = Hypothesis and B = data (or vice versa). You are claiming that the "probability of seeing your data whether or not the hypothesis is correct", is often near the "probability the hypothesis is correct independent of your data"?
Based on what? What principle leads you to think these two probabilities should be near each other? BTW, if you actually have one this is like the holy grail of statistics.
Shouldn't that be "the probability of seeing the data, _given_ that the hypothesis is correct" and "the probability of the hypothesis being correct, _given_ that the data has been observed"?
The claim is that it is reasonable to assume P(A) ~ P(B), so that P(A|B) ~ P(B|A). I am referring to P(A) and P(B) in the equation.
I agree there are debates surrounding priors, especially when you get into higher order, meta or purely hypothetical territory (there is no actual noise generator here) like this, but I think these are unavoidable.
For this to be true, P(observations) ~ P(pure noise generator) as you said earlier. Anyway, you can look at it from either perspective, doesn't really matter.
On what basis do you claim this is reasonable to expect? I suspect you have none whatsoever... in fact simple thought experiments will lead us to the opposite conclusion, and there is plenty of data telling us the same.
This is incorrect. A p-value is the probability of observing a value of the test statistic that is at least as extreme as the reported value, assuming the null hypothesis is true.
Now, the exact values of a given data set are vanishingly unlikely, so we actually ask about the probability we'd say this data or even more extreme data given that we're just looking at noise. That's a typical p value.
The whole terminology with values being "extreme" is just unintuitive and unwieldy, IMO.
It's much closer to, this result rises above the noise floor. Suggesting but not implying a possible link between X and Y, fell free to investigate.
P=0.05 is best interpreted as "There is a 95% chance that this data isn't pure noise".
Then researchers should always put more emphasis on confidence intervals. Readers of papers can then see if the size is enough to make the results relevant and not likely caused by experimental accident. Plus given the perverse incentives researchers are subject to, maybe assume that the real effect size is probably closer to the lower bound of the interval.
Null hypotheses have limited usefulness even at p=0.005. A tiny systematic bias in the experiment can make it cross that threshold and there are _always_ at least small biases. These can be caused by not exactly calibrated instruments or small differences in the way tests are performed by different researchers on the team etc. Inherent in null hypothesis testing is this ridiculous assumption that there are no systematic biases. This is its fatal flaw.
The null hypothesis test never tells you that the effect is zero or that it is any other value, it only gives you an hint about whether or not the data is too noisy to say anything at all.