More generally, scientists are incentivised to find novel findings (i.e. unexpectedly low p-values) or lose their job.
Given that, the plot doesn't surprise me at all (Also, people will normally not report a bunch of non-significant results, which is a similar but unrelated problem).
Are you saying that in other disciplines, the distribution of p-values in published papers does not follow this pattern?
Publishing introduces a systematic bias, because it's difficult to get published where p>0.05 (or whatever the disciplinary standard is).
That explains why the p-values above 0.05 are rare compared to values below 0.05. But it fails to explain why p-values above 0.02 are rare compared to values below 0.02.
Is that enough to tip the curve the other way across the range of p-values? Well, something is, and I am open to alternative suggestions.
One other point: while the datum immediately below 0.05 would normally be considered an outlier, the fact that it is next to a discontinuity (actual or perceived) renders that call less clear. Personally, I suspect it is not an accidental outlier, but given that it does not produce much distortion in the overall trend, I am less inclined to see the 0.05 threshold (actual or perceived) as a problem than I did before I saw this chart.
Don't be fooled by the line someone drew on the chart. There's no particular reason to view this as a smooth nonlinear relationship except that somebody clearly wanted you to do that when they prepared the chart.
I could describe the same data, with different graphical aids, as:
- uniform distribution ("75 papers") between an eyeballed p < .02 and p < .05
- large spike ("95 papers") at exactly p = 0.4999
- sharp decline between p < .05 and p < .06
- uniform distribution ("19 papers") from p < .06 to p < .10
- bizarre, elevated sawtooth distribution between p < .01 and p < .02
And if I describe it that way, the spike at .05 is having exactly the effect you'd expect, drawing papers away from their rightful place somewhere above .05. If the p-value chart were a histogram like all the others instead of a scatterplot with a misleading visual aid, it would look pretty similar to the other charts.
I think we are both, in our own ways, making the point that there is more going on here than the spike just below 0.05 - namely, the regular pattern that you identified in your original post. If we differ, it seems to be because I think it is explicable.
WRT p-values of 0.05: I almost, but did not, say that if you curve-fitted above and below 0.05 independently, there would be a gap between the two, and maybe even if you left out the value immediately below 0.05. No doubt that would also happen for other values, but I am guessing that this gap would peak at 0.05. If I have time in the near future, I may try it. If you do, and find that I am wrong, I will be happy to recant.
It looks to me like the y-axis is measured in number of papers. The lower a p-value is, the more papers there are that happened to find a result beating the p-value.
So low p-values are more likely to occur a priori than high p-values are. This is most certainly not true in general. We might guess that psychologists are fudging their p-values somehow, or that journals are much, much, much, much, much, much, much more likely to publish "chewing a stalk of grass makes you walk slower, p < 0.013" than they are to publish "chewing a stalk of grass makes you walk slower, p < 0.04".
I've emphasized the level of bias the journals would need to be showing -- over fine distinctions in a value that is most often treated as a binary yes or no -- because it is much easier to get p < 0.04 than it is to get p < 0.013.