For example, you could have an extremely unlikely hypothesis (eg. "dice are controlled by alien telepathy"), test it, and still come out with p < 0.05 because of something much more plausible ("the dice were badly made and are biased"). This is known as the base rate fallacy; an extraordinary claim requires extraordinary proof. Moreover, to prove an extraordinary claim, you must show it is more likely not just compared to chance, but compared to every less extraordinary alternative, including "the experimenters are faking the data".
It's also entirely possible to get a result at p < 0.05 that makes the hypothesis you're testing less likely. Suppose you want to know how far away the nearest star is. The value in the textbook is 13.4 light-years, and you think it's really 12,000. You take some measurements, and get values of 19.6, 17.4, 20.1, 20.4 and 18.5.
Now, this is a significant result at p < 0.05 - if the star is 13.4 light-years away, then getting these numbers has a probability of less than 5%. However, these results completely rule out the hypothesis you're testing. The numbers you get are pretty unlikely if the real number is 13.4, but extraordinarily unlikely if the real number is 12,000, so this experiment makes the 13.4 number more credible. This kind of thing is why lots of researchers still believe in psychic powers - they keep testing for psychic powers, and keep getting results at p < 0.05, but don't notice their results make psychic powers less plausible. (Not joking - see http://commonsenseatheism.com/wp-content/uploads/2010/11/Wag... for a detailed explanation.)
This is an elementary mistake that every freshman statistics course warns against, and yet it's absolutely pervasive.
The hypothesis in question is the null hypothesis which must be assumed true not false for the p-value to mean a 100*p% chance of seeing the observed data.
I can see why elementary mistakes are so pervasive.
It's got priiiiooooooors.
Reproducibility by researchers who know no more about the experimental setup than what was published is a better way of guarding against this than upping the significance hurdle. Experienced researchers may have a long history of ignoring confounding factors - I am not sure maturity matters much. I think the best time to have your complacency about constructing statistical investogations torn apart is while you are doing your PhD.
The thing I fear is underexperienced multidisciplinary researchers coming up with convoluted procedures that put their hands and eyes on their work in so many places that small sources of error have compounded themselves and entrenched themselves into nooks and crannies.
Very few degree programs give scientists enough background in statistics and experimental rigor to prevent these problems.
Perhaps that's a little bit of projection, but I don't use any but the most rudimentary statistics in my publications and I don't make claims about significance.
I think your comment about them being less likely to be able to handle depth from a personality/self-habits point of view is really projecting and probably based on experiences with a few bad apples. My experience doesn't reflect that at all.
I feel like a lot of the problems with statistics come from overly insular journals and conferences where everyone does it wrong and nobody realizes that because they're all doing the same thing. I've seen this in some computer science fields.
People seriously just take standard deviations and then plug it into the formula to spit out p values. Without thinking about things like: "should the values be normally distributed? or log-normal?" "How does error propagate through this formula I'm using?" "Is the major source of error in the replicates that I'm using (versus something I might be normalizing to, like a mass measurement)?" "Am I in the linear range of my calibration standards?" "Is using these data quantitatively (versus qualitatively) an honest thing to do?"
This screams systematic error and error propagation to me. It's possible that we don't need to up the p-value, we just need to make sure researchers aren't stupid and can properly account for all sources of errors. The problem is that's often an acquired skill over time, not something younger researchers typically think about, especially those who aren't multidisciplinary.