> Due to the sheer number of hypotheses get tested all the time in particle physics, 3 sigma effects appear all the time, and then disappear before getting a larger certainty.

See, eg, the look-elsewhere effect [1] or the problem of multiple comparisons [2].

---

[1] https://en.wikipedia.org/wiki/Look-elsewhere_effect

[2] https://en.wikipedia.org/wiki/Multiple_comparisons_problem