You're confusing rationally concluding something with feeling morally righteous for believing it. It's not irrational to disbelieve something that people only believe because they'll feel like a bad person or be punished for doubting. See religion, for example.
The only reason most people believe the holocaust happened is because they heard about it from general society. Same way they believe God created the world. Almost no layman has actually studied it. It's just a kind of common faith where being a believer is what's important rather than the content of the belief.
I'm not saying it didn't happen, just that the vast majority of believers aren't believing out of rationality but out of indoctrination.
Same is true of all sorts of beliefs in things that don't directly affect us. We believe them because everyone assures us they're true, not because we sat down and worked out the conclusion for ourselves.
This is literally schizophrenia, when two such communities meet, a massacre inevitably occurs.
Now, I notice that I use "we all agree" and "vast majority," which is no way to explain an objective fact, but what we all agree on as a community or society _is_ reality. A society or community that has a different reality(s) than us is probably not a society or community that we would associate ourselves with.
This operates on several levels and dimensions; the common realities I share with my local Islamic community are different realities than I share with my tech community, or Toastmasters community, or even family.
Going back to the original point, yes, there are no realities that the entire global community can agree on, not even something as seemingly incontroversial as medicine (Christian Science for example https://rpl.hds.harvard.edu/religion-context/case-studies/mi... ), but no individual is part of the global community. We choose our communities based on the realities that we accept to be true.