Your example isn't great because newtonian physics is still correct and used everywhere.

I agree with your point, reality is certainly not statistical, and e.g. there would have been circles in which heliocentricity or a flat earth or some pre-germ theory of disease would have been agreed upon fact. Arguing that those aren't science is hindsight, at the time they would have been learned consensus.