The landscape is different today, which attracts a different sort of character. Back when scientific pursuit was a passion, you mostly had the creative and innately curious types who were drawn to the field all working on the same problems.
Now you have a large swathe of careerists with a handful of curious types, which invariably means the curious types get drowned out in the discourse and ultimately chased away from scientific pursuit.
The incentives are different as well. Peer review and showcasing ideas/work for grant money are new things. All of your work has to go through a committee of your non-curious careerist B-tier colleagues. If they don't like what you're doing, you won't get published and you won't get a grant.
It's now become a Job with the objective of getting all your work approved by committees. So it's different today.
As a doctoral student I had a paper rejected because I didn't toe the dogmatic line of one prestigious reviewer (seriously, the review feedback lacked only the word "dogma"). Terrible, right? But this is a common theme over centuries as near as I can tell, and here we are, living in a magnificent future!
We will continue to benefit from the scientific process, despite all our individual flaws, and despite how easily we each fool ourselves (and sometimes others).