sungx105
But this simplified explanation misses out on one key aspect of Monte Carlo: sometimes different kinds of Monte Carlo moves can be designed that can allow it to more efficiently sample the phase space than other methods such as gradient descent.
Unfortunately, doing so is can be very involved, and is not always very general, so it isn't as easy to do as using other methods for exploring phase space.
You go to grad school and spend ~6 years working extremely long hours at nearly minimum wage with no benefits. You then spend 2-3 years on a postdoc still with relatively low compensation. There still are far too few academic positions available. You now either become an adjunct (and get caught in that vicious cycle) or you do another postdoc. Repeat ad nauseum (or until you give up and move to industry anyway)
While the system is highly flawed now, I would argue that collaborating to build perfect experiments carried out by multiple labs would not be a good idea. You point out if people made the same sorts of choices then perhaps everyone just has the same biases and errors. But if all the people on one experiment collaborate, then they'll definitely all have the same biases. Instead, independent verification in which the nitty-gritty details are abstracted away for a more general methods section would allow for more alternate takes at approaching a problem.
The authors actually do point out that their sampling method collects data in a way that is independent of the hypothesis they test, so it is not an example of cherry-picking examples that support their ideas.
While I cannot comment on how valid the model is because I'm definitely not an expert in that area, it seems pretty solid; they gave neighboring areas the capability to develop military techniques of certain strengths, the capability to lose it, and saw where civilizations tended to develop the different areas would "fight it out" and transfer military technique back and forth, and the result of their simulation appeared to be quite similar to the map of that time period.
There was also some talk in the comments about overfitting, and while as a person who works in chemical simulation I understand those concerns, this work seems to involve simply their taking initial conditions and plugging it into their simple simulation, and obtaining a result which was a remarkably good match to the actual world map of the time.
I do think the Ars article, like most scientific reporting, restates the conclusion in a way that is a stronger statement than the actual paper. Unfortunately, the way it was said changes the meaning of what was said in a subtle but important way. But.. that's typical scientific reporting, I guess. Overall the work is pretty cool, showing a computational model for studying the spread of military technology in a field that doesn't tend to frequently use computational models (according to the paper).
The problem, though, is not everything is like particle physics. Particle physics is able to be publicized as a field which will give us a total understanding of the entire universe: a Theory of Everything.
But the example I gave is not as fundamental. It can give us a Theory of Understanding Many Types of Useful Molecules, which is awesome, but I would argue that it really just isn't as sexy a subject to the general populace, and so it simply cannot get as much funding.
I think this is what I'm trying to say: some projects are very accessible, like drug discovery. These will get funded. Some projects are extremely esoteric. But they are potentially revolutionary - particle physics, which theoretically can lead to us understanding everything, ever. But for all of those projects, there are thousands more that lie somewhere in between: esoteric enough that the benefits of the study are not immediately understandable, and without the potential of enormous impact. These, I think, will not get funded.
But science is not built entirely on revolutionary work that redefines our understanding and existence, even though many of the most-admired scientists are typically people who did this type of work (Newton, Einstein, etc). Most of it is small additions to the body of knowledge that is currently known, and does not have the same wow factor or potential application. And these will have a much harder time getting funded.
Say I study the bonds between metal atoms and carbon atoms. Understanding the strength of interaction can potentially lead to understanding the reactions and dynamics of chemicals used on a daily basis - but not directly. Completing such work would not immediately lead to a deeper understanding of chemistry, but would just be a small step that could one day lead to major technological or scientific advancement. How could something like this be sold?