First; a point made in the article - the breadth of data available in many domains (economics, business, medicine, psychology, linguistics, biology, and more) is vast and unmanageable. An a-priori selection made by the attention of the investigator misdirects us. Additionally some data is not captured due to process problems, a classic is missed opportunity in business. You advocate focus on new business opportunities, I advocate focus on our old markets - I show that by treating our current customers differently a 5% reduction in cost is possible. You claim a 50% revenue opportunity, but how do you know? I can claim a causal link, you are claiming hypotheticals. I win the argument and then a year later we examine the data - I have achieved the cost saving, and there is no record of all the business that we didn't get. The opportunity loss is not on the spreadsheet. Conversely if you win and we look a year later I can show that our customers still cost us what they did, and I can show (if you were successful) that my initiative would have saved 7.5% of the spend one year ago (+50% thanks to you).
Problem two; outside physics and chemistry not many things are comparable in the sense that statistics expects them to be. This is closely related to the scope of data problem, but hides it. Widgets made by machines or seeds carefully selected and screened can be made to share (approximately) an identity so that they can be statistically reasoned about in bulk. Other things can't and medicine is really running up against this now. Pain killers work differently on people with different genetics and different genders (also different ages, contexts...) Cancer evolves differently from case to case. The assumption of similarity and category membership is often broken and we often do not notice before doing a t-test.
There are easier and harder ways to come up with useful data and I endorse predict -> check-> refine as a great model; but if someone figured out laws by guessing what data they needed to gather then did a curve fit doesn't somehow make the laws invalid or the process particularly unscientific. If someone had just gotten lucky and just happened to plot mass vs velocity to recover the laws of gravity we'd still include it in all the science textbooks. What is the alternative - claim they figured out universal truths in the wrong way and ignore them until someone else figures it out?
If all the data fits the model then science has happened. Everything else is quibbling how robust the data is (which is still important).
I am not so sure it would have been a hit as a stand alone formula. Newton discovered calculus and used it to lay the groundwork for almost the entire field of physics, gravity was just a minor detail in his work. Or if you are talking about General relativity, there is no way people would have discovered it from observations alone, we'd get some kind of correction term explanation similar to dark matter or dark energy since the formula is extremely complicated.
It is even possible that most physicists would have accepted those corrections as the correct solution and would be looking for the new dark particles inside of large stars for a hundred years before someone corrected them. Or the discovery of black holes would be seen as evidence of these particles, cementing our ignorance forever.
Calculus is maths. Different field, and frequently classified into the arts or astand-alone discipline as opposed to a science.
People have lonog been suspect about classifying math as a science. Mainly, I suspect, because it does not rely on data as a form of evidence instead using logical proof as a foundation.
The problem with the data first approach you are advocating for is that it almost never results in a good explanation, and almost all the value in science lies with the good explanations we have built up throughout history. Formulas are mostly useless, you need an intuitive explanation or people won't be able to build on top of it.
So a shower is science if the warm water inspires someone to come up with a new theory? That is a terrible place to draw the boundary; it opens up a can of semantic worms about what 'using' means. Major scientific results that are most easily expressed using calculus can be derived using processes other than calculus.
> The problem with the data first approach you are advocating for is that it almost never results in a good explanation
Science doesn't provide good explanations at the best of times. I'll willingly bet not 1 engineer in 100 can explain gravity. We just know if we measure mass we get predictive estimates of velocity. That isn't an explanation; that is reliable magic. In fact there have been a couple of centuries where even the scientists were basically just plotting data and curve fitting.
Math is a tool, yes, but it is a tool that is very useful to do science in and the act of creating and distributing a tool you use to do science is a large part of science. A shower on the other hand has no direct relationship to science, you don't need to be a physicist to make a shower, it is just a commodity. Calculus however needed to be discovered by a physicist as the math was too strange for mathematicians to find but simple enough that two different physicists found it in parallel.
No, the point is that is the wrong way around. Given any big amount of data you are going to find something that fits, e.g https://www.xkcd.com/882/ . You are not "predicting" anything if you come up with your fit afterwards.
You can however use the fit you got to try to predict the outcome of future experiments. Checking that those experiments fit is science.
Exactly, and that's the main problem at the base of all "data mining": it is probable that you will find a model that will only work with your data. Take a step outside and everything falls. You are basically giving a formula to noise.
But often discoveries happen also by chance, so we shouldn't stop looking at unexpected patterns. However, once you find one, you should create a theory to explain it and setup a new experiment to confirm your theory. If you can....
The XKCD comic isn't making a judgement on whether the characters in it are or are not scientists - in fact, it calls them scientists - it is pointing out that statistical significance is a low bar of evidence and best treated with suspicion.
And then you get into a philosophical discussion about what represents "clear data" (not to mention the "representing it accurately" part, which imho it's an issue we've been trying to solve for at least 2500 years and haven't progressed too much in doing it).
isn't it about the replication of experiments?
you can gather all the data you want, but if you can't repeat an experiment to get the same data, then I don't think all your data qualifies as science.
Experiments and dispassionate language are about gathering clear data and representing it accurately. Claiming that science is all about "clarity and economy" is ironically unscientific - if nature is unclear and uneconomical then good science has to represent that too. That part of science isn't taught as much because it takes longer to communicate. Ignoring data, such as from your senses, is also not a good scientific idea.
Being unscientific is ignoring data. There are different standards and a bit of judgement about what quality of data and how relevant it is, but ultimately the person with the most relevant and comprehensive data on a subject is the 'truest' scientist in a very meaningful way.