Data can sometimes reveal what an expert spent years to learn and sometimes much more. So when used as an exploratory tool, it has good benefits. And to a degree, it can predict future outcomes to a good degree if we have a great theory about the target domain that explains almost everything. Unfortunately, this tempts people to try using data in domains where their expertise doesn't justify their predictions and then it becomes a gamble where the winners get a lot of publicity.
I saw a documentary last year on YouTube about how F1 teams are using data (ML) to make decisions on each aspect of the race. On the last lap, they decide to trust their experience and intuition rather than listening to the ML suggestions, which costs them a couple of positions. Maybe not today, but we are not far away from the time when the F1 drivers stop listening to "humans" ;)
Link? That sounds pretty interesting.
Don't remember the episode number but it's one of these: https://youtu.be/UwsrzCVZAb8
Data can easily miss a whole lot of subtlety. Subject knowledge, theory, and intuition can capture much more of that subtlety, at the cost of being more vulnerable to bias. Making decisions purely based on data misses out a lot. And can still lead to bias, not caused by the decision maker, but by the data gathering process.
In my mind, data exists to detect bias, and illuminate interesting phenomena. But decisions should ultimately be based on in depth knowledge and theory. And leeway should be given for those to be somewhat misaligned to data.