Hmm; I don't think we fully understand each other.
Absolutely do the study; in fact, do several.
What then, though? A "study of studies", or any method of reconciling or making sense of these multiple studies, is the "meta analysis" in question. It's going to happen, in some way, somehow, by somebody. There is no way to reconcile multiple studies without doing some form of what we are calling meta-analysis. Even if it's just you or me googling it up and then deciding which of the studies to trust, that's a "meta-analysis".
My claim, therefore, is that it's far better to acknowledge this need and reality, and have dedicated teams of experienced professionals do it with an open, consistent method; then for each of us to mentally, subconsciously, "biasedly", to pick & choose and prioritize the studies.
[in the end, proper meta-analysis, and study replication, are the methods of how we discover inconsistency, contradiction, data issues, or other problems, whether in psychology or other domains. It's not a method of "fixing something", but a method of collecting, analyzing, and reporting on multiple studies and data points]
With a potentially large # of studies on a specific subject, some of which may be contradicting, meta-analysis will be done.
Whether by accredited scientists, or institutions who do this on a professional basis with open and established criteria and consistency in approach (though there may still be flaws in the approach); and/or by individuals when confronted with large # of studies, when I feel bias would almost certainly be the primary driver.
While you absolutely cannot eliminate the latter, it feels it would be foolish to dismiss or eliminate the former.
As long as:
a) World is a bit tricky (phenomena difficult to define and isolate; limitations in equipment and measurements; limitations in humans)
b) We use statistical analysis as primary differentiator of significance There will continue to be contradicting studies.
Not disagreeing it's always better to improve studies and experiments, but it feels like meta-analysis of multiple studies is not just a necessary evil, but a good and desirable next step.
Thoughts?