News for hackers. Stuff that matters.
Re:Why “Trusting the Science” Is Complicated by AnthonyMouse on Monday February 15, 2021 @01:33AM (#26139777)
> There are reasonable accepted empirical approaches in accepted use today. Any team that follows these methods and reports honestly can be trusted more than a team that doesn't.

The underlying problem isn't actually a problem with science, it's a problem with politics.

You need to hire some scientists to do your experiments. If they follow the scientific method, you get results using the best system we know how.

But if they follow the scientific method then the Republicans work to defund climate science research and the Democrats work to cancel any scientist whose results challenge the orthodoxy on race or gender.

And if politics interferes then it's not science anymore. If research leading to politically inconvenient results has its funding withdrawn or scientists are intimidated into self-censorship then it's all just selection bias and fudging the numbers.

Maybe what we need is to do something like the court system, where you have multiple research funding "judges" which have lifetime appointments and each get a given amount of research money to dole out every year, and their job is to decide how to spend it. Then they can spend it however they like and can only be removed for e.g. corruption, not for funding research somebody doesn't like.

Re:Why “Trusting the Science” Is Complicated by dataduck on Monday February 15, 2021 @05:01AM (#26141106)
This is the fundamental problem with "evidence led policy". As soon as science can reliably affect policy outcomes, people who want to change policy outcomes will corrupt the science.
Re:Why “Trusting the Science” Is Complicated by ufmace on Monday February 15, 2021 @10:40AM (#26143717)
Yup this. In much the same way as things like judging the productivity of software developers by the number of lines of code they write. It sounds like a good idea to the inexperienced, and it may even seem sort of true if you look at it in the right way. But we all know, as soon as you try to make consequential decisions based on it, it will immediately be gamed and not true anymore.
Re:Why “Trusting the Science” Is Complicated by sriku on Tuesday February 16, 2021 @12:00AM (#26151095)
You raise an interesting hypothesis - that all interactions between science and policy making is subject to Goodhart's law (https://en.wikipedia.org/wiki/Goodhart%27s_law).

In as much as science deals with things that cannot be affected by policy (ex: stellar physics or nuclear fusion), the "law" doesn't apply I think, except maybe in perception (people may believe that a particular nuclear fusion technique doesn't work even though it's been demonstrated but policy forbade it to be implemented).

For some cases, we would want Goodhart's law to apply - ex: climate change. We'd want actions to be taken to reduce the impact on the planet as we gather and report climate parameters accurately. Similarly, we'd want accurate reporting of gender discrimination in society to result in the reduction of such discrimination. The question really is the way the change is effected. If "salary difference" is an indicator of gender discrimination for example, law makers can pass laws that fix it, thereby removing its effectiveness as an indicator .. but we're happy for even that at some level. That would be similar to a government sending teams to re-freeze melting glaciers because climate scientists are using it as an indicator of global warming.

True policy thinkers, though, would want to ask the "five whys" and work to fix the underlying problem.

Re:Why “Trusting the Science” Is Complicated by ntsplnkv2 on Monday February 15, 2021 @09:34AM (#26143081)
It won't matter.

Eventually the governments who support technology will win - either via war or economic advantage.

Re:Why “Trusting the Science” Is Complicated by AnthonyMouse on Monday February 15, 2021 @11:22AM (#26144127)
How is that not mattering? So if China uses science and the US descends further into hyper-partisan culture war, that doesn't matter?

It matters kind of a lot whether the winner is a liberal democracy with free speech and civil liberties or a totalitarian communist dictatorship.

Re:Why “Trusting the Science” Is Complicated by ntsplnkv2 on Monday February 15, 2021 @12:52PM (#26145129)
My point was that eventually the anti-science policymakers will eventually lose out. Societies that value technology and science will win in the end - they will have economic advantages and advantages in war.

> It matters kind of a lot whether the winner is a liberal democracy with free speech and civil liberties or a totalitarian communist dictatorship.

This is irrelevant to the conversation, but yes that is why we spend trillions on defense technology, so that if it comes to blows we don't lose out.