Re:Goodhart's Law
by mumblemumble on Friday September 17, 2021 @04:24PM
(#28570491)
I would argue that it's worthwhile to measure as much as you can, insofar as it facilitates orderly decisionmaking processes.
The problem is that people tend to think that all measurement is necessarily quantitative. I think that this might be a version of the streetlight effect? Quantitative measurements tend to be much easier to collect and analyze than qualitative measurements. Oftentimes you can let it all run on autopilot, whereas doing good qualitative work always requires concentration, effort, and expertise.
Re:Goodhart's Law
by dragonwriter on Friday September 17, 2021 @08:05PM
(#28572258)
> I would argue that it's worthwhile to measure as much as you can, insofar as it facilitates orderly decisionmaking processes.
That would be true, if measurement was free. It never is, and it often is quite costly.
What is qualitative measurement? As soon as it is quantitative, it isn't qualitative anymore, is it?
Re:Goodhart's Law
by mumblemumble on Sunday September 19, 2021 @06:52PM
(#28588686)
Here's the first Google hit: https://study.com/academy/lesson/the-difference-between-qual...
Re:Goodhart's Law
by rossdavidh on Friday September 17, 2021 @03:09PM
(#28569688)
In my experience, if it isn't measured, then it is assumed that the policy (whatever it is) is working. If you don't measure, you can't be surprised by "wow, it didn't work like we thought". Therefore, mistakes don't get recognized or corrected. There are many tools, but measurement is one of the only ones that brings unexpected bad news to the user, and that is invaluable.
It wasn't Drucker either.
https://medium.com/centre-for-public-impact/what-gets-measur...
This whole mindless must measure, must measure mentality has been criticized since the 50s. Measurement is a tool. There are many tools.