News for hackers. Stuff that matters.
Re:Goodhart's Law by gftsantana on Friday September 17, 2021 @04:07PM (#28570321)
My first job was at the anti-fraud department at a telecom company in the early 00s. Our job was basically to determine whether a new landline or mobile contract was fraudulent or not. Some requests would be flagged by an automated piece of software that was basically a black box to most employees, myself included. We would basically look at the documents sent by the clients and, sometimes, ask a few questions via phone.

I was very young at the time, but I remember basically deriving Goodhart's law after a few months in the job. I don't remember clearly most of the things that led me to that conclusion, but I do remember the most extreme: at some point, management started requiring us to block clearly non-fraudulent phones because the directors decided to increase the blocked installations target. It would include even old contracts by good paying customers that happened to be flagged.

I remember trying to talk to people about this, but the idea that trying to reach a target by any means necessary is usually not a good idea was incomprehensible to most people. Years later, I realized that others knew exactly what was going on; they just didn't care, and I was naive for not seeing that.

It was a few years later when I learned about perverse incentives, Goodhart's law, the cobra effect, etc., and it allowed me to have more productive conversations with people about targets and incentives.

There are no comments on this comment yet.