Goodhart law deals with proxies - i.e. measures not of direct target, which is hard to measure, but of something that is easy to measure, and you consider it related to the actual target. If you need wins and measure wins, wins are not a proxy and thus Goodhart law does not apply. But if, say, it were hard for you to count wins in soccer for some reason, and you measured time spent with the ball per player instead, thinking that if players own the ball all the time, they surely going to win eventually - you'd get a team that is engaged in pointless passing around instead of actually scoring goals.

For surgeons, the direct target is for the patient to get better - or at least live longer and with higher quality of life. Proxy is the measure of a mortality of a specific surgeon. It is not a direct target - as the surgeon can achieve zero mortality by not doing any surgery at all, but his patients would die of suffer from the lack of treatment.

Pro tip for those targeting a high win percentage. If you win your first game (beginners luck?), then stop playing and never play again.

That's what the article is describing, surgeons won't even play the game, they won't even attempt to help certain people because it might hurt their "win percentage".

another pro tip: compete only against weaker opponents (this is common in Boxing).

Wins are likely from systems with fixed rules that don't reward innovation, so winning by the rules of a system doesn't necessarily mean a positive outcome. An example might be the system that lead to the housing crisis. Certainly a few people who gamed things the right way "won" big, but only at the expense of millions of homeowners.

Mortality rates are one measure of many that can tell a story, but they're like incarceration rates in that they've become a perverse incentive that exacerbates an issue. If we say that putting more people in prison is a good thing, that's predicated on an assumption that only those that are being put in prisons are people worthy of being there, whose freedom is more costly to society than their imprisonment. When a district is rewarded for putting people in prisons, what types of things might you expect to happen to the legal systems and populations in those places? Would prosecutors then be incentivized to trump up charges and force people into serving prison time unnecessarily to make themselves look like bigger "winners"? Might police make more frivolous arrests or even stir up trouble in communities to paint a picture of rampant criminality so they can look to be "tough on crime"? Wouldn't you expect to see an all-causes decrease in long term crime?

The problem isn't just cherry picking, but if people can game a system built around limited and gameable measures, then it's going to encourage min-maxing for profit.

the problem here is that mortality rate is not correlated/normalized with the patient risk factor. the problem lies in the performance indicator itself, not strictly people trying to optimizing for what they are asked for. it is likely inhumane to refuse patients that have slightly above average risk - even if there are some considerations to be had, like having replacement organs go to waste, but that is already covered by the donor lists queues - but given people can find themselves out of work for having bad mortality rates, optimizing for the performance indicator it's the only logical conclusion.

also, normalizing for the patient risk will likely just lead to people overstating patient risk, because once you reduce performance to an index, people will focus on what's measured before on what the intended goal for the measurement is.

Physicians already have good reasons to overstate the patient's severity. It's part of the payment formula used by CMS. But there are plenty of incentives not to lie such as jail time.

I'm not an expert in cardiatric medicine or anything, but couldn't patient risk be determined by objective factors like patient age and history of previous heart issues?

This is like asking why software project estimation can't just be done by looking at the size of the project to be built.

I don't see how the analogy works. The entire point is you have a medical record with objective characteristics that can be tied to the risk of any given procedure.

My entire point is that isn't true at all - you have a medical record with some objective characteristics, but it is not possible to calculate the risk of a procedure based on that information. Just like in software, we have a project requirements document with many objective characteristics and we are unable to calculate the time required based on those characteristics.

So what is the risk calculation based on? Presumably the doctors refusing to do procedures they deem high-risk are not determining completely at random. The risk calculation doesn't have to be perfectly accurate, and it seems to me as though heart procedures are better defined than software projects.

(And besides that, although software estimation is notoriously inaccurate, it is still done and written into contracts all the time, because it serves a useful purpose that outweighs its inaccuracy)

Another way of looking at it is that the patients who are refused treatment count as losses. So the current system is not actually maximizing wins, it's maximizing win percentage for doctors.

Ya, I went down this line of thinking too. It does seem like this might too be a poor incentive because now doctors would operate on everyone to try and claim wins, even if the likelihood is exceedingly low. There’s probably some measure like “healthy six months from now” that incentivizes them to try in situations with high mortality where operating is the only chance to improve things but keep them from operating if the risk is too high.

Goodhart is talking about life, not a self-contained system like a game.

>Some measures should be targeted, the problem here is that it is done in an non genuine way

The "not in a genuine way" is exactly what he's talking about.

Your question was the same question I had. The wikipedia entry didn't seem clear to me for what the law is and is not.

I disagree with the downvotes your comment's received.

Mortality rate doesn't account for the outcomes of surgeries which are declined due to risk.

In your scenario, the equivalent for win percentage would be trying to set an easier schedule.