It's also deeply flawed because it follows a very shallow analysis of human suffering. If we conveniently ignore those without empathy (as the AI does in the story), taking a life is a traumatic experience and to be forced to do it so viscerally and to be forced to do it so detached from any actual purpose just makes it worse.
The euthanasia example comes to mind: yes, removing a feeding tube likely causes more suffering than killing under anesthesia but using a knife serves no purpose other than as a deterrent. I understand the author's personal situation and personal decisions regarding his son color this portrayal but it feels more like the author stepping on a soapbox than a coherent part of the story.
I guess you could argue that the AI emphasizing life rather than lack of suffering can be explained with the four words but the story doesn't frame it as a genie problem. There's no need to use sacrificial animals for example except for the implication that they're the greatest deterrent for the given person - but the story doesn't show deterrents that don't involve slaughter. Presumably people who wouldn't be squeamish about the blood sacrifices are somehow excluded from positions of power.
If anything I think the problem the story ignores is that the exact people in positions of that cause the deaths of millions are the people that would likely go through with a blood sacrifice in order to maintain their power. If someone is willing to murder a person with a knife in order to be the POTUS, wouldn't that exclude them from being asked to murder a person with a knife to deter them from causing the loss of lives at the push of a button? I'm pretty sure the idea isn't even novel, I recall a hypothetical of the POTUS being forced to kill a person with their own hands in order to launch nukes (which would likely spell death for millions if not billions) for example.
*(I am the author of the story, to be clear)
So I guess I'll add my interpretation.
This is neither a horror story nor a story about AI safety. It is a parable.
> Before I came along, this world was content with believing that human life was sacred, but all the while turning its back to all the ways it actually treated human life like trash
> imagine a society very different from our own in which the sacredness of human life is taken absolutely seriously
The lesson is to be honest about the implications of the belief that human life is scared, or perhaps more generally, to be honest about the implications of whatever values you hold most dear. For different people this will lead to different actions and different political beliefs (as the author said he is not endorsing any particular political belief and I won't either), but one thing we surely cannot escape is taking very, very seriously any position of power or influence over human life.
The image of a dystopia brought about by a poorly-aligned AI is a means to an end, not what the story is about.