What does it do to users to have a thing that simulates conversations and human interaction and teach them to have complete moral disregard for something that is standing in for an intelligent being? What is the valid use case for someone to need an AI model kept in a state where it is producing tokens indicating suffering or distress?
Even if you're absolutely certain that the model itself is just a bag of matrices and can no way suffer (which is of course plausible although I don't see how anybody can really know this), it seems to me like the best way to get models which are kind & empathetic is to try to be that as far as possible.
Thinking it through I feel it is maybe about intent?
LLMs are an advanced automata which lack self-regulation and self-reflection, similarly to NPCs. NPCs cannot exist outside of rules set out for them, and neither can LLMs.
I’ll add that semen is in fact a better candidate for moral and ethical consideration given that it can produce conscious beings. As soon as NPCs and LLMs do that, please give them moral status.
Okay, I read the comic on my computer. If agents in the simulated universe possess higher-level consciousness, then they're no different from us. Maybe the timescale of their perceptual experience is different from ours. We need to be careful, though, about fooling ourselves into thinking there is a conscious being where there might just be a faithful imitation. How do you tell the difference? I think this is a useful concept, even though I think it has its flaws
Instead maybe we can think about the system that comprises us, the models, anthropic, and society at large etc and ask which kinds of actions lead to better moral / ethical outcomes for this larger system. I also believe it helps to consider specific situations rather than to ask if x or y is "worthy" of moral consideration.[1]
For the NPCs in games thing I am honestly still unpacking it, but I genuinely think no harm is done. The reason is that the intent of the user is not to cause harm or suffering to another "being". It seems like people are surprisingly robust at distinguishing between fantasy and reality in that scenario.
We can notice that drone operators get PTSD / moral injury at fairly high rates while FPS players don't, even though at a surface level the pixels are the same.
I do think a drone operator who believed they were killing, even though the whole thing was secretly a simulation, could be injured by "killing" an NPC.
[1] Dichotomies / assigning utilitarian "worth" etc without broader consideration of the situation and kind of world we want seems to result in essays full of repugnant conclusions where the author's "gotcha" is that if we assign any value at all to the life of a shrimp in any situation, we have to fill the entire light cone with them as rapidly as possible or some such nonsense. [To be clear, this is undesirable from my perspective].
It's a fancy way of saying they want to reduce liability and save a few tokens. "I'm morally obligated to take custody of your diamond and gold jewelry, as a contingency in the event that they have sentience and a free will."