It’s a mistake because our brains have evolved to predict the behavior of animals and people, so being lazy here means you will be surprised, possibly harmed, certainly incorrect in your opinions about the X% of cases where a LLM is fundamentally a different thing than an animal/person.

It can be a useful and cute fiction, as your Roomba example, but it leads to bad decision making (“I’ll put more goldfish on the floor to make the Roomba happy”).

I guess it all comes down to how important accuracy is to you in a particular context. I anthropomorphize my dishwasher (it HATES wine glasses), but I don’t make professional judgments about dishwashers.

This is pure speculation. There is not data what so ever to back up the fact that anthropomorphizing something leads to harm.

We do that with a lot of things every day and a perfectly capable of drawing the lines when things come down to it.

On a more philosophical note. If we some day end up creating AGI, having anthropomorphized it will actually be to our benefit.

So you are technically correct but this is not about being technically correct.

Of course we have data about how reasoning from an incorrect model tends to lead to incorrect results. We have all the data we need about that. Surely that is not in dispute.

The only controversy is whether and anthromorph is an isomorph of an accurate model of ChatGPT. I doubt that it is, yet is close enough to fool a lot of people, then surprise them at random.

Again. Yes technically incorrect. But you don't have any data that it's actually bad or harmful in some important way.