All these chatbots already have public APIs which tens of thousands of random developers are using for various tasks.

While it's fun to debate airy fairy philosophical topics, there is a practical issue that needs to be dealt with: people are using chatbots to do things, and these chatbots are unpredictable in sophisticated, occasionally dangerous ways.

It doesn't take much imagination. One difference between a ten year old Garmin GPS device and ChatGPT is that there's a 0% chance the former will regurgitate behavior from Demon Seed and purposefully direct me to drive off a cliff.

Ok, but there's thousands of people larping D&D, and that doesn't mean they're going to suddenly be given magic spells and cast fireballs everywhere.

These studies are entirely the same as roleplaying, and that roleplay only effects people stupid enough to wire an LLM into an integral system.