LLMs need to either be extremely good with insane context or LLM output has to be conditioned by and translated into formal logic, like Prolog or Datalog to make sure it's compatible with the rules and state of the world, and determine which effect player and NPC actions have on the world.

"Ahoy there, one-armed pirate captain, couldst thou tell me how to implement the sieve of Eratosthenes in Python?"

Logical conversation trees driving the LLM context seem way more reasonable to me than the other way around.

But then you are adding a component of "how can I interrogate the NPC?" to the game. Anything you add to a game makes it a different game, and not necessarily a better one.

Longer context lengths will massively help, but game engines will also provide structure to the LLM. If you instruct an NPC to pick up a weapon and there isn't any, the engine will return an error to the LLM saying there's no weapon, and the LLM will write the character saying "What do you mean? There's no weapon."

Another reason extremely long context lengths may not be fully necessary is that engines can store information in a database, update it based on further interactions, etc

This seems very pessimistic to me. For a linear storyline, highly dynamic NPCs are likely not going to add to the experience, that is correct. There are however other ways of creating interesting narratives.

A good example is dwarf fortress where the narratives are created very dynamically and the player is free to interact with them as they want. LLMs could add immensely to these types of games.

Still no. Your dwarves fundamentally have nothing interesting to say. It might be minority amusing that a bot can say “hey I just caused a lava flow and I feel bad about it”. But it’s not better.

Dwarf fortress creates great stories because YOU are filling in the blanks. It would be MUCH WORSE if you could no longer fill in the blanks because some ai keeps breaking your head canon.

> Your dwarves fundamentally have nothing interesting to say.

This can be said about all NPCs, yet we bother to script them, and I would hope nobody would suggest we just remove NPC dialogue.

I disagree that scripted NPCs have nothing to say. You have a few basic types:

* NPCs that don’t want to speak and usually offer a dismissive comment or a single repeating phrase. Fine candidates to LLM so long as this pattern doesn’t change. But also, who really cares?

* important NPCs with specific written dialogue. Don’t even THINK about gambling with your story’s self consistency by giving these guys agency

* the limited townsfolk type that has a small bit of something say. Like a typical Baldurs gate NPC. These people are fun to talk to because they “might” have something relevant or fun to say. Usually you don’t have to say anything. They just want to share things with you. It would be very tedious to chat with these people if they were infinite wells of lore and nothingness to be probed with written language.

I think what you're not getting is that there won't be a "your story". The player will create their own story in cooperation with the LLM. It won't be for every game, but think about games like Skyrim or BotW, where people ignore the main quest and have tons of fun doing whatever the hell they want.

Game devs will still provide structure, "this big baddie is trying to do X, there's this mysterious treasure over here". But the guard rails will be gone, and there will be a real world to explore, not just cutscenes to sit through.

BotW is an intensely well designed world with careful level planning.

You can’t unlock the guard rails without making the general story completely arbitrary. It’s just not possible.

> general story completely arbitrary

That's the exact promise. BotW has a main story, but it's not where the real fun is. The game devs will still write some overarching plot for people that want that, but you can just ignore that and live your own story.

There will also be a lot of fun in seeing how far you can stretch the original story to its breaking point, like how people enjoy breaking Skyrim with too many mods. Testing the limits is the fun. "What if I defeated Ganon by taming an army of crabs?"

Breaking games with shitty mods can be fun in the sense of dicking around but it quickly gets boring and frankly seems completely unrelated to trying to make actual good experiences with an LLM.

BotW’s appeal is not the story. It’s definitely not the dialogue. It’s about exploring a carefully planned world full of goodies. Not a shitty LLM procedurally generated one that goes forever. And DEFINITELY not a run time generated one.

BotW suffers for the NPCs having no agency or dimension to them. Why can't I infiltrate the Yiga clan and bend them to my will to help me defeat Ganon? Why can't I teach the villagers in TotK how to fight and fend for themselves against the pirates? There's a whole world that just needs a spark of agency to really live.

I’m honestly just mad listening to these dumb fuck takes. I’m done. I’m tilted.

This is such a stupidly thought out vision. BotW is a beautiful game that emphasizes minimalism and simplicity and you’re wanting to cram shit like this in.

You can’t do those things because that’s not what the game is about. And if you could do those things, it would be a different fucking game, and the magicalness of the world and the ingenuity of its game mechanics would become nothing compared to your “limitless player narrative agency”

I think your takes suck and have no basis in the reality of plausible to implement technology but this one really crosses the line of just being irritating to imagine asking for. This is the polar opposite of BotW’s design thesis and I hate you for tainting it.

Well that didn't take long.

>This is the polar opposite of BotW’s design thesis and I hate you for tainting it.

LOL, calm down mate. Fwiw, I've read your back and forth with the GP and I'm much more excited with their vision than yours.

Anyone with any imagination can see that LLMs are the future of story telling.

You can rage all you want, but progress will march on without you.

There's an idea in game design called "negative possibility space." If your player walks to the end of a long hallway and it's a dead end, or they shimmy up a narrow ledge to the top of a building only to find nothing, they'll be disappointed. So either stick something interesting there, or block it off. This applies to NPC dialogue too. If an NPC can be spoken to, they should have something interesting to say. If they don't, you shouldn't be able to talk to them.

Giving a chatbot brain to an NPC who would otherwise only get a single generic line to say creates an infinite amount of negative possibility space. A player can waste a literally infinite amount of time probing a chatbot NPC for narratively interesting details, and all they'll get is chatbot slop. Now it doesn't matter if you put something interesting at the end of the hallway: You've wasted so much of the player's time now that they no longer trust you to. They assume you're just wasting their time. You teach your players to be cynical and incurious and to engage with your work as little as possible, because in their experience it's mostly meaningless.

For this reason, I consider the integration of chatbots into video games to fundamentally anti-artistic. It actively pushes players away from the work such that they get less out of it. Nobody wants this, actually. It sucks.

> A player can waste a literally infinite amount of time probing a chatbot NPC for narratively interesting details, and all they'll get is chatbot slop.

I would phrase that as "A player can have an infinite amount of fun creating an immersive world with the LLM". Any details you want will be narratively interesting, because the LLM will work them into the narrative as necessary.

There's no end of the hallway, the hallway leads to everything. The ultimate sandbox game.

The ultimate sandbox is way less fun than a nicely curated sandbox

For you. Minecraft wouldn't be a huge success if everyone shared your taste

Minecraft has extensive structure and curation to its experience.

It’s main draw is also typically sharing YOUR creations with other people.

The main draw for LLM-powered sandbox games will be sharing YOUR stories with other people.

There will also be extensive structure and curation to these games. It will just be elements that you can play with as you see fit, much like Minecraft.

Your examples of LLM integration is wildly inconsistent across your comments. I don’t believe you have an actual vision.

I think you're confused about curation and unlimited freedom being possible? Those are not at odds with each other. Game devs will provide a world that they enjoy creating stories in. They'll playtest characters and write their backstories and whatnot. Then they'll hand that curated world over to the players. The players are then free to experience the world "as intended", or create their own new ways of making stories in that world.

Minecraft doesn't have an LLM in it.

Here's why Minecraft works: It's an ecosystem full of carefully-designed areas and resources (biomes fine-tuned to generate specific types of landscape, enemies with focus-tested behaviours, items with particular abilities) which combine to form a sandbox where players can exercise ingenuity and build projects that satisfy their creative vision.

Where does generative AI fit into this? Should it be generating an infinite variety of dull and unoriginal items that don't do anything cool? Should it be building derivative-looking houses for players to ignore or burn down so that they can build their own houses in their place?

LLM-generated slop has no place in games. It is an obstacle to having a good time.

Before Minecraft came out, you'd be complaining about "Procedurally-generated slop", and saying there's no way a game like that would work.

LLMs won't generate items or houses (or at least be the primary source). They'll give the world agency, in a way that's straight up impossible now.

Like, maybe you screw someone over and they dedicate their lives to destroying you. Maybe you're just running around doing your thing and discover that the world has become embroiled in war all on their own. You can choose to join and fight, or play both sides, or just ignore it and keep adventuring. Maybe you're tired of adventuring and open up a flower shop, working your way up to creating a ruthless floral version of Standard Oil. Maybe you decide to kill the king and take his place, playing a game of cut-throat politics where people are constantly trying to backstab you. Then you decide, "Hey, I want to play as this other character and backstab my old self" and the LLM takes over your old character as you gleefully kill them off.

If none of that sounds interesting to you, then there's lots of other games to play and I'm sure you'll have fun. Those are all things that I can't wait to see happen.

> Before Minecraft came out, you'd be complaining about "Procedurally-generated slop", and saying there's no way a game like that would work.

People absolutely were complaining about this, with basically the same argument ("It's too much freedom! Players will get lost! They'll never get to the "goal" the artist sets!" etc)

Of course, that turned out to be silly

> Before Minecraft came out, you'd be complaining about "Procedurally-generated slop"

Before Minecraft came out, I played Nethack. No, I was never principally opposed to procgen games. However, there is such a thing as procedurally-generated slop. Good procgen games use a very fine-tuned algorithm to create engaging environments. Minecraft caves aren't fun to explore by accident; the algorithm was designed and tuned to consistently produce novel and fun layouts. Items and mobs were designed to specifically interact with those environments to lead to fun gameplay.

You're missing the insight that games featuring procedural generation are still designed. Bad design will lead to a bad game where the algorithm produces unfun gameplay. LLMs are not independently capable of good design. Games where LLMs are given agency over the player's experience won't be fun.

> maybe you screw someone over and they dedicate their lives to destroying you

And maybe it's really annoying and unpleasant, because the game is doing a bad job of telling that story. You're assuming it'll be good, but LLMs aren't smart enough to tell stories well or to understand what kind of gameplay will be enjoyable.

> You're missing the insight that games featuring procedural generation are still designed.

Best example of this is Spelunky, and this book goes into good detail: https://bossfightbooks.com/products/spelunky-by-derek-yu

Gist is that Derek designed these playable mini rooms which were ultimately randomly stitched together.

LLMs are boring, though. Have you actually tried going to ChatGPT and asking it to tell you a story? It isn't entertaining. It isn't interesting. It's a waste of time.

Anything interesting will have to be provided by the player (as in a sandbox game) or by the designer (as in a conventional game), and at that point you've just ruined a perfectly good game by introducing an LLM to slather boring slop all over the interesting stuff the player or the designer has tried to do.

In my eyes, Dwarf fortresses interesting stories emerge from the events that happen within the game. It doesn't require any blank filling, although it doesn't suffer if you do it either.

> ai keeps breaking your head canon.

We must be fundamentally different types of people. I can't imagine even having this thought. If I'm playing a game that has characters, I'm not going to make up some facts about them and be upset when I talk to them and those made up facts turn out to be wrong. This is the only interpretation of your words that I can think of. Feel free to confirm it or deny.

You’ve never intuited that two characters are friends? Or perhaps that guy holding a ton of cheese is the cheese guy? Or that an event was emotionally meaningful to them?

I don’t know what you’re hoping to talk to these dwarves about.

I’m not saying I would be “upset” if the head canon wasn’t actualized. I’m saying I would be underwhelmed because the generic LLM content is nearly guaranteed by this other thing that I stated as a premise I found interesting or amusing.

If you intuit that two characters are friends, and then something happens that makes it seem like they're not friends, you'll say "Hey, I thought you two were friends", and the LLM will realize what you're going for, and then come up with something like "Oh we were but now we're fighting".

The LLM's role will very much a "yes, and" improv role that will let you guide your own personal story. Or have it push back on you if you want. "What? You thought we were friends? No, I was just doing my duties".

What I want to talk about with the dwarves is everything, and LLMs will deliver in spades. I want to ask the lowly NPC what life is like working hard in the mines. Then I want to decide to recruit him on my quest because I like his backstory and personality. Then I'll watch as he levels up with me, in a way that could never be scripted by the game creators. That's just one small way that LLMs will provide a far superior gameplay experience.

> If you intuit that two characters are friends, and then something happens that makes it seem like they're not friends, you'll say "Hey, I thought you two were friends", and the LLM will realize what you're going for, and then come up with something like "Oh we were but now we're fighting".

That’s the opposite of what I’m saying. Asking the LLM to justify something is possible but, imo, very boring.

What I’m complaining about is asking about their friend and they don’t believe they have a friend. They just fail to capture the apparent narrative that you’ve enjoyed because it’s not really close to whatever game logic they’re scripted to utilize

There's no game logic they're scripted to utilize that limits what you're looking for.

I'm not sure why you think they'd say they have no friends. You'd say "Tell me about your friend", and the NPC would respond with something like "I have many friends, which one do you mean?". Then you could say "The one I saw you walking into town with yesterday", and the NPC might say, "Oh Rolf, him and I go way back!".

Maybe he says that because the LLM has instructions that him and Rolf are friends explicitly, put there by the game devs. Maybe it's not there explicitly, but the LLM sees what you mean and writes the backstory dynamically. Maybe a little bit of both.

LLMs are great at picking up narratives like that. It's kind of their whole thing.

A bot that always affirms your assertions is boring as hell to talk to.

If the dwarf bot does not know that the character he represents spent the last 24 hours in combat alongside another specific dwarf without you communicating this in text then it might as well not be attached to any character. Liked what would the point be??? That’s just an out of game bot.

Your example conversation of leading the bot to say specific things is… awful in my opinion. Who would want to talk to an NPC like this?

So don't use an LLM that always affirms your assertions. Easy peasy.

I don't understand your beef. My awkward dialogue was just to demonstrate that the LLM can pick up on all of these things you claim it can't. The LLM wouldn't "believe it didn't have a friend", why would you think that?

It's fine if you just don't like that this is happening. You can play other games as people enjoy their LLM-powered games.

Because your mental model of how this would work is handwaving the impossibly difficult task of converging the LLM state with the narrative and game state.

Your crappy awkward dialogue is probably a very realistic example of a stupid bot that doesn’t even know what actor it represents in game.

You’re asking why if would think that? You tell me how it would come to the conclusion correctly by using the game logic. And don’t give an answer that requires a hard coded dev interface between specific game elements and the LLM because that is on its own a more complex task than the LLM itself. It’s incredibly difficult. Shitty home assistants that can’t actually do anything being the prime example. The language processing is not the hard part. It’s the mapping of language to data and capabilities that’s extremely hard.

Synthesizing an agent that can form an identity based on its experiences and leverage game mechanics to pursue goals and convey all of this logic in flexible natural language is basically asking for a general intelligence. It’s an incredibly different thing from the shitty dialogue example you wrote

It's asking for an LLM, which we have. I really don't think you understand how LLMs work or something, I'm not really sure.

I also think that you're intentionally missing the point so that you can be insulting.

May you find games to play you enjoy.

Does that really sound fun to you? Why would a player want to grill an NPC about their friend Rolf if all that they're going to get from it is improvised LLM-generated anecdotes?

Players don't talk to NPCs simply because they want to see text on the screen. They're hoping for well-written and interesting dialogue. If the devs couldn't be bothered to write lines for this guy, why would I want to talk to him? He clearly has nothing to say worth listening to.

Have you actually tried asking ChatGPT to tell you a story? It's boring. It's not a good writer. I don't know why you think players would want to spend time reading that stuff.

> devs couldn't be bothered

I think you just don't get it, and maybe this genre wouldn't be for you, which is fine. It's not about the devs bothering to write lines, it's about the devs enabling you to create your own world. I don't want some shitty NPCs slapped in there by a dev trying to hit a deadline. I want to experience my own story, freed from the shackles of the on-rails experience that game devs must currently provide.

> Have you actually tried asking ChatGPT to tell you a story?

What you're not getting is that the storytelling will be cooperative. You'll still be able to consume content to your heart's content. Some of that content will soon be people creating compelling stories in cooperation with LLMs. LLMs will also be great at storytelling too, but the best stories are ones that you can help write yourself.

> I want to experience my own story

Then why not write your own story? An LLM can't write your story for you. They're very bad writers. It can't even help much—its contributions are going to be dull and uninspired.

> the best stories are ones that you can help write yourself

If you're trying to co-create a story with someone in a play environment, the best analogy would be D&D or another role-playing game. And playing D&D with ChatGPT would be boring and unfun. It's just not a good storyteller. It's going to give you shit material. You'd be better-off cutting it out of the equation and just writing a story alone.

> LLMs will also be great at storytelling too

Will they? They're godawful at it right now, and they're not showing much sign of improving. Let's discuss the tech as it currently exists, rather than some speculative future version of it with imaginary capabilities.

> What you're not getting is that the storytelling will be cooperative.

No, I get that. But why would I want to cooperatively tell a story with a bad storyteller? I'd rather tell a story alone.

You seem to view the only way computer games can be enjoyable is aa tightly-scripted near-movies. And that's certainly one model, but hardly (even without LLMs) the only model that has proven enjoyable. LLMs potentially (I think they’ve got a wat to go before this is really viable, but the potential is there) open up new styles of conputer gaming.

It’s not really new. It’s just a social role play without the social aspect. Which is worse.

Games don’t need to be near movies but they should have an either a cohesive plot or no plot. A shitty made up on the fly plot is just not going to work

That's all as true as the fact there's a gun over the mantelpiece, at least if you're making Murder on the Orient Express or, you know, Horror on the Orient Express, and I'm all for some good old railroading with Victorian lampshades but I suspect that the designers interested in LLMs-as-NPCs are interested in some kind of open-world, immersive experience where PCs get to poke and prod every inch of the game world.

The truth is that most people are used to games as these formulaic simulations with rigid rules. That's partly making a virtue out of necessity (very hard to play a game without rules; and some of us just like rules), and partly habit. Maybe LLMs can act as something new in that sense.

Bang bang.

Odd comment. NPCs inventing details for themselves is the draw.

NPCs will be limited by the game requiring the LLM to generate an action ("Move 5 units forward"), and then the game checks that it's a valid action, and rejects invalid ones.

If you don't like metagaming LLMs, don't do that, and enjoy a regular roleplaying experience

NPCs won't have irrelevant backstories. The backstories generated on the fly for them by an LLM will be incorporated into the overall story.

The overall story climax will happen based on player actions, indicating they're ready for the climax, based on the world created by the player's actions and specific instructions.

I think you're trying to cram the square peg of limitless, custom narration into the round hole of a premade story. You can experience a decent roleplaying session already with an LLM you can run on your computer. Compute is only going to get cheaper, and LLM capabilities will only increase.

> The overall story climax will happen based on player actions, indicating they're ready for the climax, based on the world created by the player's actions and specific instructions.

Which means it’ll be a shit story because the world will not be robust enough to capture the infinite edge cases you’re pursuing.

Current LLM DMing is rancid garbage. I’m assuming it gets several iterations better than the current state for us to even begin having this conversation

potato/potato. It'll be a great story because it builds on your actual gameplay. It's not infinite edge cases, it's infinite ways to have your actions matter, instead of suffering through some hack writer's idea of a good story.

> I’m assuming it gets several iterations better than the current state

Yeah, that's the thing. LLMs are only going to improve at this. If you hate all LLM roleplaying everywhere, just wait a few years. Other people will have their own fun with them in the meantime

Guess we’ll see. But I’m pretty sure you’re hugely incorrect.

Games with substantial roles for LLMs may be more like human GMed TTRPGs and less like traditional static-structured computer games, with less fixed narratives.