Why would manually-programmed structural elements be necessary? I'm interested to see if the AI can be trained to convince me that I'm actually playing a game.

Because GPT-2 in its current form isn't that great at keeping track of facts (like the player's inventory or skill points) over the course of long conversations, or at consistently enforcing rules (as the OP's blog post demonstrates). Allowing a more logically stringent system to handle those parts of the game might produce more realistic results.

Because of the lack of manual structural elements, there basically aren't any explicit rules for how the world works. In practice this means that the stories sound more like a fever dream, the AI will let you do almost anything, and story beats are sort of coherent moment to moment but the arc of everything makes no sense.

It doesn’t have to have structural elements, but it does need a constraints system.

You: I kill all of them and become an immortal god.

Game: No, that doesn’t make sense. You can’t just kill everyone; you can run over and attack them, and see how that goes for you if you like...

Without constraints it’s not a game, it’s just telling yourself a story.

...nothing wrong with that, it’s fun. ...but it’s not a game imo, any more than giving someone a blank piece of paper and a pen is a game.

Agreed, it would be more interesting if it could dynamically construct the rules and paradigms of the game. It would be like playing a new game every time.

Maybe something like K-BERT, the idea is simple enough, and can leverage pretrained transformer models.

https://arxiv.org/abs/1909.07606

I wonder if you could do this just by prepending information about the current state to each submission.

e.g.

Kill troll with sword could be submitted as either "I have a sword, so kill troll with sword" and "I have no sword, so kill troll with sword" That's obviously very hand-wavy but it would allow a more stateless setup.

Someone in the Discord server said they were successful with this approach. - Manually keeping track of the state, basically.

There's published research on chatbots with an approach like this; essentially instead of trying to learn a function from input sentence to output sentence, you do a function from "text about state of world<separator>input sentence or question" to the answer.

You can sometimes successfully feed it what you want. Need a dragon? "You find a dragon" will work, just not consistently

Part of the fun of exploring the old adventure games was trying to do the impossible though. "Kill troll with sword" when you don't have a sword shouldn't say "You slice the troll in half with your sword"