See, that's the thing.

No matter how much effort you put in to prompting and coaxing it, GPT-3 really couldn't manage anything but the simplest encounter-based game design.

There were things that GPT-3 did really well -- it could write you a 600-word essay on the characteristics and atmosphere of each chamber or passage you find yourself in -- but it couldn't stick to a narrative or engage in anything resembling long-term planning. So, ultimately, you're left with something quite strange and dreamlike: A campaign that's vividly -- even hyper-realistically -- described, but where nothing makes sense and there's no logical continuity over time.

GPT-4 knows what it's doing, though. It's very different. As far as I can tell right now, it combines the outstanding descriptive capabilities of GPT-3 with the ability to (a) remember things it has previously said and establish logical relations over time, and (b) make sure that the campaign appropriately mirrors a "normal" D&D campaign. GPT-3 would frequently mess up and have you fight dragons at Level 1; GPT-4 more appropriately sticks to goblins and bandits. And, when it comes to tactics, pacing, etc., it's really quite good. I suppose it's a little bit conventionally-minded, but that's to be expected.

Almost scary to think that it'll only get better from here on out, because GPT-4 is damn near "good human DM" tier, and already much better than a bad DM.

I've been extremely impressed with GPT-4 so far, generating a campaign for my kids. The encounters it created were very well fleshed out (monsters, minions, stats and abilities for all, including very flavourful lair actions for bosses). It's given me NPCs and sample dialogue for each, locations, and a central storyline that is well-threaded throughout. I was genuinely astonished even at its first attempt. I don't know if it could stand up on its own as an interactive dungeon master, but as a tool I am loving it.

What does mindless modern encounter mean? My sense was that modern players are more plot and narrative focus whereas old school players were down to stuff like roll for a random encounter while traveling and actually play out a fight with… Wolves! That didn’t matter at all to the plot. It was just what you did.

Edit: but I could be wildly off here?

My feeling is that all this varies far more between groups and possibly even geographic regions that anything else. My memory of the late 90s/early 2000s was that storytelling was the focus, dice where used sparingly and that the "rule of cool" was the only rule that mattered. All much more so than what I see today.

I've however also spoken to other people, living in different places, and they remember the same time period being all about rules lawyering and "war gaming".

I've also observed a Europe/US divide where the US tends more heavily towards "war gaming" focused role playing with physical maps and moving miniatures around on a table, while Europe tends towards more story focused role playing. But I could be imagining that.

I have been seeing the opposite myself.

In the 90s and 00s it was almost always about the minis and tactical gameplay. DnD 3.5 was a lot more numbers heavy so it kept a lot of the story people out of the game.

Now DnD 5 the numbers have such a gentle, predictable curve with bounded accuracy and damage and stuff, you may as well ignore them because there's nothing exciting about rolling the dice anymore.

So the story is a lot heavier focus, the character journey. I never used to encounter people who got upset if their character died or groups who had the mentality that characters must always fail forward for the sake of the story.

That stuff is everywhere now.

Another aspect I just thought about was that in 90s and 00s there was a lot more diversity of games that were popular. Most people I knew played at least 2 or 3 different games with very different styles and I knew lots of people who never played DnD. Now everybody I talk to seems to play DnD only.

Just the fact that the tile is "D&D map makers..." instead of "RPG map makers..." is very telling.

Also true. Everyone I knew would have a game of Shadowrun or World of Darkness or whatever else if you wanted. At the very least there were Pathfinder groups.

But now I don't even see Pathfinder out there much.

I think DnD podcasts and webseries and such were incredibly successful in cementing DnD as the only game in town. They brought in a ton of new players who have only heard of DnD and only want to play DnD.

My group has gone through a few systems and we still refer to it as a "D&D" group and just understand that as referring to TTRPGs generally. I wonder how many there are like us.

Note that "DM" was mentioned, so I'm strictly talking about D&D here, other games and genres focus a lot more on the narrative aspects of RPGs in a planned manner (NB: Although I personally would suspect that the "AI"s we got might do even better at PbtA games than D&D)

Here, at least since mid to late 3rd edition, we got a very strong focus on combat "encounters", where you got pre-packaged situations where combat would arise. A combination of monsters, location and possibly events happening during that time. Monsters might even be built for this specific situation, even though their names are generic enough (i.e. a "Orc Battlescout").

Your dungeon FSM gets to a certain state and this encounter gets triggered.

Whereas the canonic ideal of older games was a more fluid setup. Often described as "Combat as War" vs. today's "Combat as Sport". Sure, played very simply, you got an even less sophisticated environment: Room #13 has 4 orcs, guarding a chest, which has 380 copper coins. Kick down door, fight orcs, spend coins on torches for next dungeon. But no experienced DM sees a dungeon that way. Because next to that, you got room #14, where the orc chieftain is preparing the prized pigs they stole from the goblins in room #27, which are currently sneaking up to get it back. Then, as you say, you might have your random encounter, so while the players are heading for the orc room, wolves attack (the goblins aren't guarding the wolf-forest-gates, the DM muses). The fight makes a lot of noise. The orcs look. The orc chieftain looks. The goblins use that distraction. The characters might notice that or not, etc.

Even rather simple setups can achieve rather complicated results, if you're not emulating a 12 year old mastering their first adventure where they just picked monsters from the tables in their Dungeon Masters Guide that are appropriate for a level 2 party of characters.

That's a bare minimum where I'd consider a "AI" DM-like, otherwise you mostly got a NLP interface on gameplay that I could've had with the old SSI Gold Box games on an C64.