The main thing about GPT-3 is that they wanted to demonstrate one-shot fine-tuning and succeeded at it.

So the model can be transformed to output part-of-speech words, dependency grammar trees or named entities in input even if training data is sparse. Similarily, you could fine tune it to produce game lore and then see how it works for that. The model easily switches to different modes of operation and achieves state-of-the-art or close to state-of-the-art performance.

It's quite funny how NLP folks tried to solve low level tasks (POS tagging, NER, Named entity relationship extraction, dependency parsing, sentiment classification etc.) to get to higher level tasks (good summarization, machine translation, text generation, question & answering) and now a single model captures all the low level stuff for free and does high level stuff so good that finetuning it to do low level stuff is unnecessary.

This, the difference between one-shot fine-tuning, vs fine tuning for GPT-2, is one of the major breakthroughs. Since GPT-3 is so hot in the past few days, people seem to forgot or not realize lots of the GPT-3 examples shown off today were possible with GPT-2, with the catch that you had to fine-tune your own GPT-2 model to fit your problem domain (game plots, poems, music, bots that chats like certain characters, etc). GPT-3 makes that fine tuning process unnecessary (although practically you probably can't/can't afford to fine-tune your GPT-3 model)

Are you sure that set out to proof one-shot works? Maybe they found fine-tuning performance disappointing and decided to publish this instead.

Seems like a minor technical challenge (at least in the case of games)

1. Set up a pseudo-adversary NN trained to recognize context-correct speech based on a small corpus. 2. Craft a GPT-3 prompt to get N 9s of accuracy 3.Retry if the answer fails the test from the other NN 4. Set a cap on retries based on how many 9s your prompt got 5. If cap exceeded, return a context-free or limited context response

Sorry. Fixed list:

1. Set up a pseudo-adversary NN trained to recognize context-correct speech based on a small corpus.

2. Craft a GPT-3 prompt to get N 9s of accuracy

3.Retry if the answer fails the test from the other NN

4. Set a cap on retries based on how many 9s your prompt got

5. If cap exceeded, return a context-free or limited context response

This solution would indeed work well enough for me (I'd be paranoid enough to add a tiny blacklist at the end just in case).

> There needs to be a way of absolutely limiting it,

Usually, it's fine-tuning (training a model starting from an earlier model). They can use a small amount of text to fine-tune GPT-3 to their liking.

Well, I've seen examples where GPT-3 strayed quite amusingly. I'm hardly an NN expert, but my understanding that one can't assure the model wouldn't do this (beside retraining on the gamedata corpus alone, which would obviously impair the NN). There are good reasons for game devs to want a measure of certainty here.

Someone else in the thread suggested using a verifier based on game data and maybe that would be fine. The key IMHO must be some kind of NN trained only on game data, either GPT-3 itself or a verifier of some sort.

In the AI dungeon app, regardless of the setting, you can ask the NPCs to sit down and play super smash brothers with you, and they will.

Why not? It would work for any game set in real past or potential future of today.

There are issues here beyond immersion. One of them is that so far, nobody can peek inside the GPT-3 NN and find out how the game devs trained it.

If a player got something extremely disagreeable from an NPC, was this a fluke or did a dev intentionally add it in training as to make it more likely? There's no way to prove innocence. Add in trigger-happy social media + governments, the potential cost to devs & publishers could get all the way to bans/boycotts/legal threats. Most companies do not wish to risk this, so mitigations must be in place.