High-Throughput Generative Inference of Large Language Models with a Single GPU

Re:High-Throughput Generative Inference of Large Language Mo... by fisjy on Tuesday March 14, 2023 @04:21AM (#35148646)

This is my thinking also. LLMs as chatbots are the least interesting possibility imho. I can foresee LLMs being used as an 'OS' for NPCs, where context from the environment/player is updated and appropriate actions are made by the NPCs according to the 'persona'.

I expect this would lead to much richer emergent behaviour, such as the player donning beggar clothes and walking into a city eliciting a very different response than walking around with a sword in one hand.

As a gamedev, this is an area I'm actively investigating.

[ Parent | on HN ]

Re:High-Throughput Generative Inference of Large Language Mo... by fnordpiglet on Tuesday March 14, 2023 @10:30AM (#35152117)

I think it can serve for more than that though. If you construct a prompt that provides a structured enough output you can then ask it to “tell me a story about X” then ask if clarifying details about objects in X, and from that generate an entire story arc with characters etc and load it into a game runtime using the structured format of the responses (I.e., create places and things and npcs). As the player interacts you can have it continue to expand the story assembled. Further you can integrate the visual AIs to make characters and places based on the language AIs descriptions. Then of course you make characters chat bots - but I think the whole thing generalizes to a lot more - the statefulness of the chat can also be used as the statefulness of an unfolding story.

[ Parent | on HN ]