How much framerate are you willing to give up so that NPCs don't repeat themselves? The elephant in the room with LLMs in games is that the users GPU is usually already being maxed out by pushing pixels, so any amount of compute allocated to running AI models instead is going to have a very tangible cost.

They could be offloaded to the cloud instead but that brings its own set of issues (ongoing upkeep costs, finite capacity, servers will inevitably shut down...)

> The elephant in the room ...

For now. LLMs are still relatively new. The technology will continue to be optimized, and we'll be shipping tiny LLMs along with games soon enough.

(Aren't companies already running LLMs on phones?)

The housing market will only go up, it's always gone up.

I can only imagine nvidia/amd pouncing on the chance to sell companion "AI Cards" that enable LLM NPCs in games. If the tech is good (and insane progress in tiny local models points to "yes"), then I can easily imagine a large contingent of gamers forking over another $600 for a second GPU.

It might not even be necessary though, similar to how physixs (sp?) cards died out quick in the early 00's, the models might become slim enough and the GPU's fast enough that the extra load is easily manageable.

The vast majority of games need to run on consoles to be financially viable, anything that requires an entire second GPU is a complete non-starter. The only way it's going to work is if the models get small enough to run on a negligible slice of GPU or CPU time, with a reasonable memory footprint (<1GB), on whatever the current console hardware happens to be.

I see your point, but it hasn't really stopped novel features from coming to PC in the past. The PC market is about 50% larger than the console market too.

Console manufacturers would salivate at the thought of new hardware that makes people want to buy a new console instead of playing the ones they've already got. Once someone gets it working reasonably on PC, it will quickly spread to a new generation of consoles with new hardware that makes people want to upgrade.

Instead you'll just get variations on "Sorry, but as a large language model trained by OpenAl, I cannot condone the shooting of adventurers in the knees with arrows."

Any production deployment would probably do pre-fetching of potential initial dialogues as some kind of background job.

On this note - and as a response to the point a commenter made that too much freedom in dialogue might actually be a bad thing - LLMs could then be used to expand on the initial dialogue in a way that feels more 'organic' every time you hear it, but is still close to the artists' vision.