There is for whichever company wants to make use of it - say for example LLM-powered NPCs in singleplayer video games (which I'm working on) - pretty much economically unviable with the current model if you want to run everything on the cloud.

You're selling a game once rather than selling a subscription, so you're locked in to paying for cloud-based LLM services indefinitely, unless you want to shut the game down after X years, which really annoys people. Also, you're incentivised to be stingy with the LLM-based features because the more you use it the more it costs you, but by running it locally you can offload that cost to the consumer.

would love to learn more and help out on NPCs llms. What sort of performance hit are you seeing?

Sure, here's my writeup about it: https://jgibbs.dev/blogs/local-llm-npcs-in-unreal-engine. There's barely any performance hit, only a slight hitch when it starts generating a response.

There’s indeed a lot of GPU time not utilised even by modern games. I can definitely imagine new games utilising llms much more even and with local inference option.

At some point soon, LLMs matching current SOTA will run locally in a browser. If applications for LLMs emerge, having one available will be table stakes.