Cloud AI for Video Games Is Dead on Arrival, On-Device Is the Future

Re:Cloud AI for Video Games Is Dead on Arrival, On-Device Is... by maiybe on Wednesday January 08, 2025 @12:31PM (#42636463)

Looking for feedback for an AI engine that runs sophisticated game AI locally instead of in the cloud, literally reducing costs to 0. Our tech demo (GARP) runs 20+ autonomous NPCs with memory, planning, and real-time interaction on a single RTX 3090 - something that previously cost $500/day with cloud APIs.

The engine is composable, modular, and integrates with major game engines. We're enabling developers to create deep, responsive game worlds without the burden of cloud computing costs or API rate limits.

Would love to hear the community's thoughts on local vs. cloud AI for gaming applications.

[ Parent | on HN ]

Re:Cloud AI for Video Games Is Dead on Arrival, On-Device Is... by jsheard on Wednesday January 08, 2025 @12:42PM (#42636567)

How much GPU memory are you using? Demanding games already use most if not all available VRAM just for rendering so there isn't a great deal of room left for big AI models. Even if you target games with simple graphics, the size of the AI model would still dictate the min-spec for it to be playable.

[ Parent | on HN ]

Re:Cloud AI for Video Games Is Dead on Arrival, On-Device Is... by maiybe on Wednesday January 08, 2025 @12:52PM (#42636661)

Under the hood, we're supporting multiple models that can be selected, but haven't optimized all the quantizations possible (the space is moving fast).

The range is 1GB - 24GB, depending on model selection, but would be amazing to push lower than that. 24GB is high end as only the NVIDIA XX90s can support those.

[ Parent | on HN ]

Re:Cloud AI for Video Games Is Dead on Arrival, On-Device Is... by jsheard on Wednesday January 08, 2025 @01:00PM (#42636727)

1-2GB might be workable if the model still performs adequately at that level, but anything more than that sounds very hard to justify for as long as the median Steam user and console baseline (Xbox Series S) only have 8GB of VRAM to go around.

[ Parent | on HN ]

Re:Cloud AI for Video Games Is Dead on Arrival, On-Device Is... by maiybe on Wednesday January 08, 2025 @01:19PM (#42636901)

Depends on the fidelity of the graphics, but I agree with you that the smaller the VRAM usage, the broader base we can support on e.g. Steam. 1GB - 2GB would be the sweet spot for all game types, which 1B parameter quantized models can hit.

There is some evidence that next gen consoles will feature AMD NPUs, and I suspect there will be more available RAM. There's definitely positive tailwinds that will change the hardware landscape over time.

[ Parent | on HN ]

Re:Cloud AI for Video Games Is Dead on Arrival, On-Device Is... by dataminded on Wednesday January 08, 2025 @12:45PM (#42636592)

Very interesting. How do we get our hands on it to try it out?

[ Parent | on HN ]

Re:Cloud AI for Video Games Is Dead on Arrival, On-Device Is... by maiybe on Wednesday January 08, 2025 @01:01PM (#42636733)

We are in closed alpha, but eager to talk with folks about what they're working on. Easiest is to reach out to: [email protected]

[ Parent | on HN ]