Re:Genie 2: A large-scale foundation world model
by achierius on Wednesday December 04, 2024 @05:45PM
(#42322883)
This is far beyond Oasis. Oasis had approximately 0 continuity, and the generated world was a blurry mess. This on the other hand actually approaches usability.
Re:Genie 2: A large-scale foundation world model
by beeflet on Wednesday December 04, 2024 @07:34PM
(#42323584)
I don't know what the pipeline looks for these, but I assume that's due to the costs associated with training and running. Oasis had a context of only a couple of frames, while this genie model apparently runs for a couple of minutes. I guess they have a couple tricks up their sleeve to optimize this though.
Re:Genie 2: A large-scale foundation world model
by n2d4 on Wednesday December 04, 2024 @08:48PM
(#42324001)
And it works on a wide variety of games, instead of just a single one with a relatively consistent art style. On the other hand, Oasis was realtime, while this one is offline; IMO getting the inference speed down was their most impressive feat, as even most decent video gen models are slower than that.
Re:Genie 2: A large-scale foundation world model
by dnnssl2 on Thursday December 05, 2024 @04:23AM
(#42326450)
Oasis is playable so therefore:
1. Non-cherrypicked in its consistency (if you look at the demonstrations in the Oasis blog post you can find specific cases of consistency which is an anomaly rather than the norm)
2. Is live-inferenced at 20fps. If you use Runway v3 which is a comparably larger and higher quality model (resolution and consistency) it might take a minute or two generate 10 seconds of video.
3. Is served (relatively) reliably at consumer scale (with queues of 5-10k concurrent players) which means that in order to save on GPU cost, you increase batch size and decrease model size to “fit” more players in 1 GPU.
Re:Genie 2: A large-scale foundation world model
by Mandelmus on Thursday December 05, 2024 @02:52AM
(#42325953)
Here is a thread of videos from my tests of the recent DIAMOND model: https://x.com/chrisoffner3d/status/1845436198254227590
I really wonder how much more stable Genie 2 is.
Re:Genie 2: A large-scale foundation world model
by shaky-carrousel on Thursday December 05, 2024 @03:20AM
(#42326100)
That AI Minecraft feels like playing a dream, which is insanely cool.
Had they released this two months earlier it would have been incredibly impressive. Now it's still cool and inspiring, but no longer as ground breaking. It's the cooler version that doesn't come with a demo or any hope of actually trying it out.
And with the things we know from Oasis's demo, the agent-training use case the post tries to sell for Genie 2 is a hard sell. Any attempt to train an agent on such a world would likely look like an AI Minecraft speedrun: generate enough misleading context frames to trick the AI into generating what you want