News for hackers. Stuff that matters.
Re:Genie 2: A large-scale foundation world model by brink on Wednesday December 04, 2024 @12:10PM (#42319528)
What is actually of value here? There's no actual game, it's incredibly expensive to compute, the behavior is erratic.. It's cool because it's new - but that will quickly wear off, and once that's gone, what's left? There's insane amounts of money being spent on this, and for what?
Re:Genie 2: A large-scale foundation world model by adverbly on Wednesday December 04, 2024 @12:20PM (#42319635)
> What is actually of value here?

Noone knows yet. AI technology like this is closer to scientific research than it is to product development. AI is basically new magic, and people are in a "discovery" phase where we are still trying to figure out what is possible. Nothing of value was immediately created when they discovered DNA. Productization came much later when it was combined with other technologies to fit a particular use case.

Re:Genie 2: A large-scale foundation world model by Menu_Overview on Wednesday December 04, 2024 @12:23PM (#42319663)
Well, what's next? Beyond prototyping, I imagine this is an early step towards more practical agents building their own world model. Better problem solving.

Prompt: Here's a blueprint of my new house and a photo of my existing furniture. Show me some interior design options.

Re:Genie 2: A large-scale foundation world model by ilaksh on Wednesday December 04, 2024 @12:36PM (#42319810)
It's an obviously amazing research development.

You just don't like AI.

It can be used for training agents, prototyping, video generation, and is quite possibly a glimpse of a whole new type of entertainment or a new way to create video games.

What's the point of the massive amount of money spent on video games in general? Or all of the energy spent moving people back and forth to an office? Or expensive meals at restaurants? Or trillions in weaponry? Or television shows or movies?

Re:Genie 2: A large-scale foundation world model by nightski on Wednesday December 04, 2024 @12:40PM (#42319860)
Video games bring billions of real people joy. This is sitting in some lab at Google inaccessible to anyone.
Re:Genie 2: A large-scale foundation world model by lassenordahl on Wednesday December 04, 2024 @12:54PM (#42320021)
Is your argument that them sharing research progress and demos doesn't benefit anybody purely because we can't immediately play around with them?

I feel like sharing early closed-source blog-posts is part of the research process. I'm sure someone in this thread has thought of a use case that the Google team missed. Open/closed source arguments here feel premature IMO.

Re:Genie 2: A large-scale foundation world model by nightski on Wednesday December 04, 2024 @02:34PM (#42321142)
It's not part of the research process. Being part of the search process would involve a publication and sharing code/data/results/methods. It's not research unless it can be verified by peers.

This is just a marketing fluff piece that does not benefit anyone and is ego stroking at best.

Re:Genie 2: A large-scale foundation world model by lassenordahl on Wednesday December 04, 2024 @02:55PM (#42321383)
Hm yeah - I think you and I just have differing opinions on the research process. I'd be a bit more vague, and define the publication process as something similar to you.

I still think things like this are important, and at least give folks a bit of time to ideate on what will be possible in a few years. Of course having the model or architecture on hand would be nice, but I'm not holding that against Google here.

Re:Genie 2: A large-scale foundation world model by ThouYS on Wednesday December 04, 2024 @12:37PM (#42319832)
same q here. what can I do with this "world model" that I can't do with a game like minecraft or counter strike?

asked the same thing a while back, and the answers boiled down to "somehow helps RL agents train". but how exactly? no clue

Re:Genie 2: A large-scale foundation world model by ogogmad on Wednesday December 04, 2024 @12:42PM (#42319879)
Making a computer game is very expensive and time-consuming. This technology might allow a 12 year old to produce a fully working AAA-quality game on their own for almost nothing. But sigh it's an early demo that needs some improving.

[edited out some barbs I wrote because I find some comments on this website REALLY annoying]

Re:Genie 2: A large-scale foundation world model by ThouYS on Wednesday December 04, 2024 @12:56PM (#42320048)
lol
Re:Genie 2: A large-scale foundation world model by awfulneutral on Wednesday December 04, 2024 @12:38PM (#42319841)
Well, in the future you could imagine that instead of programming a game, you can just generate each individual frame on the fly at 60fps. You could be playing 2D Mario and then the game could have him morph into 3D and take off into space or something. You could also generate any software or OS frontend on the fly really, if you can make it so the AI can keep track of your data and make it consistent enough to be usable. Does this have positive or negative value? I don't know.
Re:Genie 2: A large-scale foundation world model by golol on Wednesday December 04, 2024 @12:44PM (#42319904)
Do you want household androids? Because this kind of stuff is on the level of research a bery large step towards that. Think as it as ab example where we can make a model understand a lot of physical common sense stuff, which is the goal for robotics right now.
Re:Genie 2: A large-scale foundation world model by JTyQZSnP3cQGa8B on Wednesday December 04, 2024 @02:20PM (#42320987)
I don't understand how that is relevant. I certainly would not want household androids unless I'm completely disabled.
Re:Genie 2: A large-scale foundation world model by theshackleford on Wednesday December 04, 2024 @05:29PM (#42322767)
> I certainly would not want household androids unless I'm completely disabled.

That's nice. I'm not completely disabled, but I am disabled, and I very much would appreciate them, as my capability to do things over the longer term is very much not going to go in the direction of improving. As it is, there are a lot of things I now rely on people for, that at one time, I did not.

Whilst I recognise its probably not going to happen in a time span that is useful to me, I do wish it could, so that I could be less of a burden on those around me, and maintain a relative level of independence.

Re:Genie 2: A large-scale foundation world model by furyofantares on Thursday December 05, 2024 @01:10AM (#42325429)
A lot of people are disabled right now!

Unless you have a young/quick death, there's a really good chance you will be, too.

Re:Genie 2: A large-scale foundation world model by throwaway2037 on Thursday December 05, 2024 @03:01AM (#42325998)
No trolling: You wouldn't want robots to mow your lawn, maybe do the boring bits of cooking (prep! endless stirring!), clean your house, wash your clothes? Man, sign me up!
Re:Genie 2: A large-scale foundation world model by suddenlybananas on Wednesday December 04, 2024 @12:48PM (#42319945)
This is really not the avenue for house-hold robots. Interacting with the actual physical world is very different from creating a video game.
Re:Genie 2: A large-scale foundation world model by sangnoir on Wednesday December 04, 2024 @01:03PM (#42320120)
> Interacting with the actual physical world is very different from creating a video game

The major difference being the former scales very poorly for generating training data compared to the latter. Genie 2 is not even a video game and has worse fidelity that video games, the upside is it probably scales even better than video games for generating training scenarios. If you want androids in teal life, Genie 2 (or similar systems) is how you bootstrap the agent AI. The training pipeline will be: raw video -> Genie 2 -> game engine with rules -> physical robot

Re:Genie 2: A large-scale foundation world model by youoy on Wednesday December 04, 2024 @02:12PM (#42320886)
> The training pipeline will be: raw video -> Genie 2 -> game engine with rules -> physical robot

One of those arrows is not like the others

Re:Genie 2: A large-scale foundation world model by sangnoir on Wednesday December 04, 2024 @03:32PM (#42321737)
The final step is an oversimplification: purpose-built simulator -> deconstructed robot on a lab workbench -> controlled space -> "real world" with constraints -> real world

Any model would have to succeed in one stage before it can proceed to the next one.

Re:Genie 2: A large-scale foundation world model by adverbly on Wednesday December 04, 2024 @03:49PM (#42321896)
At the risk of sounding repetitive, one of those arrows is not like the others.
Re:Genie 2: A large-scale foundation world model by sangnoir on Wednesday December 04, 2024 @04:07PM (#42322057)
...and?
Re:Genie 2: A large-scale foundation world model by mosdl on Wednesday December 04, 2024 @04:49PM (#42322400)
How does turning an image into a game help with robots? Robots don't need to guess what they can't see, they would have sensors to tell them exactly what is there (like a self driving car).
Re:Genie 2: A large-scale foundation world model by sangnoir on Wednesday December 04, 2024 @05:48PM (#42322908)
> Robots don't need to guess what they can't see, they would have sensors to tell them exactly what is there (like a self driving car).

Self driving vars have cameras as part of their sensor suite, and have models to make sense of sensor data. Video will help with perception and classification (understanding the world) with no agency needed. Game-playing will help with planning, execution, and evaluation. Both functions are necessary, and those that come after rely on earlier capabilities

Re:Genie 2: A large-scale foundation world model by Chilko on Wednesday December 04, 2024 @05:30PM (#42322777)
I have no expertise in this area, but my assumption is that this could help for a broader sort of object/world permanence for robots - e.g. if something is no longer visible to the robot's sensors (e.g. behind an obstacle, smoke, etc) then it could use a model based on this type of tech to maintain a short-term estimate of its surroundings even when operating blind.
Re:Genie 2: A large-scale foundation world model by falcor84 on Wednesday December 04, 2024 @11:17PM (#42324923)
To be able to plan ahead, robots do absolutely need to plan ahead (read: "guess" or even "imagine") what they might encounter before they sense it. In your self driving car example, for instance, it needs to come up with various scenarios for what might be around the corner ahead of a turn, and assign reasonable probabilities to these scenarios. I absolutely see how a system like this could help with it.

For example, let's say that the car is approaching an intersection, and suddenly sees a puddle on the road to the left getting brighter - a visual world model like this might extrapolate a scenario that the brightness is the result of a car moving towards the intersection assigning this some probability, and signing another probably to a scenario that it's just a flickering headlight, and the car would then decide whether and how much to slow down.

In this example there is a sensor, but it definitely doesn't tell the robot "exactly what is there", and while we could try to write rules about what it should do, the Bitter Lesson tells us it's better to just let it create its own model.

Re:Genie 2: A large-scale foundation world model by j_timberlake on Thursday December 05, 2024 @09:35PM (#42335632)
No, they actually did use a genie-like model to train robots on household chores.

Page 8 of the Genie 1 paper: https://arxiv.org/abs/2402.15391

Re:Genie 2: A large-scale foundation world model by mitthrowaway2 on Wednesday December 04, 2024 @01:18PM (#42320292)
I'm not an expert in this space but I can see the value. It allows an endless loop of generating novel scenarios and evaluating an AI agent's performance within that scenario (for example, "go up the stairs"). A world with one minute of coherence is about enough to evaluate whether the AI's actions were in the right direction or not. When you then want to run an agent on a real task in the real world, with video-input data, you can run the same policy that it learned in dream-world simulation. The real world has coherence, so the AI agent's actions just need to string together well enough minute-by-minute to work toward achieving a goal.

You could use real video games to do this but I guess there'd be a risk of over-fitting; maybe it would learn too precisely what a staircase looks like in Minecraft, but fail to generalize that to the staircase in your home. If they can simulate dream worlds (as well as, presumably, worlds from real photos), then they can train their agents this way.

This would only be training high-level decision policies (ie, WASD inputs). For something like a robot, lower level motor control loops would still be needed to execute those commands.

Of course you could just do your training in the real world directly, because it already has coherence and plenty of environmental variety. But the learning process involves lots of learning from failure, and that would probably be even more expensive than this expensive simulator.

Despite the claims I don't think it does much to help with AI safety. It can help avoid hilarious disasters of an AI-in-training crashing a speedboat onto the riverbank, but I don't think there's much here that helps with the deeper problems of value-alignment. This also seems like an effective way to train robo-killbots who perceive the world as a dreamlike first-person shooter.

Re:Genie 2: A large-scale foundation world model by modeless on Wednesday December 04, 2024 @03:45PM (#42321863)
> It's cool because it's new - but that will quickly wear off, and once that's gone, what's left?

To have this perspective you must believe that this will never get better than it currently is, its limitations will never be fixed, and it will never lead to any other applications. I don't know how people can continue to look at these things with such a lack of imagination given the pace of progress in the field.

Re:Genie 2: A large-scale foundation world model by zamadatix on Wednesday December 04, 2024 @05:30PM (#42322774)
I think the problem is less to do with imagination and more to do with being willing to fail a metric shit ton to find out how, every once in a while, you didn't fail due to some really important and surprising reason you wouldn't have found nearly as quickly only ever going after what you were already certain of.
Re:Genie 2: A large-scale foundation world model by xandrius on Wednesday December 04, 2024 @04:26PM (#42322204)
Nothing is of value until it is.
Re:Genie 2: A large-scale foundation world model by 3abiton on Wednesday December 04, 2024 @05:11PM (#42322599)
This is an incredible start. The potential is immense, yes there arekinks, but in 10 years?
Re:Genie 2: A large-scale foundation world model by rlupi on Thursday December 05, 2024 @01:29PM (#42330973)
We, humans, use dreams for consolidating memory and information recall, process emotions and rehearse feelings in different imagined contexts, mental housekeeping / pruning away partial or unnecessary information, replay recent events to review and analyze, etc.

Dreaming and sleeping is incredibly expensive, we spend 33% of our "availability" on average asleep.

This kind of work is a step toward building similar tools for general AI agents (IMHO).

Re:Genie 2: A large-scale foundation world model by araes on Thursday December 05, 2024 @04:38PM (#42333082)
Ruining the video game industry. The implication the technology is plausible takes away all further interest in having anything except "end game content."

All motivation to make further games is removed because now "somebody" can spool out a 3D adventure game instantly with a line of text. It implies you'll waste a year of your time, and just before release, out pops the dramatically better AI product to steal away all further business and reduce all time you've spent to meaningless. Everybody then waits indefinitely for the "endgame gear." https://xkcd.com/989/

Exactly like LLMs and image generators almost completely took away all business for normal writers and normal painters, because now all managers want is the AI. Now there's endless articles about how "somebody" prefers "AI" for every task. Now the market won't invest in anything unless it has "AI" in the name. Now people idiotically add "AI" to everything just to have the investment.