Wow, these images are very dreamlike and not just from the flying POV.

There's something fascinating about the familiar but nonsensical details. Who says a CPU can't go on an acid trip?

I think the fact that it has short-term but not long-term temporal coherence is part of what gives it that surreal quality. A range of any five frames looks like a plausible view shift, but over longer timeframes, it looks like details are appearing out of the horizon. Still really cool though.

> familiar and nonsensical

You might like this analysis of the phenomena

https://www.youtube.com/watch?v=0F7XBwFwA-M&vl=en

Really cool. One thing the most boring part of my mind is wondering about when seeing these is related to the legality of stuff like that: many of the drone videos this is trained on are obviously copyrighted, and the system has used them to learn how to come up with novel ones. The same applies to GPT-n, which is trained on massive amounts of copyrighted material, no? Does anyone know anything about this; there must be a couple of dissertations on this topic by now?

A much more salient and important point is generative models relationship to music. Right now, covers are legally protected as derivative works. I predict commercial usage of generative music to avoid royalties within the next decade.

Philosophically, I think that the process of learning - whether by a human or simulacrom - should be protected. That is, if I write a book, I don't need to seek the permission or pay the authors of every book I've ever read. To me, this is the only sane result for a non-dualist materialist who believes neural networks are in principle isomorphic to human minds.

As a matter of law, I believe it will have to be determined on the case law level some point. I'm not a lawyer and this is not legal advice.

Flashbacks to VistaPro on my Amiga.

(Not mine) https://m.youtube.com/watch?v=48KtJtDqkaM

Wow. Every day, we get closer to the eponymous electric dreams of androids. Personally, I can't wait to use something like this in a music video.

There's already a service for that https://wzrd.ai/

Well, I'll be. Combine that with green-screen technology, and I don't really have any excuse to not record myself and set myself to video for my favorite compositions. Thanks for putting this on my radar, friend.

Lots of issues with long-term consistency in these, but I expect now the dataset is out those shouldn't take too long to resolve.

This is the most impressive attempt to tackle temporal consistency that I've come across:

https://arxiv.org/pdf/2007.08509.pdf

https://www.youtube.com/watch?v=rlCh6-2NfSg

Thanks, that's excellent!

Agree RE long term consistency. After 10 frames I suspect what I’m seeing from that point is a journey through the trained model internals. It doesn’t seem to reference back to the original image and very quickly changes to vastly different terrain.

Still impressive!

"very quickly changes to vastly different terrain"

I anticipate potential use-cases ('level/scene generation' in game content creation) where this actually turns out to be the most valuable aspect of the output. I can't imagine that it would be too much of a stretch to 'start with a photo' as a 'seed' and use it to create a 3D world. My only concern would be 'sameyness tendency', in other words, the problem would be that a variety of seed photos of similar kinds of scenes would end up creating worlds that were too similar to one another to make 'still photo to fly-through movie to navigable detailed 3D world' a sufficiently 'creative' source of sufficiently differentiated new game worlds (or of sufficiently differentiated new fly-through movies). I suspect this could all be fixed by 'differentiator algorithms' interfering with the key variables of the 'refinement' process, as well as broadening and continually adding to the range of movies in the training data.

Wow. Huge step forward from competitors and also wow there’s a lot more to do.

looks amazing, can be very useful for procedure generated game world.

This has huge applications in video games! I can't wait for game developers to implement a truly infinite world with this.

For what sort of games?

There's a few ways I see procedural generation as worse than limited [human] curated modelling - you lose the very careful balance, the iterated architecting to enhance game play, you also lose the commonality of experience across players, and if the game is often played you lose an aspect of the player's ability to become expert at the game.

It's natural that game companies will use lots of procedural/ML generation of game aspects; but it's probably going to be "if you have a hammer, everything looks like a nail".

Television had a certain manner of aiding societal cohesion when we only had a few channels (3 when I was little). That meant lots of commonality of experience. It seems [some] games could go the same way - players getting such vast differences of experience that we lose some of that shared experience that binds people together.

Ultimately, a future of us all living in our own entirely unique VR realty seems like some version of hell.

God damn, John Carmack was right, someday we’ll be able to take footage from TV shoes and just walk around in scenes in VR.

Wow.

I am guessing this will be used in Google Maps Street View?

Why? Maps has actual data to show. Fake data isn't needed or helpful.

Reminds me of weird dreams I’ve had.

Enhance.