Here is an example: https://www.youtube.com/watch?v=Hjb-AqMD-a4
"The final animated VFX would be added in later; the screens were merely to provide interactive lighting to match the animations." -- https://ascmag.com/articles/the-mandalorian, discussed here: https://news.ycombinator.com/item?id=22378679
I don't think that's correct, the ILM team presented this technology in a video:
https://youtu.be/gUnxzVOs3rk?t=104
(paraphrased) "it was designed to both light the actors and be a background that we can photograph, so that we end up with real-time live pixels in-camera"
Also from the article you linked:
> If the content was created in advance of the shoot, then photographing actors, props and set pieces in front of this wall could create final in-camera visual effects — or “near” finals, with only technical fixes required, and with complete creative confidence in the composition and look of the shots.
I'm not an expert in film terminology but it does not sound by any stretch that this was primarily about lighting.
It was for was shot composition as well; the cinematographer could see how it would end up while shooting. The screens updated to match the camera position in real time (e.g. to show the rest of a spaceship whose door was a physical prop).
I wonder how many GPU's that system had.
This goes into way more technical details: https://ascmag.com/articles/the-Mandalorian Particularly interesting is how the system had ~10 frames of latency, so excessively fast camera turns would show lower quality renders.
Umm, Ray tracing would love to have a word :P
In all seriousness, typical animated frames for big budget films easily take hours or longer _per frame_. It really depends on what looks you're trying to achieve. Game engines have come a long way in terms of realistic graphics with realtime rendering, but it's worth noting that it's still not the same quality as a fully ray traced scene (whether that matters depends on the content I guess)
After it is filmed, they can still go back and touchup the backgrounds. Someday with ray tracing they can do real time finished products, but for now the tech works great at what its intended to do.
btw. Only about 50% of the scenes were made with this setup. rest was traditional ray-tracing.
She's been watching the third season of westworld, and it's kinda scary to me how almost every episode feels more like a game trailer than an actual story...
It is an interesting complaint. Need some time to think about that in details.
The difficulty there is that I think everyone universally acknowledges (?) GOT just fell apart like a car crash in slow motion in the later seasons. It's hard to separate that from the knowledge that they started to depart from the books and were clearly undergoing irreparable problems/pressures by the end of season 5. Don't think we can blame that solely on tech... And it's hard to think about the cgi specifically when the rest is falling apart...
Also one amazing thing was the guy talking about "shooting a 10-hour dawn" – I can imagine a lot of time is wasted trying to recreate a certain time of day. With this, you don't waste that time. You can do as many takes as you need. The sun stays right where it is.
Absolutely mind-boggling.
I watched the whole video on the MAnchurian example, and it sold it obviously for me, but I was only initially thinking of the first demo.
Clearly its hugely beneficial.
What I find interesting is, naughty dog had a bit of a staff turnover for designers and hired film animators (in this case it was a crunch), cgi etc.. Funny how they're now almost cross skillset now. Have to admit, makes me feel older every day.
REF: https://www.eurogamer.net/articles/2020-03-14-crunch-once-ag...
Meanwhile all the dynamic stuff is still fairly primitive IMO. At around 4 minute in the video they briefly mention the water effects. They don't really spend a lot of time on them and for a good reason, they don't look particularly good.
When I was a kid in the 90s I definitely expected future games to look a lot better, but I also expect gameplay and world interaction to progress massively. Fully interactive environments you can interact with like in the real world. You could destroy everything, dig holes, build things, have advanced physics, great AI for NPCs etc...
It saddens me that the AAA video game industry is almost entirely focused on eye candy first and foremost. That being said I concede that I'm clearly in the minority, after all the Uncharted games are generally considered to be good games when I find them incredibly boring.
I hope that now that we can reach near-photorealism in games they'll have to come up with something new to keep pushing the envelope.
And then you wonder why AAA studios focus on graphics?
> That being said I concede that I'm clearly in the minority
You definitely aren't in the minority. r/gaming is full of people racing to be most unimpressed by a good looking game.
All the HN comments pointing out flaws with "don't get me wrong, it looks decent!" could be predicted the second I read the submission title.
Remember when HL2 was announced you had this super fancy physics engine? How objects would bounce and realistic react with each other, how you could stack things and come up with puzzles that would just use the physics engine without hardcoded scripts? HL2 was gorgeous looking but it was more than bump-mapping and complicated light models. You couldn't port HL2 to goldsrc and have it play the same.
Meanwhile almost 2 decades later we're back to fully scripted puzzles, mostly non-interactive environments and AI that's generally limited to a glorified A* algorithm.
This tech demo doesn't look decent, it looks really good, there's no arguing about it.
On the other hand, VR based games are wildly experimenting and iterating because the control interface is so different (while at the same time fairly intuitive in that it maps to our reality better). Superhot in VR (which I have on the Quest) definitely doesn't feel like the same game from the last 25 years. Half-Life Alyx is supposed to be an amazing experience (I haven't had a chance to try it yet, and I won't for a long time likely), but that's not supposed to be because it looks so much better, but because it offers a wildly different and new experience.
So, if you're tired of feeling like you're playing the same old games, try VR. The Quest is probably the cheapest way into this space, but it's not actually cheap at $400, and it will gate you from a lot of the premier experiences available if you have a powerful PC with a good graphics card and VR headset that can interact with it (which the Quest can also do with a cable, so you aren't locked to Quest games, although maybe not at quite the visual quality of some other headsets).
Interestingly, I won an Occulus Go at a meetup about 9 months ago. I almost gave it away, I didn't really know what I had won, I thought it was going to be an upgraded google cardboard.
But man I was surprised at how cool it was. And yeah- everything feels really raw and unpolished and experimental, kind of like the early days of the internet. My biggest gripe is that the games are all kind of vapid and on rails, there are no games where you can kind of go and explore, but still its all quite interesting and has me interested in buying a "proper" VR setup.
Some titles on the Quest are a bit better in that respect, but you still have to research the game to know. I think that's also a factor of how new the space is. It's much harder to allow users to roam where they want and still have the polish allowing them to interact how they want with the environment, it took quite a few years for that to happen in the traditional game space, both because of the effort required to craft that world, and because the engines were still working on abstractions that made it easy to fill those worlds with things that could be interacted with without too much programmer/artist work.
A good example of this is all the ways people have found for break Half-Life Alyx by working around the game expectations. People have found a way to hoard items, which would normally be hard to do because you have to actually hold them, but throwing them in buckets and bringing the bucket with you (for example, hauling around a bucket with 20 grenades in it, when normally you don't have a way to carry them all). If you put too many items in a bucket and pick it up, the physics system slows to a crawl and can crash the came. They designed the game so there weren't too many interactable objects in any one scene (because that makes sense), but the wholly new paradigm means that people were able to easily do crazy stuff to break it.
I've been meaning to pick up Arizona Sunshine for Quest, and that might be along the lines of what you're looking for (but not for Go). The Walking dead game is supposed to come to Quest eventually to (but again, the Go is unlikely), and the reviews of that looks pretty open for movement and exploration.
For instance, a game like Okami on PS2 is far more impressive to me than some 4K tech demo. When it comes down to actually playing a game, I don't give a shit about the polygon count, I give a shit if it's fun to play.
Lately I've been playing this great mountain biking game called Lonely Mountains: Downhill that uses this gorgeous minimalist aesthetic. I'm also playing Trails in the Sky on the PSP. Those graphics just age beautifully.
Sometimes the simpler, the better.
But on the flipside, there's literally thousands of indie games with innovative, interesting gameplay, if not sparkling graphics.
Plus they don't actually explain how it works, how the demo was made and what's the limits of their technology. They only show the good side, so people naturally wants to know the not-so-good side as well.
And it resonates with me. I like eye candy like almost everyone and can appreciate technological marvels in CGI (and currently working with UE4 after hours, creating my own assets, I can appreciate how much work goes into modern game graphics). At the same time, I do feel many games these days try to use high-resolution textures and pretty shaders to paper over incoherent storylines and lack of gameplay depth. I think it's an entirely valid point to make.
But I agree with you that I expected games to be much "better" by 2020, back when I was a kid playing 7th Guest. I guess I couldn't understand back then how much manual, hard-to-scale labor and budget would have to go into story, dialogue, art, acting (plus salaries of A-list movie stars), mo-cap, etc. I expected in-depth scripted NPC behavior, like in 1992's Ultima VII [1], to be extremely commonplace and basic by now, but I obviously didn't understand back then what actual AI and emergent simulation requires (versus manual scripting, and testing of that scripting)
But I do agree with you in general that more advanced simulation is a huge, largely untouched opportunity in games.
If you double the number of triangles, it doesn't take long to get to a clean looking circle - the next time you double it, it doesn't look that much better unless you're zooming way in on it.
So there's two things here, I think:
* There are tons of other improvements, like dynamic lighting. This turns into more of an immersion/ realism thing, and less of a "detail" thing. It's weird when your characters shadow doesn't move the way you expect with the light sources, it's weird when one part of the room is totally dark because a block is just slightly in the way of a light source, and the light isn't bouncing around it.
* With VR, this will all matter way more. You're going to have people taking objects in the game and putting them right up to their face - suddenly the difference between X triangles and 2X triangles is really noticeable again, relative to a 3rd person view on a monitor that's at least a foot or two away from you, where objects are always at a distance. Immersion is an extremely important factor for VR, so optimizing there makes a lot of sense.
So while todays mediums may not demonstrate these wins, they open up possibilities for new mediums.
But still, the graphics in UE5 is stunning. The last time I was stunned by 3D Graphics was Crysis, and that was I think over 10 years ago.
Many games on the Switch look way better than generic AAA games on PS4 and Xbox. Granted the Switch games are still somewhat held back by limited hardware.
Personally I'd take Breath of the Wild over some generic UE4 game any day.
- Realtime dynamic GI makes baking lights unnecessary, increasing iteration time for environment artists.
- Also using this dynamic GI it is possible to create new gameplay mechanics based on dynamic lighting (for example in the demo the roof of the cave falls down and the area becomes lit, making more things visible)
- The new animation system makes developers able to create natural motions that automatically adjust to the environment, which would also save a lot of developer time.
- The demo explanation video mentions that the statue model assets are imported directly from ZBrush without any postprocessing (with the original triangle count, without baking any normal maps/LODs), which also saves time for artists importing their work to Unreal (although the file size costs might probably be a bit high to practically use this in every scenario)
What I'm noticing is there's too much focus on fancy lights, over-the-top postprocessing effects with annoying crap like ambient occlusion, screen space reflections that hog a lot of performance for just reflecting your character in water puddles placed everywhere just to show the effect off, and so on.
And not enough focus on good old textures and polygons. Approaching spherical objects closely still makes the polygons very very obvious. Staring directly at walls still shows how low res textures are.
> You could destroy everything, dig holes, build things
You might be looking for Minecraft! :D
Even CGI water looks typically very fake (except from afar), because we don't really have very good models for water/liquids. That's certainly not the priority for games either.
That said, The Mandalorian has used the Unreal engine to render real-time backgrounds for scenes, so it's good enough for that at least.
And in films they don't need to do real-time, they can take their time to render a scene.
Yes they can render it in-engine, but they can use bespoke character animations which don't have any blending artifacts, and they can put the camera on a rail and tune the assets and particle effects until they are certain every single frame can be rendered in under 16ms. They can hide billboards and other rendering tricks and be certain the camera is never going to hit them at an angle which gives them away.
Real-world game play scenarios are much more unpredictable, and the results are likely to fall well shy of this mark.
I mean... that lighting was really, really good, and I think this is the first triangle-based demo where the surfaces really don't look oddly angular almost anywhere (maybe with exception of the stalactites). But it's a far cry from the hand-tuned look of something prerendered.
Next-gen consoles will have to become more ubiquitous before the previous gen consoles are left out of game releases.
don't get me wrong, the fixed assets look amazing and I look forward to seeing more done with this system
However I could see them adjusting the physics for the demo to ensure the cloth actually fell by her side so we could see it swaying as she climbed.
Though I was totally blown away by the detail of the rock texture and very realistic lighting.
The animation was great, but the graphics were much better, and one can't help but notice the quality difference even though both are awesome works of art.
1. Better real-time global illumination
2. Ultra high poly models
Then they also briefly focused on a few other features, mostly ones that they already shipped. The Lara Croft adventure scenario was an excellent choice to let them show their new work off in a context with many sudden lighting changes, naturalistic assets, arbitrary changes in scene geometry, huge changes in horizon distance and other things that have historically been difficult for real time engines due to their reliance on very CPU intensive pre-processing passes to build static data structures.
I'm sure this is all fixable, but some part of me wonders how much larger game budgets will have to become (or correspondingly, how much less content will be offered) in order to achieve the new standard in production qualities.
- obvious SSAO when hand touched the rock wall
- odd reflections on the bird sculptures
- down-resing of textures and motion blur when she flew
- obvious uncanny valley face (pancake makeup, no subsurface scattering) on face closeup
- dust sprite clouds
were some of the things I noticed on my first watch
but it was still quite pretty. like what I'd expect uncharted 5 on the pc to look like
I think that was intended and programmed into the clip on purpose - not a side-effect of a poor engine.
Covid-19 games.
it's different from e.g. Moana, where skin is kind of marzipan-y, but still recognizably skin-like
Physics engine, character movement, etc. could still be improved.
The specs of the new XBox are slightly higher on paper, but the PS5 is doing some really interesting things in terms of optimization. Basically the XBox is going for high constant clock speeds, and the PS5 is shifting priority between the CPU and GPU components of the SOC so each gets boosted performance when it's most relevant.
Both consoles are doing very interesting things with asset streaming and decompression of assets directly from the SSD to video memory which are going to open up new opportunities not only in visual quality, but in terms of how flexibly game worlds can be designed.
Probably we will have to wait and see how all of this plays out in terms of real-world performance.
edit: Maybe deterministic isn't the correct word. What I mean is that you can design a physics system that you can ensure runs on the PS5 CPU. But then the graphics boys make an upgrade and suddenly some power is diverted to the graphics and the physics system is no longer working correctly. This is still deterministic. But a nightmare to work with when optimizing for hard-real time requirements. The only way this can be done sanely is if the developers can fix this power budget and thus restore "determinism".
'So how does boost work in this case? Put simply, the PlayStation 5 is given a set power budget tied to the thermal limits of the cooling assembly. "It's a completely different paradigm," says Cerny. "Rather than running at constant frequency and letting the power vary based on the workload, we run at essentially constant power and let the frequency vary based on the workload."'
https://www.eurogamer.net/articles/digitalfoundry-2020-plays...
The only way I can see this going well if this balance is fixed by the developer. E.g. the developer specifies I want 60% of the power budget to go to the GPU and 40% to the CPU. If this is handled dynamically by the PS5... oh boy.
Another way to think about it is that very few games are using 100% CPU capacity and 100% CPU capacity at the same time. This model gives the developer a combined compute budget, which can be allocated as required for the task.
Right, but to the GP's point, for those games which do aim to max out both at the same time (or at least get as close to that as possible), this could be a development pain point unless the developer has control over which gets prioritized. Once upon a time this sort of maximization was a common goal for console game development, though I don't know how true that holds nowadays.
But I think this style of hardware also changes the way you make optimization decisions. I.e. in a world where you have a fixed CPU/GPU budget, you might try to think of ways too move work to the CPU if for instance your GPU is saturated. But if the relative capacity is variable, you can just do things in the most efficient way overall, and you just have to make sure you don't exceed your overall budget.
How common is this though in an era where most releases are multi-platform?
So developer can expect it to perform identical to whatever balance they've set between CPU/GPU. That's my understanding at least.
hint_prioritize_cpu_begin()
...
hint_prioritize_cpu_end()
That Unreal 5 Demo was the first time ever I thought this is Hollywood Movie quality GFX ..... ( Apart from the Character ). IT IS STUNNING! And this is done Real time on PS5!
Edit: I am sorry for the tone and block capitals.... I am seriously geeking out.