While the above is accurrate I think it's a bit one sided.

Physically Based Rendering has become the dominant approach in VFX because it's a huge simplification and productivity boost. Integrating tons of special purpose hacks into one renderer isn't just hard for the renderer devs, the control parameters exposed to artists become an absolute nightmare. Combing the PBR perspective with raytracing greatly simplifies both sides of this.

Not every game is trying to be the next Far Cry. Ray tracing will be adopted by teams that value that unification and simplification over getting the very last bit of performance possible by a pile of hacks. As the hardware improves, which still seems likely, we'll see the balance point of who makes that call shift in favor of tracing IMO.

I agree that PBR is awesome, but PBR doesn't slow things down, it is just a replacement by less accurate equations (BlinnPhong ambient,diffuse,specular) with better equations (GGX albeido, roughness, metallic, clearcoat, sheen) that are generally the same complexity.

It is sort of neutral on render time for the most part.

In time, I am sure we will be in a raytraced future if GPU trends continue.

Physically based rendering is _not_ the Disney shader and renaming your diffuse maps to albedo because it sounds more sciencey.

It’s about the wholesale switch to a fully path-traced framework with a consistent mathematical foundation as opposed to the layered system of hacks and chained prepasses that prevailed in the early 2000’s

This. You can use PBR concepts in an old school renderman style pipeline, but that's not why it's been a big deal in VFX. With a PBRT style renderer, everything is handled in a single monte carlo framework. And once you get an unbiased renderer that meets your standards, you can implement schemes like irradiance caching or photon mapping that introduce bias, but greatly accelerate rendering (and reduce noise). That optimization can be done once, for everything past and future.

That's why I'm saying the combination of PBR and tracing is a huge productivity boon to both sides, art and tech. It has a performance cost, but if/when hardware can pay that cost in real time, games absolutely will use it.

Worth adding that if we get more hardware support for deep neural networks on the GPU (e.g. tensor cores), training a denoising DNN on the noisy renders can save a lot of computation and fits neatly into the pipeline.

If the ray tracing can be done at or around 1 ray per pixel or less using a network trained to do merge information over multiple frames and upscale we could probably get away with less. Maybe more if we can feed in a depth map, velocity map and a flat shader less rendering or other information to help guide the DNN.

Might even end up faster than current raster renders.

See https://www.chaosgroup.com/blog/experiments-with-v-ray-next-...

With 4K around for corner for regular gamer screens, and top of the line GPUs that cannot handle that at an acceptable speed yet, I think it's a long time before we see that. Maybe when a 200$ GPU can do 4K 60 fps, but until then 5-10% gain is seen as worth chasing.

Every pass doesn't have to be rendered at native resolution, and most aren't. When you're running a game at 4K on your PS4 pro or modern PC, very often the 4K image is being produced using a mix of temporal anti-aliasing (accumulating a full image over time across multiple frames), heuristic upsampling, etc. Many post-processing passes run well below screen resolution even if you're running at 1080p - things like SSAO and even particle effects are typically rendered at a lower resolution.

Also, many modern games have been hitting 60fps at 4k on mid-high tier cards for years - you can hit 60 at 4k with reasonable quality settings in a game like MGS5 or GTA5 on a single 970. If you get a 1080 way more stuff runs great, even at higher settings.

Many modern games have an internal resolution slider as well, so you can set the internal resolution to 80 or 90% and still get your UI rendered at full 4K resolution. If the game has temporal anti-aliasing the difference tends to be hard to spot.

Finally, if you've got a freesync or gsync monitor, a few dips down to 50fps at 4k aren't going to be super noticeable, and the games look great. :)

>It has a performance cost, but if/when hardware can pay that cost in real time, games absolutely will use it.

Which has been said since earl6 2000. But the question is, when? Or if it will ever come?

If you look at road maps we surely dont have that in next 5 years, even unlikely in 10.

Actually PBR is quite a very important workflow innovation in rasterizing engines, see my post above!

That's not really what it's about! It's not about achieving physical correctness or ray tracing. You can have a PBR renderer selectively break energy conservation for artistic reasons. It's not about ray tracing either. Current game engines like Unreal and Unity are considered PBR because they use a roughness/metalness workflow.

PBR about having maps that are easy to understand through physical analogy, and simplifying the workflow so that it's mostly shared across render engines, applications and pipelines. It's about the ability to create texture in Mari or Substance, which use rasterizing engines, and seeing the final render in Arnold look nearly identical.

Non-3D artists should note that “Physically Based Rendering” or PBR is merely a standard for defining materials. It is becoming something of an industry standard in real-time and offline rendering workflows. From a real-time(games) perspective, PBR represents a leap up in realism but from an offline(VFX) perspective, it offers no improvement in realism, and often a downgrade from past norms if implemented with no additional efforts. For someone using offline rendering, [EDIT: As per pxl_fckr’s comment, I must be short on some history of PBR. I stand corrected.]

It actually means a lot more than that in offline VFX. It was the games industry that decided it meant “use the Disney shader”.

I updated my comment. I guess I need to read up on that. I read of so many areas in which PBR stops short in offline contexts, like using constant BRDFs which is a little unnecessary with offline rendering. What am I not understanding there?

I’m not quite sure of what you’re asking exactly here? What do you mean by constant brdfs?

I don't think pure raytracing and ray traced global illumination are going to be adopted by many games if there is still such a giant problem with noise. That is far from a solved problem, even in multi-hour per from visual effects renders.

This paper shows pretty good results from essentially throwing a RNN at it:

[1] http://research.nvidia.com/sites/default/files/publications/...

Where will the 1000 frame fly through and the 10 noisy frames that they use come from? I find that paper very disingenuous when they don't count these things in their filtering time.

Not only that, but they compare everything with "1 sample per pixel" knowing that the SURE filter uses per pixel statistics of variance that require more than one sample per to start working.

I guess with games there is already temporal anti-aliasing (TAA), so that would be an effective denoising technique if one was using noisy ray/path tracing.

That's definitely popular for effects that use ray casting. At SIGGRAPH and GDC over the past couple years, a good fraction of the graphics presentations I went to ended with "and then we added TAA because we could only afford a handful of samples per pixel per frame".

What do you mean by noise ? I know that Path tracing is incremental and producer a lot of noise at first but there is also other algorithme than part tracing. Also there is good denoiser filter that can be used with fragment shader.

>My experience with the game industry, even if you have a better way of doing things, if it takes more CPU/GPU cycles that a hack that achieves basically the same quality, it will not be adopted. It is that simple.

That's not always the case. Physically based materials, tonemapping and many more were widely adopted in game engines because they're easier to get proper and predictable results with. It's the same with most rendering techniques based on real physics, they are usually easier to work with, which is a very real advantage which allows devs to make more realistic environments in less time.

It's the same process that led the non-realtime world to physically based renderers, although it is slower in GPUs because of obvious hardware constraints.

In the realtime performance vs developer time tradeoff, the magnitudes are important... and also, their value.

For a long time, games pushed the graphics envelope, but that doesn't seem the case any more. So perhaps that's less important today, and GPUs have overshot gamer needs.

OTOH, development costs of AAA games are incredibly high. with larger worlds at higher res. Even worse at 4K.

So the question of the magnitudes remains.

2018: the year of raytracing in games, finally?

A data-point is that Imagination has had hardware raytracing for several years now: https://www.imgtec.com/legacy-gpu-cores/ray-tracing/ But without publicised success.

The cost of games is not the render engine, remember that most use ue4 or unity. The cost of games 90% is asset and gameplay and UI/UX development.

By "larger worlds at higher res" I meant higher cost of asset development.

I spend a lot of time doing lighting TD work using non-realtime renderers. Every one of those examples is pretty good for realtime, but also looks soooo wrong! its not anywhere near the same quality as a proper raytracer!

-Thats not what an ao pass should look like. But you shouldnt even need ao if your GI/shadows look good

- Gi should have detail to it, every realtime example is sooooo blurry

-The screen space reflections sometimeslook ok but really depends on your scene, you get all sorts of reflections in the wrong place.

-Yucky DOF without the nice bokeh on highlights and all those edge problems you get with a z-space blur.

The beauty of raytracing with a unified sampler is it makes the algorithm for each of these features you listed incredibly simple and it mixes distributes CPU/GPU time to whats important depending on that part of the image.

scene with lots of motion blur- more primary samples, less samples to gi/shadows/reflections - automatically based on how much gi/shadows you see

scene with lots of gi - more samples in gi, less for reflections etc - automatically

You can have a complex scene with reflections/gi everywhere, and then turn a heavy DOF on and get faster frame times.

I'm not sure it's fair to describe those as hacks. Those techniques are also used for animated movies, which take many CPU-years to render. Those "hacks" are used even when artist have huge amounts of hardware, no real-time constraints, and need extremely high quality.

Techniques other than raytracing have an artistic place, not merely a pragmatic one.

Aren't animated movies pretty much universally ray traced?

No-ish:

https://en.wikipedia.org/wiki/Pixar_RenderMan

These days they're pretty much 100% path-traced. But most animated films out there in the history of 3d animation were rasterized.

The edit window has passed so I'll reply to my own comment with a nifty tidbit. The early days of animated films were particularly fun to watch as someone with a background in computer science, as it was a one upmanship competition between the major animation studios (Pixar and Dreamworks) on technology. Monster's Inc had hair. Shrek had a scene with physically-realistic caustic effects of poured milk (really, go watch it--there's an entirely gratuitous scene where a glass of milk is poured to show that they can. milk as a semi-translucent non-homogenous liquid is VERY hard to render as it turns out.) Finding Nemo had lots of water effects, etc.

The challenge of all of this was doing it using rasterization techniques, or integrating more realistic techniques using path tracing into a standard rasterization pipeline in a way that didn't kill performance. But now render farms a big enough and technology has advanced far enough that they can just path-trace everything.

Pixar didn't use ray tracing much until 2013 with Monster's University.

https://www.theverge.com/2013/6/21/4446606/how-pixar-changed...

Nowadays ray tracing is common, but as recently as five years ago it was rare. Not all high-quality animation is ray traced.

PBR proved that we reached the point where the industry had enough of hacks in some domains.

Everybody in CG knows that raytracing is the graal in that it allows a unique, universal rendering model and that triangle-based techniques are always hacks aimed at approximating a raytracing result (or even better, a radiosity-based algorithm)

We have always surfed on the edge between cramming more millions of triangles in our graphic cards and being able to make a complex calculation per pixel. It is arguable that if hardware manufacturers had taken a different path, GPUs would be more comfortable with raytracing now as it requires a very different architecture (make lookups in a big scene cheap vs matrix operations)

From time to time, big player test the water with raytracing and often are not followed by the crowd of developers afraid to change their ways.

I wish that one day we cross that bridge.

I think you meant to compare rasterization with raytracing ? You can raytrace triangle meshes and you can rasterize non-triangular geometry.

> - Screen space ambient occlusion, if using SAO, is amazing.

Yeah, no. It still looks like "draw a blurry black halo around edges and call it a day" ambient occlusion.

You can combine SSAO with (very coarse) global illumination probes to achieve a very convincing effect.

DOOM (2016) is a good example of this: http://www.adriancourreges.com/blog/2016/09/09/doom-2016-gra...

SSAO has been overused/applied poorly (just like the brown 'bloom'/color grading effect in mid-2000s games), which is why people often dislike it.

There is also hierarchical sao

I really wish i kept a bookmark for an article over at filfre.net, because in it a game dev, from back around the C64 era i believe, lamented how whenever there was a big step up in hardware performance, game developers had a bad habit of getting stuck trying to show how far they could push the hardware while ignoring game mechanics and similar.

Effectively whenever some new piece of hardware hit the market, the quality of games would be rolled back by a number of years as companies released what may well be glorified tech demos.

And as of late i wonder if the pace of GPU development have lead to perhaps the most prolonged period of such tech demos.

Perhaps that is what has allowed the indie scene to thrive as well as it has? I'm ok with that.

Honestly I'm not sure why gamedevs seem to care so much about this kind of thing (marketers on the other hand...). Minecraft was worth $2B and intentionally looked like it was rendered in mode 13h.

On the one hand I agree with you, but in the other there’s Horizon Zero Dawn, Battlefield 1, Tomb Raider, Hellblade: Senua’s Sacrifice, NHL 17 and 18, Journey, the Little Big Planet series, Cities: Skylines, GTA V...

I dunno, I think gaming is starting to come out of that prolonged oooh-look-shiny-effects kind of era and into a pretty creative space.

Cities: Skylines is a good example of artistic direction over raw graphic power. It looks nice because its Nordic design architecture and stylish GUI, despite running in a relatively old version of Unity without many bells and whistles.

> Screen space reflections also work

Screen space reflections are an amazing trick and work really well for _some_ things, like reflective floors, water bodies, or objects the player might be holding near the camera (e.g. a weapon)

Since they can't reflect the back sides of objects, or anything outside the camera frustum (which includes the player model) they can't be used for things like mirrors.

The appearance of the reflections being 'clipped' with objects leaving the frustum as you move your head is also quite jarring on VR.

Well, maybe providing an API with fallback (which let's say incorporates those hacks.) is the path towards incremental adoption. Maybe that is the big news here.

If raytracing is practical you don’t need to deal with the limitations imposed by the tricks used by current renderers. All this faking has a price too.

Maybe all these techniques exist because ray tracing wasn't really possible, so a lot of workarounds were found. If ray tracing becomes mainstream and researchers focus their attention more on that, perhaps some awesome new things will be developed that aren't possible in the existing paradigm.

I was at the GDC talks today. The gist is that this is for filling in the gaps that screen space tricks simply cannot do because the reflected surface is off screen.

I can imagine that raytracing for a few effects will be possible when you switch on "Ultra" settings on desktop games, for the guys who have V100s in their machines.

> a lot of this is already possible in the current renderers at a reduced cost.

Maybe this misses the point of standardizing an API? Isn't there considerable value in having a simple framework that can achieve all those effects, and be shared and communicated easily with others? I'm sure a lot of studios would welcome faster dev times, fewer authoring tools, and fewer geometric limits for their rendering effects.

Also worth considering is that all those specific tricks are very limited, either in applicability or in quality or both. Screen space reflections only work on flat surfaces. PCSS still needs a hand-tuned shadow map. DOF has problems at image space contact points. Etc. Let's not oversell today's workarounds as so good they should be preserved forever.

> even if you have a better way of doing things, if it takes more CPU/GPU cycles that a hack that achieves basically the same quality, it will not be adopted. It is that simple.

That's true for any one specific game on any one specific platform. OTOH, that framing misses the trend: there has been consistent and very strong pressure towards higher fidelity, more realism, better physics, faster processors, higher resolution, and overall larger numbers of everything, ever since the first video game ever made. Ray tracing might not be adopted today, but one thing I will guarantee is that next year the CPU & GPU cycles used for rendering will exceed today, and it'll be even more the year after that. When I think about that, I feel like seeing ray tracing take over in games is inevitable.

> Isn't there considerable value in having a simple framework that can achieve all those effects, and be shared and communicated easily with others?

historically, gfx apis went the opposite direction. fixed function pipeline was replaced by shaders. now we rely on the api basically to do rasterization and little else. i think in ray tracing you would want a similar distillation. so like yes, the ray cast operator might be wanted in the api, but i think computations like importance sampling from the conditional brdf are better left in shaders.

and beginners who just want to get triangles on the screen copy/paste a bunch of code they dont fully understand, like now :)

I think the distillation is there already; if you want conditional BRDF importance sampling in DXR, it goes in the shaders, they're not providing that kind of fixed functionality in the same way that rasterization APIs used to.

By "simple framework" I mean the ability to trace rays, rather than the API specifics. What I had in mind is that by being able to trace rays at all, you can very easily get all the effects on the GGP's feature list, with less effort, wider generality, and higher quality than having to code up the buffering tricks required for things like screen space reflections or percentage closer shadows.

Sadly, I doubt the copy-pasta problem is going away any time soon... I think the trend is that it's getting worse.

Maybe, but it may make it substantially more accessible and easier for the average developer that isn't a domain expert in a dozen areas of graphics rendering technology and doesn't want to use a pre-built engine.

This will make it easier to port existing shaders from high end VFX to real-time, like RenderMan and OSL shaders for sure. Trivial even.

What makes you think people are rushing out to replace all these effects? At first blush this doesn't seemed directly aimed at video games.