For raytracing it's been used to de-noise images: https://blogs.nvidia.com/blog/2017/05/10/ai-for-ray-tracing/

If you look closely, at around 0:22 it causes some artifacts in the rifle's red light matrix thingy. It could be regular compression artifacts, but they look like denoising artifacts to me.

Could it be just an intentional focus effect?

TIL, thanks. That's a more prosaic answer than I was expecting but clever and makes sense.

How "local" are such techniques? Is the DL network doing things like "this is a car, and this is a patch of it, so it must look smooth"? Or is it more like "this patch is noisy, so let's make it smooth and hope for the best"?

More on the local end. Traditional denoising is edge finding + blur; you find the edges that you want to keep crisp and you blur on both sides of the edges. ML has been really good at somehow replicating and improving upon that result without explicitly building any sort of understanding about what it is that it's denoising.

I don't think this is what they are actually using, but path tracing can be reduced to a really huge sparse sampling problem. This means that given any predictive model of how much paths through the scene will add light to a pixel, you can make that model unbiased by using https://en.wikipedia.org/wiki/Importance_sampling

So a model trained to take the samples you have so far and predict paths that will also connect eye to light sources could hugely speed up rendering. I don't know of any software that does this, so free idea if someone wants to try it.

there was a paper recently caching sparse radiance results and using ML to predict important paths from that. I believe the latest version of Octane might do something like that if you read between the lines of their marketing guff.

They're using Nvidia's denoiser which is based on a pretrained neural network.

Edit: look up "optix denoiser"

Take a look at the youtube channel 2minutepapers [0], it covers various ML/DL papers and their results. The person creating the videos has a background in writing rendering software. He has various videos covering papers that use ML to improve rendering and graphics. Not long ago he created a own paper about teaching an AI the concept of metallic, translucent materials and more [1].

I really recommend checking his channel, videos are short and fun and easy to understand even without ML background. You can really sense his passion and excitement for this topic.

[0] https://www.youtube.com/user/keeroyz [1] https://www.youtube.com/watch?v=6FzVhIV_t3s

Their current raytracer is limited by the available compute power, so it's forced to use as few samples per pixel as possible, which results in a noisy picture.

However, there's a synergy between their on-chip ML cores and their raytracing engine: they use ML cores (which are typically useless in graphic workloads) to de-noise ray-traced images.

To my knowledge, it's a buzzword. That being said, I'm also not an expert, and would like to hear from one to be sure, since it would be interesting af if they actually are using machine learning for Ray tracing