Using this now for anything like a commercial game is way past the point of diminishing returns. In their paper the SNR between their solution and default Unity’s blur shader is a couple percent while the computational costs are extremely steep.
At 512x512 resolution their fast solution adds 3ms to render time of each frame using, if I’m reading it right, 4 GPUs, which is a drop in 10fps. That’s mind boggling slow for real-time graphics, especially VR which is even more resolution and frame rate sensitive.
If instead of using the default Unity dof blur you really focused on tuning a blur algorithm, which no one really bothers with because it’s not worth the effort, you could probably make up much of the difference.
But really once you have eye tracking, status quo dof blurring will be more than good enough.
The case they mention where someone would be in VR all day makes sense, but let’s all hope we never reach that dystopic vision.
Don't most modern real-time DOF techniques rely on some sort of G-buffer with primitive IDs? In the article it explicitly says they wanted to just work with RGBZ so that existing titles can just be post-process blurred.
Finally, I think the time required is probably closer to 50ms per frame per GPU. If you assume that the content was running at about 60 fps (16ms/frame) on a single GPU, and linearly sped up via magic to 240 fps (4 and change ms), then the delta to 60 fps is 12 ms across four GPUs or about 48 ms on a single GPU.
[1] https://ai.googleblog.com/2018/11/learning-to-predict-depth-...
> These fast but inaccurate methods of creating “game blur” ran counter to Half Dome’s mission, which is to faithfully reproduce the way light falls on the human retina.
... but traditional Z-based blur is no less faithful than their overall "whole screen at single shifting plane of focus depending on gaze" approach. All of computer graphics is "more to do with cinematography than realism" anyway, realism is nice but if you have to choose between "looks realistic" and "looks good" you go for "looks good" every time.
Also, as others have mentioned, getting sufficient resolution for really high quality VR basically requires foveated rendering, at which point the bits that you're blurring are, by definition, not what you're looking at (since you're rendering at lower resolution outside of the fovea) and a blur algorithm that needs four GPUs to run in realtime is a complete waste of resources.
Edit: Watched the video. Their 'Circle of Confusion' map is literally just 'focus_Z - pixel_Z'. I really don't see what deep learning adds here.