On top of this, it's also very difficult to get the ray differentials correct when merging paths, so you end up point-sampling the textures, meaning huge amounts of texture IO.
When it comes to any sort of bounce, forward raytracing is painful, anything that helps is good.
Most renderers don't take into account much cache coherency of textures at all, which makes me think you work for Disney?
It's rare that we actually have glass/metal objects with lights in/around them such that bi-directional / VCM actually makes sense - even for stuff like eye caustics we've found that uni-directional does a pretty good job. And other situations like car headlights behind glass with metal reflectors behind the light, just turn transmission visibility off for the glass part, and yes, it's not fully-accurate (in terms of refraction and light leak lines), but we're generally using IES profiles for stuff like this so we get accurate light spread patterns anyway.
Well, they do in that camera rays generally (and light rays in bi-directional/VCM) end up using the higher mipmap levels of textures, so you're reading a lot more data for these samples, hence pushing stuff out of cache much more: we've seen this with PRMan 19/20 in RIS: using incremental can have a 3x slowdown in some cases compared to non-incremental, as the camera rays are much more coherent per-bucket in non-incremental, so the highest level mipmaps are kept in cache much more. With incremental, you're only sending the bucket size number of samples and equivalent texture reads for the camera rays, with secondary bounces generally using much smaller/lower mipmap tiles for the texture request (and you can get away with box filtering these in 95% of the cases), then moving on to the next bucket, which will probably need completely different high-level mipmap tiles for its camera rays. With texture IO often being the bottleneck in VFX rendering, this is a huge issue.
Nope, still in London for a bit, then off to NZ...
So you're next in line to try to pull the sword from the Manuka stone?
It is something that people are getting to work in the end but it is a consistently painful disaster.
So basically forward path tracing is not adequate. That doesn't mean it can't be forced to work through huge render farms, hacks, compositor painting and who knows what else.
The state of the art keeps improving but I've never heard any call the switch to forward path tracing a painful disaster, quite the opposite.
Forward raytracing as it currently stands is awful to use at the moment on a large scale for final motioned blurred images that can be sold to a client, but because it is simpler it is still better than alternatives. That is because getting the same results out of (REYES) renderman was something left to a handful of gurus and fanatics.
Now we have a state where renders take 8 hours per frame per character per pass but people still like it better. So be it, but that is still very painful.
This is %100 not true. Forward path tracing is not adequate for anything except for direct illumination. Any enclosed space that has most of it's illumination from some sort of bounce is not feasible without some sort of illumination caching. Bidirectional path tracing and VCM do not carry any addition complexity from an artist's point of view. Also avoiding SDS paths is not always something that artists can simply avoid, and any illumination that is missing is room for improvement.
Do you have links to Solid Angle's patents? Renderman actually samples emissive geometry exceptionally well.