On the other hand, wasn't GPU mode added much later than CPU mode? Maybe there has been less development time put into it.
The CPU is never faster than the GPU. CPU + CUDA is way faster, but when I use OptiX (RTX) it is unbelievable how fast it is.
The GPU Raytracer just shines here.
The Blender benchmarks: https://opendata.blender.org/
There's quite considerable variation in the device ranking if you look at each scene render time - the GPU is faster on the quicker-to-render (sub 1 min) scenes, but slower on the slower-to-render scenes (1+ min).
I wonder why that is? I see the 3990X is hugely faster than anything at these slow scenes, which is pretty interesting.
I'm not familiar with rendering, but in the deep learning world it is moderately easy to get considerable speed-ups by using multiple GPUs. Is the same possible here?
So it all depends in what is being rendered and how a render core is optimized for it.
Also things like b-tree calculations and other algorithms might influence this.
Maybe that time will come but I am not sure if that will happen soon.
Raytracing is not an ideal match workload wise for a GPU, too much divergence, but there are plenty of workloads where GPUs will have an edge for along time to come.
AMD cards offer comparable compute performance and are sometimes even better. What AMD lacks, though, are ready-to-use software packages like CUDA, cuDNN, cuBLAS, and the like.
NVIDIA basically bought their way into the research community and solidified their strong foothold by keeping CUDA proprietary.
It's fascinating how other companies (Microsoft) get bashed on a regular basis for using the exact same tactics (see Direct3D vs OpenGL on Windows), while people are perfectly fine with NVIDIA doing it...
I assure you, the affected scientists who have to fight tooth and nail for funding are less than impressed that they are dependent on the mercy of a single vendor.
This is really important, and I don't think people quite realise the scale. I was doing a Physics MSc in 2012 and it seemed that almost everybody in computational physics was doing some variant of reimplementing Y with CUDA.
That's not to say it was the wrong tool for many of the jobs – though for some it apparently was, judging by the levels of stress – but I did find it a bit weird how this was an approach to research that pretty much completely ignored vendor lock-in.
Also, year-on-year performance increases for nVidia GPUs are relatively boring, while Threadrippers routinely double in performance. I have just built a 3970X/RX5700XT workstation, and while I have not benchmarked it yet I expect the CPU to smash the GPU for Cycles raytracing.
Also worth noting that CUDA is a proprietary architecture from a single company - targeting it locks you in, and you need to run proprietary blobs to make it work.
All in all, CUDA works well now but I think it's ultimately a dead-end architecture - hugely parallel CPU architectures will eclipse it.