You seem to think that ray tracing as a very limited thing. I guess it stems from the fact that the nomenclature has become a bit murky in recent years. Ray tracing often is used to refer to the initial algorithm developed by Whitted. This has pretty much all these limitations you mention, but it is an obsolete method. What usually gets mixed in with ray tracing in marketing and non-expert discussions is also path tracing and its various variations. These methods are capable of solving the rendering equation exactly for a huge range of scenarios. Unfortunately, some variants deal with certain situations better than with others and there is no overall best algorithm. But you can most certainly trace absolutely perfect soft shadows or complex caustics with these methods. Color spills are a walk in the park even for simple unidirectional path tracing. The main limiting factor right now is that these are all Monte Carlo methods and need to draw high numbers of samples for each pixel to converge. This is why fast and accurate denoisers are such a huge deal at the moment. They are another hack, a last one that ideally fades away as the number of rays per second increases. Beyond that, it "just" comes down to modelling surfaces and materials accurately enough to get a photorealistic image.
As you mention photon mapping: it has pretty severe limitations, notably around the handling of specular paths and the biased artifacts created by the filter kernel (loght leaks, certain SDS-type paths aren't covered). There are some improvements like (A)PPM which converge to unbiased solutions with infinite runtime. At that point, path tracing based methods become more attractive w.r.t. runtime, memory requirements and implementation complexity.
My point was mainly that ray tracing is still a hack like polygon based rasterization is, that's a combination of clever tricks to get something to look good enough with modest resources.
Even ray tracing isn't the real deal and isn't trying to closely simulate reality. You need to use hacks in ray tracing for soft vs sharp shadows, for light patterns on glass objects like wine glasses (caustics), to get illuminated surfaces to spread their colour to other objects (colour bleeding) and more.
Phonton mapping gets closer but is even more expensive:
https://en.wikipedia.org/wiki/Photon_mapping
Phonton mapping is trying to model the path of all photons being emitted from light sources back to the camera vs ray tracing that works backwards by tracing rays out of the camera back to the light sources i.e. ray tracing is an optimisation.