Many of the arguments claiming that Moore's law would end were based on the physical properties of light in lithography and the material characteristics of silicon substrate and gate oxide interactions. These are issues that people thought were physical limitations, but we found ways around them.
Theoretical max efficiency of PV also relies on assumptions about the physical properties of light in absorption and the materials being used. I'm not saying we know how to get around these limitations right now. I'm saying that we can't predict future innovations that will make the impossible possible.
Theoretical max efficiency of PV is about 40%, in the same way that theoretical max transistor density is one per atom. It's about the characteristics of sunlight as much as anything, and advances in manufacturing may bring us closer to the limit but not beyond it.