I doubt many people of the time thought that the photovoltaic effect was compelling evidence that hinted at a whole new layer of physics beneath the classical theories. Only hindsight is 20/20. There a lot of loose threads in modern physics that seem unrelated or even inconsequential, but that might ultimately turn out to be revolutionary. So like you said, us being close or very far from a total understanding are both plausible.