But it is really like that. It depends on the groundwork that's there, technical capabilities and tooling. Once that's in place it only takes a lucky guy to pluck the discovery out of the air.

Consider dual-space methods in crystallography. Herb Hauptman is credited with the discovery. My undergraduate advisor obliquely mentioned that he had done something similar in 1986 - obtain atom positions from direct methods, then do a few cycles of tangent refinement from the highest peaks. It reduces map noise considerably.

Computers were simply too slow at that time to use this as a general method of solving 1000-atom structures, so the approach remained an afterthought. But the possibilities were noted once the computing power was there.