Then someone (Newton, Darwin, Einstein) comes along and assembles a new framework based on new observations/theories that do a better job of explaining things. Eg the switch from creationism to evolution, or from newtonian physics to relativity.
Here are some more examples:
https://en.wikipedia.org/wiki/Paradigm_shift#Examples_of_par...
I guess what I'm trying to say is that the incentives are strongly biased to what can be easily seen as a increment on what has been done before, and that this is biased against work that may draw on what has been done before but is presenting a fairly different view to what is currently accepted.
Could Darwin have presented his view as an incremental extension of some existing view? (I am aware there were existing theories of evolution, but there weren't any like Darwin's or Wallace's to be built upon).
Einstein didn't get funding, i.e. an academic position. He did his initial groundbreaking work while having a day job in a patent office. He published his four groundbreaking papers in 1905 and was offered a lecturer position only on 1908.
Tenure with reduced teaching responsibilities is the way big thinking gets funded these days.
EDIT: I would also argue that developments like the printing press and the world wide web would have been difficult to justify as incremental extensions of existing work.
I'm not saying these developments came from nowhere, nor that they didn't build on previous work. I'm specifically arguing against the idea that it's fruitful to just focus on work that would be seen as incremental improvements on existing work.