https://moalquraishi.wordpress.com/2018/12/09/alphafold-casp...
https://moalquraishi.wordpress.com/2020/12/08/alphafold2-cas...
Yes, progress was (not exactly) coming through tons of tiny incremental advances, right up until an industrial lab got involved - as a side project no less - at which point the academics were blown out of the water.
I did CASP back in 2001 or 2002 using alignment methods and protein threading, nothing special (we did terribly). But I told everybody at the conference that ML would eventually surpass the best human predictions, and people said (correctly) that at the time: we didn't have enough data (3D structures) to train on, we didn't have algorithms that were any good (practical backprop for deep NNs, embeddings, transformers), and there wasn't enough CPU time. The data problem was addressed (by having huge numbers of sequence alignments for most protein structures), the algorithms have obviously gotten better, especially transformers, and DeepMind has a lot of CPU (well, TPU).
So basically my prediction came true when the underlying restrictions were removed (which to me seems "obvious"). The academic community would have done this two years after DM if DM hadn't done it (although their wins probably would be much smaller). DM just got there faster, by exploiting a number of advantages.
None of this really moves anything forward. Being able to predict protein structures of homologous proteins is not the grand challenge problem, and what DM did tells us nothing about the physics of folding. And it doesn't produce results good enough to do structure based drug design.
Really, they should have just put out a PR that said they did well in CASP and would be open sourcing the model, the training data, and a trained model, and then done that six months later.
My take is, there isn't a lot of obvious unexplored ground. Where "obvious" includes many things fresh graduate student would come up thinking about some logical first steps. Possibly also many things what most professors would consider.
However, that does not say anything about unobvious unexplored ground. Also, many professors seem to think about in paper-sized increments. Narratives about blessings of incremental steps to expand the body of knowledge sound suspiciously self-serving to researches who publish incremental steps.
(And given the noise in ML experiments, are the incremental steps incremental or merely steps in random directions?) [1]
One way I have started to think about this: How many of the results one uses everyday are product of small increments? How small the increments were? Is another study really helpful to finding the next the such result? Naive case for anti-incrementalist science is naturally false, as every result is built on something, but I wonder if the prominent narrative is discounting the role of genuine novel insights and ideas to the detriment of science, as people do not try to obtain them. The question "how" is above my paygrade (FYI, I am no longer in academia), but I suspect one place where useful things often have came from people who manage to ask new but interesting and useful questions, which open up the much desired new unexplored territory. Which are good questions nobody in the field seems to ask?
[1] Wasn't there a paper some years back how most of the advances and tricks in neural net structures turned out not very important nor generalizable compared to more traditional models given more compute? Can't find the reference now.
All research is incremental.
Even though in context it appears to speak to the right idea, I just want to make sure the casual reader doesn't emphasize too much on the "modern" part. This idea of groundbreaking research happening out-of-the-blue is nothing but a pop-sci fantasy.
Every groundbreaking idea, whether it be Newton/Leibniz's calculus, Marx's manifesto, or more recently the deep learning revolution led by Bengio/Hinton/LeCun/Schmidhuber, are only groundbreaking in hindsight.
Ideas take time to come into the shape that appeals to the spirit of the times. Sometimes, they win a lottery (e.g. the Hardware lottery [1]).
I don't have a PhD so I can't comment first hand on the experience, but one thing I remember from a conversation with the head of a research institute at a large state university, is that there isn't really a lot of unexplored ground. Modern research is incremental. And those tiny discoveries multiplied by thousands of PhD researchers gradually expand our overall body of knowledge. I hope that doesnt come across as discouraging. It's just that it stuck with me for some reason.