The trick is of course that there are several ways to approach this: you can develop more flexible tools, become more independent in the lab (if you have lab work), learn more theory, learn more tools, buy more compute power, pick easy problems etc.
Another angle I have thought about (and learned the hard way), is that there is a big difference between research and development. My topic is closely related to engineering and thus it is sometimes attractive to want to "solve a problem". Unfortunately, many measures of "good research" supposes a relevant application, but "solving a problem" is much more difficult than "understanding a problem". The understanding truly has to come first.
Also, these two pieces of advice (Short learning feedback; focus on understanding not solving), are not guarantees to produce "groundbreaking" research. In the end, the most important product of a PhD education is you and the skills you develop.
* A PhD in Denmark is a bit special, in that it is very short (essentially limited to 3 years) and thus has a slightly different scope.
I think 3-4 years is the norm in most countries where you do a (two-years) master's before you do a PhD.
> try to keep the feedback loops short
Yes, just as it is almost mantra for startups, I agree that this is extremely important as well from my not so smooth experience so far. I think its also what makes DL kind of hard as well, experiments can take a long time to run depending on access to hardware.
I don't have a PhD so I can't comment first hand on the experience, but one thing I remember from a conversation with the head of a research institute at a large state university, is that there isn't really a lot of unexplored ground. Modern research is incremental. And those tiny discoveries multiplied by thousands of PhD researchers gradually expand our overall body of knowledge. I hope that doesnt come across as discouraging. It's just that it stuck with me for some reason.
https://moalquraishi.wordpress.com/2018/12/09/alphafold-casp...
https://moalquraishi.wordpress.com/2020/12/08/alphafold2-cas...
Yes, progress was (not exactly) coming through tons of tiny incremental advances, right up until an industrial lab got involved - as a side project no less - at which point the academics were blown out of the water.
I did CASP back in 2001 or 2002 using alignment methods and protein threading, nothing special (we did terribly). But I told everybody at the conference that ML would eventually surpass the best human predictions, and people said (correctly) that at the time: we didn't have enough data (3D structures) to train on, we didn't have algorithms that were any good (practical backprop for deep NNs, embeddings, transformers), and there wasn't enough CPU time. The data problem was addressed (by having huge numbers of sequence alignments for most protein structures), the algorithms have obviously gotten better, especially transformers, and DeepMind has a lot of CPU (well, TPU).
So basically my prediction came true when the underlying restrictions were removed (which to me seems "obvious"). The academic community would have done this two years after DM if DM hadn't done it (although their wins probably would be much smaller). DM just got there faster, by exploiting a number of advantages.
None of this really moves anything forward. Being able to predict protein structures of homologous proteins is not the grand challenge problem, and what DM did tells us nothing about the physics of folding. And it doesn't produce results good enough to do structure based drug design.
Really, they should have just put out a PR that said they did well in CASP and would be open sourcing the model, the training data, and a trained model, and then done that six months later.
My take is, there isn't a lot of obvious unexplored ground. Where "obvious" includes many things fresh graduate student would come up thinking about some logical first steps. Possibly also many things what most professors would consider.
However, that does not say anything about unobvious unexplored ground. Also, many professors seem to think about in paper-sized increments. Narratives about blessings of incremental steps to expand the body of knowledge sound suspiciously self-serving to researches who publish incremental steps.
(And given the noise in ML experiments, are the incremental steps incremental or merely steps in random directions?) [1]
One way I have started to think about this: How many of the results one uses everyday are product of small increments? How small the increments were? Is another study really helpful to finding the next the such result? Naive case for anti-incrementalist science is naturally false, as every result is built on something, but I wonder if the prominent narrative is discounting the role of genuine novel insights and ideas to the detriment of science, as people do not try to obtain them. The question "how" is above my paygrade (FYI, I am no longer in academia), but I suspect one place where useful things often have came from people who manage to ask new but interesting and useful questions, which open up the much desired new unexplored territory. Which are good questions nobody in the field seems to ask?
[1] Wasn't there a paper some years back how most of the advances and tricks in neural net structures turned out not very important nor generalizable compared to more traditional models given more compute? Can't find the reference now.
All research is incremental.
Even though in context it appears to speak to the right idea, I just want to make sure the casual reader doesn't emphasize too much on the "modern" part. This idea of groundbreaking research happening out-of-the-blue is nothing but a pop-sci fantasy.
Every groundbreaking idea, whether it be Newton/Leibniz's calculus, Marx's manifesto, or more recently the deep learning revolution led by Bengio/Hinton/LeCun/Schmidhuber, are only groundbreaking in hindsight.
Ideas take time to come into the shape that appeals to the spirit of the times. Sometimes, they win a lottery (e.g. the Hardware lottery [1]).
Resource constraint also seems to be a serious thing, looking at a full list of ablation experiments, together with multiple runs to obtain error bars, could well take up to a week. If any of the experiments reveal bad results it could also mean back to the drawing board. It seems one would need multiple readily available GPUs in order to be able to really experiment at a comfortable pace.
I also wonder if choice of topic has an important part, since I was sort of assigned my topic which I just decided to do my best to see through. I feel like its important to find a topic where your reasonably confident that there is large margins of potential improvement due to something like lack of exploration or focus by other people. Similarly it might be a good idea to bail if simple initial experimentation yields bad results, or if you find yourself wrangling with increasingly complicated methods for long times but still no obvious signs of progress.
I'd love to hear some others chip in with their experiences on how best to manage the research process cause I feel like I haven't quite gotten the catch of it yet. Are there any good tricks (in the strategy sense, not fraud) that could make the process easier?