The justification in TFA smell like premature optimization. A nominal bottleneck is just that-nominal. It’s a good thing to keep in your back pocket but you generally don’t want to expend extra effort optimizing something until you have to.

I suspect that the cost of long compilation times (preLLM even) was actually quite high and the author is discounting/excluding those other factors and focusing on LLM feedback loops primarily as their justification.

As an aside short feedback loops are important, but the article ignores one major reason why: humans learn most effectively when the time between action and feedback is reduced because the contents of our working memory degrades over time (take the explanation with a grain of salt). LLM has no such restriction, so the only thing that matters is that the codegen can keep up with the demands from the actual product.