Compiling ASM and even C is very fast these days. No generics, small compilation units etc... Even with optims it's usually a non issue in my experience.

For a datapoint it takes me ~2 minutes to build a reasonably fully featured ARM linux kernel with a cold cache on a 5 year old i5. wc counts 1399 CC, 65 AS and 411 LD calls. And of course incremental rebuilds are only a fraction of that.