"I've seen arguments that some people desperately need every last bit of performance, and even a few cycles inside a loop is the difference between a viable product and failure. Assuming that's true, then they should be crafting assembly code by hand, or they should be writing a custom code generator with domain-specific knowledge built-in."

Well, I work in the games industry and that "last bit of performance" is quite frequently the difference between having a 60fps game and something that doesn't quite hit the vsync and looks a lot worse. Seriously, this is one of the very few industries where I can say with a straight face "don't do context switching because the 5 microsecond delay is too much" and "a missed L2 cache hit costs us 50 cycles, and it a major problem here".