As time goes on you keep accumulating these details in your head. Then you just think about the low-level stuff the same way you think about the higher-level stuff, like algorithmic complexity, code structure, etc.

It does get tedious if you need to get something out of the door fast, but have to shave off another 30% of processing time per unit of work. Honestly, I just try to partition the working set and throw more hardware at the problem in these scenarios.