Once you have a big knob to scale (or automatic scaling), you next need to identify points on the curve where app rearchitecture is needed, like adding a caching tier or moving activities out-of-band. It sounds like Heroku wants to help do this -- which is great.

Alternatively, I wonder if Five Runs-style companies will partner will cloud vendors, figure out your current cloud mapping and give you this kind of analysis.