I would say building from source is a horrible idea if you want to scale. For us we had some sweet custom built from source setups that were fast as hell, but once we started putting more servers online it was a hassle to set each one up identically. Better to go with a standard install and be able to reproduce it easily - because no matter what you efficiency gains by compiling are, if you need a new server you will need a new server.

And if the answer is "visualized it" - then compiling is a waste of your time to being with. Better to run native from packages than virtualized from source.