I agree front end is where the majority of the latency occurs. Luckily there are automated solutions out there to tweak performance but my favorite is http://blaze.io because they do all the performance adjustments in real-time and you get the power of a leading CDN.