In the interest of sharing...this is what has worked for us:

1. We don't have any references to external CSS and JS files. This means there is almost no reason for the browser to stop painting the webpage as soon as it starts getting the document or even fractions of it. This means that the user never gets impatient cause it seems that his device is doing nothing. I know this seems like a huge management hassle in terms of changing things, but we got over that by using a server side helper library that spits out the HTML of all the components based on certain input parameters. That way the HTML of the user interface is even more modular and centralized than CSS etc. Besides, it also makes the making of new pages a craft project of sticking different user interface blocks together.

2. All user-interface images for buttons and icons and all are in image sprites. So, an average page is just 2-3 requests. Also sprites once cached load beautifully. In fact, it also makes for a much more pleasing page load as opposed to some parts of the page coming in and some coming in a little bit later.

3. All background repeating images are make 1px thick and saved as optimized as JPEGS and then made part of the CSS with base64 encoding.

4. Of course we very aggressively cache and reduce DB calls and combine DB calls etc etc.

Having done all this, we find that our pages load...well more or less instantly at least with respect to what matters to an average human. But this instant loading is only on a network where the initial connection time is not relevant. On 2G & 3G even, all the optimizations in the world cant save you from the fact that the device takes a LONG time to just connect as Ilya Grigorik has mentioned in his presentation.