Guys from Taobao (http://www.taobao.com) have open-sourced pretty much everything you need to do that:
- ngx_echo (http://github.com/agentzh/echo-nginx-module) for asynchronous pipelining,
- ngx_drizzle (http://github.com/chaoslawful/drizzle-nginx-module) for fetching data from Drizzle/MySQL/SQLite,
- ngx_postgres (http://labs.frickle.com/nginx_ngx_postgres/) for fetching data from PostgreSQL,
- ngx_rds_json (http://github.com/agentzh/rds-json-nginx-module) for converting database-responses into JSON.
Also, this isn't new concept, at least few Chinese companies I know of use similar rendering process.
I've prepared simple proof-of-concept configuration for nginx:
http://labs.frickle.com/misc/nginx_bigpipe.conf
As you can notice, every "sub-page" is generated individually. Using presented configuration everything is chunked and flushed, so it will be sent to the client right away. Response on the client side looks like this:
http://labs.frickle.com/misc/nginx_bigpipe.output
DISCLAIMER: I don't know how Taobao is using released modules internally or if they use them in production already (but I know some portals do).
This does kind of beg the question, why use nginx at all? It provides you with a lot of protection against malformed requests and general fuckery. If you need it you can push the middle layer back into nginx as a module which would be screaming fast.
This reminds me of Heroku who do something like this with their 'routing mesh.' client -> nginx -> routing mesh (erlang) -> thin (ruby app server). The erlang process knows which ec2 instance has the ruby process to serve the request and initiates and is basically a smart proxy. You could quite easily query 6 backends simultaneously for each pagelet and pipe the JSON out to the client then throw in the footer as well.
I think this deserves some experimentation.
nginx + erlang + rails (serving JSON).