Re:BigPipe: Pipelining web pages for high performance
by liuliu on Friday June 04, 2010 @11:07PM
(#1406261)
With all respect to the people in taobao, but this is different. The whole page (without css, JavaScript and images) in facebook case generated through one http request. In your case, they use Ajax to get content through serveral http requests. IMHO the facebook method is better. It avoid http request overhead and parallized steps as many as possible.
Re:BigPipe: Pipelining web pages for high performance
by piotrSikora on Saturday June 05, 2010 @12:26AM
(#1406341)
You must have missed ngx_echo module, because it's the part that makes this exactly as described in Facebook's BigPipe blog post.
I've prepared simple proof-of-concept configuration for nginx:
http://labs.frickle.com/misc/nginx_bigpipe.conf
As you can notice, every "sub-page" is generated individually. Using presented configuration everything is chunked and flushed, so it will be sent to the client right away. Response on the client side looks like this:
http://labs.frickle.com/misc/nginx_bigpipe.output
DISCLAIMER: I don't know how Taobao is using released modules internally or if they use them in production already (but I know some portals do).
Guys from Taobao (http://www.taobao.com) have open-sourced pretty much everything you need to do that:
- ngx_echo (http://github.com/agentzh/echo-nginx-module) for asynchronous pipelining,
- ngx_drizzle (http://github.com/chaoslawful/drizzle-nginx-module) for fetching data from Drizzle/MySQL/SQLite,
- ngx_postgres (http://labs.frickle.com/nginx_ngx_postgres/) for fetching data from PostgreSQL,
- ngx_rds_json (http://github.com/agentzh/rds-json-nginx-module) for converting database-responses into JSON.
Also, this isn't new concept, at least few Chinese companies I know of use similar rendering process.