Re:Fast polling using C, memcached, nginx and libevent
by rbranson on Saturday September 25, 2010 @06:04PM
(#1727982)
How does it compare performance-wise to the Netty solution? Is it better at 100k+ connections?
Re:Fast polling using C, memcached, nginx and libevent
by amix on Saturday September 25, 2010 @06:32PM
(#1728034)
Our node.js solution uses around 10x less memory than the Netty solution. It also serves many more req. pr. sec on a lot of open connections. Of course, Netty might have improved since we used it and we might have used it wrongly. Generally my experience was good with Netty and buying a lot of memory is cheap, so Java developers should definitely check it out.
A quick note is that we have implemented a custom java.nio solution and it had similar memory usages like Netty. It was much harder to code, so I would not recommend using java.nio directly, especially since there is little to gain in terms of perfomance/scalability.
Re:Fast polling using C, memcached, nginx and libevent
by jemfinch on Saturday September 25, 2010 @08:36PM
(#1728225)
Did you ever end up testing Varnish in the polling solution?
Re:Fast polling using C, memcached, nginx and libevent
by deepu_256 on Sunday September 26, 2010 @10:05PM
(#1730504)
amix,
i am sure with the suggestions that Zed made and from your own experience of using that in production you would have made modifications to the code.
is there any way you can post the code that you last used in production?
We wanted to do realtime notifications and polling isn't a way to do this. We tried different options and in the end wrote our own comet solution using JBoss Netty. Working with JBoss Netty was really great, but we hit perfomance limits on lots of open connections (100.000+).
At that time node.js was released and I did some experiments with it and found it it had great perfomance and very low memory footprint. I also had an university course on the V8 VM (by the actual Google team :)) so I was pretty interested in V8 and node.js at that time. I experimented more with this and eventually we rewrote our comet solution to node.js
Fast forward today we are still running node.js and processing many millions notifications each day. I would recommend digging into node if you are interested in building a scaleable notification system. Our comet implementation for node.js is under 1 KLOC.
Other posts you might be interested in:
* http://amix.dk/blog/post/19484#Comet-with-node-js-and-V8
* http://amix.dk/blog/post/19456#Plurk-Comet-Handling-of-100-0...
* http://amix.dk/blog/post/19489#Comet-long-polling-for-all-br...