News for hackers. Stuff that matters.
Re:Fast polling using C, memcached, nginx and libevent by amix on Saturday September 25, 2010 @03:54PM (#1727663)
It should be noted that this post was written over one year ago and we are not using polling any longer, but comet solution.

We wanted to do realtime notifications and polling isn't a way to do this. We tried different options and in the end wrote our own comet solution using JBoss Netty. Working with JBoss Netty was really great, but we hit perfomance limits on lots of open connections (100.000+).

At that time node.js was released and I did some experiments with it and found it it had great perfomance and very low memory footprint. I also had an university course on the V8 VM (by the actual Google team :)) so I was pretty interested in V8 and node.js at that time. I experimented more with this and eventually we rewrote our comet solution to node.js

Fast forward today we are still running node.js and processing many millions notifications each day. I would recommend digging into node if you are interested in building a scaleable notification system. Our comet implementation for node.js is under 1 KLOC.

Other posts you might be interested in:

* http://amix.dk/blog/post/19484#Comet-with-node-js-and-V8

* http://amix.dk/blog/post/19456#Plurk-Comet-Handling-of-100-0...

* http://amix.dk/blog/post/19489#Comet-long-polling-for-all-br...

Re:Fast polling using C, memcached, nginx and libevent by rbranson on Saturday September 25, 2010 @06:04PM (#1727982)
How does it compare performance-wise to the Netty solution? Is it better at 100k+ connections?
Re:Fast polling using C, memcached, nginx and libevent by amix on Saturday September 25, 2010 @06:32PM (#1728034)
Our node.js solution uses around 10x less memory than the Netty solution. It also serves many more req. pr. sec on a lot of open connections. Of course, Netty might have improved since we used it and we might have used it wrongly. Generally my experience was good with Netty and buying a lot of memory is cheap, so Java developers should definitely check it out.

A quick note is that we have implemented a custom java.nio solution and it had similar memory usages like Netty. It was much harder to code, so I would not recommend using java.nio directly, especially since there is little to gain in terms of perfomance/scalability.

Re:Fast polling using C, memcached, nginx and libevent by jemfinch on Saturday September 25, 2010 @08:36PM (#1728225)
Did you ever end up testing Varnish in the polling solution?
Re:Fast polling using C, memcached, nginx and libevent by deepu_256 on Sunday September 26, 2010 @10:05PM (#1730504)
amix, i am sure with the suggestions that Zed made and from your own experience of using that in production you would have made modifications to the code.

     is there any way you can post the code that you last used in production?
Re:Fast polling using C, memcached, nginx and libevent by bsaunder on Saturday September 25, 2010 @07:20PM (#1728113)
Thanks for taking the time to provide an update.
Re:Fast polling using C, memcached, nginx and libevent by xtacy on Saturday September 25, 2010 @01:34PM (#1727392)
What is the bottleneck here? I am surprised that the CPU utilisation is just 2%, which means, either the disk or the network is the bottleneck. Both these aren't discussed. And of course, one shouldn't forget the fact that if CPU is the bottleneck, we need "true" concurrency and there should be threads on every single core. :-)
Re:Fast polling using C, memcached, nginx and libevent by amix on Saturday September 25, 2010 @06:44PM (#1728057)
Hitting a non-blocking C stack is much better than hitting a blocking Python stack - especially when you have tons of clients hammering with polling requests. So the optimization was to reduce the load on our Python servers by handling most of the polling requests in a C based server.
Re:Fast polling using C, memcached, nginx and libevent by aiurtourist on Saturday September 25, 2010 @01:46PM (#1727418)
Check the second comment on that post. I'm glad someone else looked at that guy's C and thought "Whoooaaa, oh no."

How can we take posts like this seriously?

Re:Fast polling using C, memcached, nginx and libevent by rgrieselhuber on Saturday September 25, 2010 @03:35PM (#1727617)
Zed Shaw's response was more helpful than yours.

He provided recommendations to fix the mistakes instead of just dissing the whole thing. If you want people to write better code, share what you know.

Makes me wish Zed had a "Learn C the Hard Way" to complement what he's doing with Python.

Re:Fast polling using C, memcached, nginx and libevent by vimalg2 on Sunday September 26, 2010 @01:44PM (#1729550)
I think thats a really great idea.

I sure would like a 'hard way' courseware for C. (in addition to my trusty copy of K&R C for reference)

Re:Fast polling using C, memcached, nginx and libevent by bsaunder on Saturday September 25, 2010 @02:03PM (#1727448)
Just because of some novice mistakes, you want to discount the whole post?

Thank you for calling attention to the second comment.

Re:Fast polling using C, memcached, nginx and libevent by jrockway on Saturday September 25, 2010 @05:20PM (#1727899)
The mistakes are harmful. When you mess up your string handling in Perl or Python, you end up with double-encoded utf8, or something. When you mess up your string handling in C, other people on the Internet get a shell on your computer.

C is serious business.

(Incidentally, I've written event-driven programs with libev in both C and Perl. My load is usually so low that doing it all in Perl does not affect the number of requests I can handle, nor does it impact latency. The IO is the bottleneck; the article even says it only uses 2% CPU. Why risk insecurity when you don't know C and it doesn't even improve throughput?)

Re:Fast polling using C, memcached, nginx and libevent by jcsalterego on Saturday September 25, 2010 @02:57PM (#1727548)
It was nice to see zedshaw reviewing C code in the comments there.
Re:Fast polling using C, memcached, nginx and libevent by mahmud on Sunday September 26, 2010 @01:11PM (#1729498)
Zed is a code-review maniac. I have seen his work with a friend's startup. Would highly recommend that man for his insight and eye for quality code.
Re:Fast polling using C, memcached, nginx and libevent by deepu_256 on Sunday September 26, 2010 @09:56PM (#1730481)
Yup. i learnt a lot by seeing Zed's code.

But it was very nice indeed that he takes time to review code voluntarily on blog comments.

Re:Fast polling using C, memcached, nginx and libevent by BenSchaechter on Saturday September 25, 2010 @02:10PM (#1727462)
I'm interested to hear other peoples' thoughts on comet versus polling. I've done polling solutions before and am in the process of setting up a comet based solution.

In my opinion, having fewer requests to the server is better. But if someone can combat me on this, I'd really be interested in hearing another voice.