Re:Fast polling using C, memached, nginx and libevent
by axod on Wednesday March 25, 2009 @11:17AM
(#531896)
>> "From what I have seen, all comet solutions are very hacky, quirky and very hard to scale. The scaling is especially hard since the whole web-platform is built around the request-response model."
I disagree about this. There's nothing hacky/quirky about comet, and it can scale really well.
Some numbers from a Mibbit backend:
Peak connected users: 4,000
Peak HTTP Req/s: 1,500
HTTP requests per day: 44 million
Packets sent per day: 42 million
Packets recv per day: 51 million
(On a single 1.4GB VPS). Using *OMG* java!
If you're not using comet, that HTTP req figure would be way higher, as a far larger number of HTTP requests would be unuseful (no data available). Wasting time, bandwidth, etc As well as obviously adding latency. With comet, the data is available in most instances as soon as the server has it ready. With standard polling, the user has to wait for the next poll.
Comet is pretty easy to scale once you understand exactly what is going on, and how best to organize things. Mibbit uses a keep-alive long polling XHR setup which works really well. The keep-alive ensures that most of the time, you just have a tcp/ip connection open from server to client, with some HTTP request/response spam every so often, and the actual packets.
Having said that, I wouldn't use any of the open source comet things I've seen out there.
(I do plan to open source Mibbit at some point, which might be useful for people).
Re:Fast polling using C, memached, nginx and libevent
by amix on Wednesday March 25, 2009 @05:58PM
(#532502)
I think Java is a pretty good choice, especially for doing comet or any other problem where performance is required. Today Java's perfomance almost matches the speed of C or C++... I have some time ago played around with using Jetty's continuations and it's pretty impressive (and highly scaleable).
In my next iteration I'll probably try to go with the long polling model and I'll probably explore C and Java.
Re:Fast polling using C, memached, nginx and libevent
by axod on Wednesday March 25, 2009 @06:09PM
(#532517)
Cool :)
Forgot to also mention CPU usage is negligable. Memory can be a challenge with Java, but often solved by reducing object churn, tidying up better etc. The java.nio package really is pretty awesome.
Re:Fast polling using C, memached, nginx and libevent
by lhorie on Wednesday March 25, 2009 @11:53AM
(#531951)
Out of curiosity, what transport are you using? Long polling or long-lived iframes? I find that the former doesn't give me low enough latency for my purposes and the latter tends to get quirky for me when I have a high spiked data throughput.
Re:Fast polling using C, memached, nginx and libevent
by axod on Wednesday March 25, 2009 @12:02PM
(#531965)
long polling XHR. Latency is extremely good if the server software is up to the task. As soon as the server sends an HTTP header and data response, it's there on the XHR response. (Also iframes mean memory leak, and possibly 'loading' spinner. Neither of which you get with XHR).
You can also do some neat optimizations like decide that a latency of 100ms is ok enough, and so delay sending any data for up to 100ms to see if you can piggy back on the next recv request and reduce the HTTP request count by 1. Also in order to batch up sends.
For example:
Without optimization--
0ms Long polling Recv
800ms Send some data
900ms Long polling Recv (After some data was received)
With optimization--
0ms Long polling Recv
(Send queued for up to 100ms)
900ms Data sent, in Long polling Recv request.
Re:Fast polling using C, memached, nginx and libevent
by jeresig on Wednesday March 25, 2009 @04:47PM
(#532409)
"There's nothing hacky/quirky about comet... long polling XHR."
Now, unless I'm misinformed, long polling XHR does not work in Internet Explorer, although you do list it as a supported platform:
http://wiki.mibbit.com/index.php/Platforms
Re:Fast polling using C, memached, nginx and libevent
by axod on Wednesday March 25, 2009 @05:59PM
(#532503)
It's absolutely supported.
IE will wait patiently for data to be received, just like other browsers. It'll do keep-alive also with XHR, so you get that benefit as well. You can also remove some HTTP headers in js to get some bandwidth optimization :)
One possibility you may be thinking of is XHR reuse, which is a pain in IE, as far as I remember there's a couple of bugs if you try to reuse the same XHR object. There's hoops to jump through depending on the state or something, and it's just more reliable to not reuse the object on IE.
At the moment, I create a new XHR object for each request, to be sure there aren't issues with IE (I believe XHR reuse isn't an issue on other browsers).
Re:Fast polling using C, memached, nginx and libevent
by blasdel on Wednesday March 25, 2009 @04:33PM
(#532386)
What connection scheduling mechanism are you using? Is it non-blocking asynchronous, or are you using threads? What's the resource overhead per open connection?
Re:Fast polling using C, memached, nginx and libevent
by axod on Wednesday March 25, 2009 @05:58PM
(#532501)
non blocking async. Single thread. There's a bit of memory used for buffers, but nothing major. It's been up to around 14k connections total so far in the real world, more in testing.
Re:Fast polling using C, memached, nginx and libevent
by liuliu on Wednesday March 25, 2009 @08:40AM
(#531733)
If it is a poll server, why not write a lite script of lua inside nginx or just a plugin of nginx? That would take full features of nginx and I guess the result would be better.
Re:Fast polling using C, memached, nginx and libevent
by staunch on Wednesday March 25, 2009 @11:10AM
(#531889)
I think modifying the nginx memcached module would have been ideal. Running a new server that does almost nothing does not seem like a great way to go.
Re:Fast polling using C, memached, nginx and libevent
by lhorie on Thursday March 26, 2009 @10:04AM
(#533447)
I don't have data available right now, but I have done quite a bit of research on server push technologies for one of my projects. It just comes down to bandwidth: regular polling consumes bandwidth (and the related client and server resources) even when there's no actual data being sent.
I disagree about this. There's nothing hacky/quirky about comet, and it can scale really well. Some numbers from a Mibbit backend:
If you're not using comet, that HTTP req figure would be way higher, as a far larger number of HTTP requests would be unuseful (no data available). Wasting time, bandwidth, etc As well as obviously adding latency. With comet, the data is available in most instances as soon as the server has it ready. With standard polling, the user has to wait for the next poll.Comet is pretty easy to scale once you understand exactly what is going on, and how best to organize things. Mibbit uses a keep-alive long polling XHR setup which works really well. The keep-alive ensures that most of the time, you just have a tcp/ip connection open from server to client, with some HTTP request/response spam every so often, and the actual packets.
Having said that, I wouldn't use any of the open source comet things I've seen out there.
(I do plan to open source Mibbit at some point, which might be useful for people).
In my next iteration I'll probably try to go with the long polling model and I'll probably explore C and Java.
Thanks for sharing your architectural notes.
Forgot to also mention CPU usage is negligable. Memory can be a challenge with Java, but often solved by reducing object churn, tidying up better etc. The java.nio package really is pretty awesome.
You can also do some neat optimizations like decide that a latency of 100ms is ok enough, and so delay sending any data for up to 100ms to see if you can piggy back on the next recv request and reduce the HTTP request count by 1. Also in order to batch up sends. For example:
Now, unless I'm misinformed, long polling XHR does not work in Internet Explorer, although you do list it as a supported platform: http://wiki.mibbit.com/index.php/Platforms
Could you expand on this at all?
IE will wait patiently for data to be received, just like other browsers. It'll do keep-alive also with XHR, so you get that benefit as well. You can also remove some HTTP headers in js to get some bandwidth optimization :)
One possibility you may be thinking of is XHR reuse, which is a pain in IE, as far as I remember there's a couple of bugs if you try to reuse the same XHR object. There's hoops to jump through depending on the state or something, and it's just more reliable to not reuse the object on IE.
At the moment, I create a new XHR object for each request, to be sure there aren't issues with IE (I believe XHR reuse isn't an issue on other browsers).
-Ben
Any data or story to back that argument? Just curious.