Re:600K concurrent HTTP connections with Clojure and http-kit
by alexkus on Monday January 28, 2013 @05:44AM
(#5127980)
For large number (1M+) of connections (doing websockets or long polling) to replicate the functionality of a COMET server I've been playing with rolling my own TCP handling via libnetfilter_queue as I simple don't need the ~10kB of r/w buffers on each socket.
Linux (as far as I'm aware) doesn't allow tuning of r/w buffer sizes on a per interface basis otherwise I'd have one interface for the COMET server with drastically reduced r/w buffer sizes and the remaining interfaces with 'normal' TCP r/w buffer sizes to ensure the other things running on that host run without problem.
Re:600K concurrent HTTP connections with Clojure and http-kit
by justincormack on Monday January 28, 2013 @03:57AM
(#5127735)
He has 16G of RAM so may have not tuned this, and not noticed.
Most notably, 600k active real TCP connections in the kernel would use somewhere around 6GB of memory assuming an average of ~10kB of memory per socket for the R/W buffers and other data. EDIT: Thats just the server side, double to include the client sockets.
Most attempts I've see at this number of real TCP connections required a lot more tweaking of kernel TCP settings to achieve.