News for hackers. Stuff that matters.
Re:600K concurrent HTTP connections with Clojure and http-kit by javajosh on Monday January 28, 2013 @01:59AM (#5127520)
Running locally doesn't mean you bypass TCP/IP.
Re:600K concurrent HTTP connections with Clojure and http-kit by mbell on Monday January 28, 2013 @03:54AM (#5127729)
While true, I'm suspect given this setup that some layer isn't just dropping to a unix socket when it notices localhost.

Most notably, 600k active real TCP connections in the kernel would use somewhere around 6GB of memory assuming an average of ~10kB of memory per socket for the R/W buffers and other data. EDIT: Thats just the server side, double to include the client sockets.

Most attempts I've see at this number of real TCP connections required a lot more tweaking of kernel TCP settings to achieve.

Re:600K concurrent HTTP connections with Clojure and http-kit by alexkus on Monday January 28, 2013 @05:44AM (#5127980)
For large number (1M+) of connections (doing websockets or long polling) to replicate the functionality of a COMET server I've been playing with rolling my own TCP handling via libnetfilter_queue as I simple don't need the ~10kB of r/w buffers on each socket.

Linux (as far as I'm aware) doesn't allow tuning of r/w buffer sizes on a per interface basis otherwise I'd have one interface for the COMET server with drastically reduced r/w buffer sizes and the remaining interfaces with 'normal' TCP r/w buffer sizes to ensure the other things running on that host run without problem.

Re:600K concurrent HTTP connections with Clojure and http-kit by justincormack on Monday January 28, 2013 @03:57AM (#5127735)
He has 16G of RAM so may have not tuned this, and not noticed.
Re:600K concurrent HTTP connections with Clojure and http-kit by justincormack on Monday January 28, 2013 @03:55AM (#5127732)
It bypasses checksumming, but no not the whole stack. Although not sure in this case it bypasses checksumming as the interfaces were created as aliases on a physical interface, which may mean it does do checksumming still.
Re:600K concurrent HTTP connections with Clojure and http-kit by voidlogic on Monday January 28, 2013 @03:52PM (#5130564)
Many OS's kernels use their local domain socket codepath/interface for 127.0.0.1 <-> 127.0.0.1 communications, completely bypassing TCP/IP... This is transparent to the application.