One must be careful when setting connection limits like this. A lot of people still use proxy servers and with modern browsers it quite easy to hit 20 concurrent connections per IP address.
It's important to understand connlimit will cause people to queue, not to get blocked, and if 20 people are connecting all at the very exact millisecond from the same ip, well it cannot hurt to queue them for server stability.
HTTP Keep-alive connections play a big role in the number of connections as well so this should be considered when choosing the number.
You can use the optional second part the keepalive_timeout setting in nginx to send timeout hints to modern browsers, ie.
keepalive_timeout 10 10;
Bad idea. This way you're actively increasing the latency of your site. This way, for each asset that has to be fetched you're forcing the client to open a new connection, which can add more than 150 ms of delay per item (thanks to the three way TCP handshake).
What I would suggest is setting the KeepAlive timeout to a value that could handle each individual page-load. This way all the page elements will have a chance to use the connections that has been already opened.
It's great for an ajax-heavy site, especially when it's all behind SSL. Using a CustomLog I log total request time, from connection time to the request has been served (conditional log when the request is handled by the backend) and I can see it's halved since I could use the threaded Apache.
Currently I have 100 threads per Apache process, and ~20 of them handling 2000 idle connections. I'm sure this can be tweaked some more.
Apaches manages SSL etc. and just proxies to my application servers.
There's also an event-based Apache module which I haven't tried.
There might be some magic value at which KeepAlive will be helpful during non-peak periods without crippling the server during peak periods, but for a well-engineered site, the extra 10ms delay per request shouldn't be a big enough deal to warrant risking a full-on site outage later on.
Also, this has already been discussed to death on HN: http://www.hnsearch.com/search#request/all&q=keepalive&#...
I was fairly careful with the test(s), and the 10ms difference seemed to be consistent. So that's odd. I need to investigate that further.
2. set vm.swappiness = 0 to make sure crippling hard drive swap doesn't start until it absolutely has to
3. Use IPTABLES xt_connlimit to make sure people aren't abusing connections, even by accident - no client should have more than 20 connections to port 80, maybe even as low as 5 if your server is under a "friendly" ddos. If you are reverse proxying to apache, connlimit is a MUST.