Ask HN: What load balancing software do you use ?
I'm looking into software load balancers and so far the 4 that seem to standout are:
- Nginx (http://nginx.net/) - Pound (http://www.apsis.ch/pound/) - HAProxy (http://haproxy.1wt.eu/) - Perlbal (http://www.danga.com/perlbal/)
Does anybody have experience using any of these, in live production environments (under reasonably heavy usage), and if so what pros and cons do you see with them ?
(
37 comments
| 47 points
| on HN
)
[internet] <-> [Nginx] <-> [HAProxy] <-> [app servers]
Nginx is a great webserver, but isn't a good load balancer. You can install a patch that improves the balancer - http://brainspl.at/articles/2007/11/09/a-fair-proxy-balancer... - but it still isn't as nice as HAProxy
With HAProxy the status of the system is visible. For the largest site I use HAProxy on I keep my status page public - http://userscripts.org/haproxy - but it isn't required.
HAProxy is particularly good for rails since you can say each app server can only have 1 request at any time. This makes requests queue at the HAProxy layer, so if an app server has a request that takes a extra long time you don't have requests waiting for that app server to finish - instead you wait for the next available app server in a FIFO queue.
Combining HAProxy with munin gives great stats for tuning your system - whereas just nginx with the patch had no visibility into where bottlenecks might be.
I
- Do you run the two products on the same physical machine ?
- What does Nginx do that HAProxy doesn't ? (i.e. why not just stick with HAProxy ?)
- I noticed that Nginx has some weighting options for load balancing (see http://wiki.codemongers.com/NginxLoadBalanceExample). Are these just not sophisticated enough for your needs ?
I use Nginx for things like serving from memcached (someone else commented on that), rewrite rules, redirects, logging, ...
Also Nginx is good because I have 5 different virtual hosts on the box, so I can proxy to different app servers based on hostname/location.
Here is my commented nginx config for userscripts.org:
Here is my haproxy config for userscripts.org: Nginx does have weighting options, but I was unable to get the affect of having only 1 request hitting an app server at any time (it could be possible, but switching to HAProxy was easy, and in a month I've not had a second of down-time, and I restart all the app servers every hour via log rotate) Not having request queue at the app server level helped, combined with HAProxy pinging for health between requests has lead to faster requests, as queueing at the app server level leads to delays if you get behind a slow request.One question that will help decide how things are structured is SSL. If you need it, putting haproxy up front gives you a bit of trouble -- haproxy doesn't support SSL (though it will blindly forward SSL at layer 4 if you want). So you'd need stunnel, nginx, Apache, or something else up front to decode.
http://barry.wordpress.com/2008/04/28/load-balancer-update/
I wouldn't recommend DNS round robin for load balancing. (We did it for a while, many problems and flaws in the approach.)
http://www.igvita.com/2008/02/11/nginx-and-memcached-a-400-b...
http://leavingcorporate.com/2008/03/03/session-based-load-ba...
Both have worked pretty well so far. As anortherjesse said, there's not a lot of feedback, but it's done everything I need so far.
One drawback is that you can't take system out of rotation as quickly, but this can be somewhat mitigated by setting very low TTL (60 seconds).
If you do go the multiple A record route, do so in combination with one of the load balancers you asked about, and ideally with IP failover. But probably this is a lot more complexity than you want.
There simply is no way to guarantee that a user will remain on one host for the duration of a session, if you use round-robin DNS (or even semi-deterministic DNS like F5 or GSS).
Your best efforts will be thwarted by load-balanced browsing proxies that 'optimise' the user experience by querying primary and secondary DNS, then distributing your HTTP requests across multiple servers. HTH
PerlBal looks really cool, as being written in Perl means it has some of the same kinds of flexibility that Squid has (a good reason for Squid is that you can write your own balancing algorithm in any language you like in a redirector script--I always used Perl, or Python when I was working with the Zope guys--so, you can actually do crazy stuff like choose the right server based on keeping them "primed" for the content users are asking for based on URL, or you can use destination URL hashing and achieve the same effect even if you have millions of URLs). Squid also has experimental support for ESI (Edge Side Includes) which is pretty awesome...build a page from disparate and wholly unrelated servers using a simple templating system, and caching them. I don't think any other Open Source product out there has ESI (experimental or otherwise).
In my experience, balancing works better with smarts, thus my big long tirade about Squid (which is historically about the smartest option...though not necessarily the fastest...but fast enough).
LVS has its place, and I've used it when the situation is appropriate, it just isn't all that common that you really need to be able to saturate a Gb pipe, and the installation, maintenance, and configuration burden is much higher for a kernel level tool.
So, perhaps I should have been careful not to sound so dismissive of a tool that I like (and I'll note that you can find posts by me on the LVS mailing list from the first few months of the projects existence). My previous company even sponsored development of several related management tools.
It's probably not as fast as Nginx, but I haven't found our load balancers to be a bottleneck. In fact, we've been doing load balancing on a pair of really, really wimpy servers (Celeron CPU and 512MB of RAM) running Apache on OpenBSD for about a half-dozen different apps for the last year, and never seen the average load climb up over about 0.5, even while handling upwards of 300 requests per second.
Works great.
Daniel
However, I recently noticed a memory leak. We use healthchecks on our production machines, which means a consistent rate of hits every few seconds, 24/7. After about 3 months, Pound had chewed up 1.7GB of RAM, which caused memory usage alerts in our monitors.
Not a big deal -- you can always restart the process -- but I'm still evaluating alternatives.
For our $WORK applications, we just have an apache that proxies to the backend FastCGI apps. We don't have a ton of load, so that works fine. (We might be switching to nginx, which is much simpler than Apache for this use case.)
I definitely prefer Nginx. It seems much faster, and has definitely given fewer headaches. It seems like an issue with Pound crops up every few weeks. My Pound config file is about 600 lines long now, and it is starting to get unmaintainable.
We used HAProxy and Heartbeat2 to provide fully redundant load balancers. On our smaller sites, we actually run the HAProxy and Heartbeat2 on two of the web servers for that cluster, so you don't need dedicated hardware if you don't want it. If you do this, and you're on softlayer, I'd recommend sending the back-channel traffic along the internal network to avoid using 2X your real bandwidth.
You can read about how to set it up here:
http://www.digitalsanctuary.com/tech-blog/debian/13-steps-to...
It was very smooth to setup with debian, but my first experience setting it up was with redhat and that was tortuous