Fast and unstable at high concurrency.

Do you have any details about that?

Most of the problems stem from the thread pool - and most of my personal observations come from using both in production.

Up to a certain high load, Varnish works well, then all hell breaks loose - Nginx in comparison was much more stable and used less resources, had a higher max throughput, and when it hit the max - (assuming it filenode limit was high enough) would still work..just slowly. Varnish just died.

There is this sad idea that "threads are bad" and that event is the cat's pajamas. But even nginx admits that there are situations where threads make sense, hence their semi-recent addition to adding them.

I encourage people to read http://capriccio.cs.berkeley.edu/pubs/threads-hotos-2003.pdf. Yeah, it's old, but so is the 10k Concurrency paper which is still touted as the reason why people should use nginx instead of anything else. Threads will only get better.

Sounds like you had a gigantic malloc and used too much memory and let Linux kill it.

http://stackoverflow.com/questions/16693677/why-isnt-varnish...

If you really need a large malloc, I highly recommend running Varnish on FreeBSD instead of Linux as the Linux kernels VM system is horrible when memory is overcommitted.

Interesting - curious about what the actual failures were as I've never seen failures like that with fairly large Varnish caches. nginx wasn't an option for us until they made it handle the HTTP Vary header correctly so I don't have recent comparative experience with it as a cache.