The setup they're using is an nginx serving 8 unicorn workers with a backlog of 256. They then throw requests at that with a concurrency of 20. DB pool is 256 too. It seems to me quite likely that the unicorn queue fills up very quickly and it starts rejecting requests, which could be an error. It's hard to see how a maximum of 8 workers would ever get close to the 256 available DB connections.
At first glance the unicorn setup is totally inadequate for the amount of traffic being thrown at it. The first thing to do would be massively increase both the number of workers and the backlog, otherwise this almost instantly turns into an overflowing request queue and literally millions of errors.
There's no denying, though, that this kind of request flood is not exactly rails' strong point and if you're expecting massive numbers of fairly simple requests you're probably better off with something else.
Realistically, I doubt many humans can distinguish between 1ms and 100-200ms. response times.
However, the purpose of this project is not actually to measure how quickly platforms and frameworks can execute fundamental/trivial operations. Rather, these tasks are a proxy for real-world applications. Across the board, we can reasonably assume that real applications will perform 10x, 50x, 100x, or even slower than these tests. The question is, where does that put your application? If your application runs 100x slower than your platform/framework, does that put your application's response time at 200ms or 2,000ms?
That's a difference users do notice.
If anything to me this data confirms that Rails is an amazing tool because not only do you get to develop quickly, but you also get pretty good average latency (or at least the potential depending on what you add to your app in terms of 3rd party libraries). And what Rails isn't good at is throughput, which is almost never a problem for an early stage company.
Working at a startup it's a huge success if I ever have to handle a lot of connections to my app, but today and everyday, I want fast response times on a page load.