The problem with only 1 concurrent request is that if that request takes long to process then it can block other requests that are queued up in that dyno. Lets say that 1% of the requests take long. If a dyno can process only 1 request concurrently then at each request there is a 0.01 chance that that will block other requests. If however it can process 4 concurrent requests, then the dyno will only be blocked if it gets 4 long requests at the same time. So there is only a 0.01^4 = 0.00000001 chance that a dyno will be blocked.
There are workarounds, which require some legwork and forsight.
Reducing the pitfalls inherit in deploying a web app is generally what Heroku is selling. It is why they are making the big bucks. It would have been nice if Heroku would have been able to find a solution and implement smarter routing so that this just became a non-issue.
There are systems that do benefit from other kinds of routing but there is very little margin to favor non-concurrent request handling in mostly state-less components like web servers. There are lots of responses on the web that take a much deeper look at the heroku router if you care to search. The conclusion is not so clear that Heroku is doing it wrong.
That's not a change; that's how routing on Cedar has always worked.
- http://wiki.nginx.org/LoadBalanceExample
- http://serverfault.com/questions/537450/what-algorithm-does-...
Routing requests to random dyno is definitely not the smartest routing algorithm because it kind of defeats the purpose of having a routing layer. If you are not aware of the issue between Heroku and Rap Genius, I will think it is important enough for anybody who considers Heroku to be aware of. Here is the URL with relevant information: http://news.rapgenius.com/James-somers-herokus-ugly-secret-a...