Re:Heroku XL
by adrianpike on Monday February 03, 2014 @12:47PM
(#7172216)
Not quite. If you keep routing to a blocked backend, any requests that hit that backend will have unacceptable latencies. Since the routing is essentially random, you'll still be fielding about 1/4 of your requests to that backend, so one long request winds up making 1/4 of all your requests have unacceptable latencies.
I don't think that's how it works. Within a single dyno with concurrency of 4 a single request can not block block any requests. It can only block 1 thread, but then the other requests will be processed on the other threads. Routing is only random to dynos, within dynos it is not random.
At the end of the day, there must be some applications that cannot be fixed by increasing concurrency to the limit in a standard dyno. They would not bother implementing XL dynos if this was not the case.
The problem with only 1 concurrent request is that if that request takes long to process then it can block other requests that are queued up in that dyno. Lets say that 1% of the requests take long. If a dyno can process only 1 request concurrently then at each request there is a 0.01 chance that that will block other requests. If however it can process 4 concurrent requests, then the dyno will only be blocked if it gets 4 long requests at the same time. So there is only a 0.01^4 = 0.00000001 chance that a dyno will be blocked.