News for hackers. Stuff that matters.
Re:Money Trees – Rap Genius Response to Heroku by btilly on Monday February 18, 2013 @12:13PM (#5239286)
I suggested an intermediate solution to them.

The front line router picks a random appropriate dyno, then hands off the request to the router responsible. That router does intelligent routing to the dynos it is responsible for.

As long as you cluster dynos for a given app behind a set of routers, this essentially solves the problem on Bamboo for the cost of an extra step at the start of each request.

Re:Money Trees – Rap Genius Response to Heroku by mononcqc on Monday February 18, 2013 @12:30PM (#5239383)
What's unclear for this kind of scheme is what impact it has on stacks or dynos that can actually take concurrent requests. There's the distinct possibility that a concurrent server will see its average time get worse under such circumstances.

People simulate their stack quite quickly, but do not necessarily simulate what might happen to other ones.

Otherwise, it also means that you'll possibly have twice the data flying around your network due to the overhead of routing from one layer to the other one. Effects of these may be hard to predict.

It's also difficult to know what the impact would be of a router that handles a lot of traffic for an app being cycled or replaced, or crashing. Does this mean you create single point of failure for apps? Do you go for redundancy, failovers, sharding or anything else to fix this? What about netsplits between units across layers? Random distribution has the advantage of generally being pretty reliable across all of these scenarios.

Re:Money Trees – Rap Genius Response to Heroku by btilly on Monday February 18, 2013 @01:01PM (#5239555)
Bamboo is explicitly for stacks that can't take concurrent requests.

The data issues are not so bad. You just do as load balancers do and do the handoff in the initial TCP handshake. Only a small fraction of packets go through the first router.

As for the rest, the architectural issues are similar to the ones that Heroku was dealing with in 2009, except that each router has a constrained set of dynos it is responsible for. You would have standard health checks for routers, and the ability to migrate dynos from one router to another. There are issues to solve, but they are fairly reasonable.