News for hackers. Stuff that matters.
Re:Money Trees – Rap Genius Response to Heroku by jxf on Sunday February 17, 2013 @09:28PM (#5237814)
To what extent would using something like Amazon's ELB mitigate this sort of issue in a bring-your-own-cloud approach? Completely?

I've been looking at using something like Cloud66 and an ELB to move off of Heroku.

Re:Money Trees – Rap Genius Response to Heroku by jcheng on Sunday February 17, 2013 @09:37PM (#5237841)
I don't think it would help at all, as ELB doesn't intelligently route either.
Re:Money Trees – Rap Genius Response to Heroku by jxf on Sunday February 17, 2013 @09:56PM (#5237903)
My understanding was that you can specify how the "health check" works for each EC2 node, so if you can report that a particular node is having issues, ELB can observe that. But upon further reading, you're right: you can't do anything other than report that a node is "up" or "down", it seems.
Re:Money Trees – Rap Genius Response to Heroku by pilif on Monday February 18, 2013 @12:44PM (#5239464)
well. I'm talking out of my ass here, but I have this idea:

Lets say you set the health check to check an URL of your app that maps to a very, very cheap rack app that does nothing but return some really short string.

Then you set the health check timeout to a very, very low value (like 5-10ms or something).

Now all hosts that don't respond within that low timeout are seen as down with no requests routed to them.

So if that one node that is able to process one request is busy, the health check will fail and the request will be routed elsewhere.

This is a very poor mans solution with some drawbacks:

1) there's still a race condition here between determining that the host is up and sending a request to it, so you might still end up with a request being queued.

2) now you are practically doubling the latency between the load balancer and the app server

3) you are creating quite a bit of load to that small rack app which might have a negative impact on the overall performance.

So in the end, this might be a very bad idea (I didn't think this through fully and I'm not in a position for trying it out), but it might also be a stop-gap measure until the problem is solved for real. Maybe worth trying this out in a staging environment - or with a percentage of hosts.

Re:Money Trees – Rap Genius Response to Heroku by bgentry on Monday February 18, 2013 @01:03PM (#5239561)
ELB health checks have a minimum interval of 2 seconds.

Also, when you hit the threshold for consecutive failures (2 or 3), the ELB immediately closes all connections to that backend without waiting for their responses to finish.

Re:Money Trees – Rap Genius Response to Heroku by 33degrees on Monday February 18, 2013 @01:18PM (#5239632)
The difference is that an EC2 instance is a lot more powerful than a heroku dyno, so you definitely wouldn't be running a single rails instance on each; you'd be running several, using something like passenger or haproxy, something that does do intelligent routing.
Re:Money Trees – Rap Genius Response to Heroku by tibbon on Sunday February 17, 2013 @09:44PM (#5237867)
No? That's unfortunate. Is it random too?
Re:Money Trees – Rap Genius Response to Heroku by pollop on Sunday February 17, 2013 @09:40PM (#5237849)
>off of

You should be looking at improving your English!

Re:Money Trees – Rap Genius Response to Heroku by mathrawka on Sunday February 17, 2013 @09:43PM (#5237863)
Welcome to Hacker News! Please feel free to look over the Guidelines to see how your comments can be improved: http://ycombinator.com/newsguidelines.html