News for hackers. Stuff that matters.
Re:HireFire - The Heroku Worker Manager by freerobby on Monday April 11, 2011 @01:13PM (#2433311)
Nice work, glad to see heroku API gems getting some love.

I wrote a similar gem back in January and have had good luck with it: https://github.com/freerobby/middle_management

It's fully configurable via environment variables, so you can make changes without redeploying. It also lets you specify a minimum number of workers (even if there are zero jobs) so that you don't have to wait for workers to spin up if you don't want to.

Re:HireFire - The Heroku Worker Manager by meskyanichi on Monday April 11, 2011 @01:58PM (#2433510)
Thanks. I like your approach of not having to deploy to change the min/max value of workers. Though it's hard to implement (I think) if you're working with a worker/ratio configuration like HireFire. Currently HireFire also supports the Mongoid ODM. I'm looking in to implementing the Resque worker as well so people can use HireFire with Resque with Redis on Heroku as well.

Thanks for your input!

Re:HireFire - The Heroku Worker Manager by freerobby on Monday April 11, 2011 @02:06PM (#2433547)
If I could fork your code and get it working with environment variables, would you be interested in pulling that in? (Not sure if you made a deliberate decision to avoid that).

MiddleManagement works great for my needs, but if these libraries serve the same use case(s), I'd rather converge towards one library than have things spread out.

One other thing you may want to add is a retry loop for the heroku API calls. Every now and then, the heroku gem will throw a RestClient::ServiceUnavailable: 503 Service Unavailable exception. In the worst case, I'm not sure if this is best handled in the gem or at the application layer; however it's definitely worth retrying a couple times if you can avoid the blowup completely. I got some feedback from the Boston.rb group on handling this if you're interested: http://groups.google.com/group/boston-rubygroup/browse_threa...

Re:HireFire - The Heroku Worker Manager by meskyanichi on Monday April 11, 2011 @08:02PM (#2435082)
Thanks for the heads up. Seems like a good idea to catch the 503 errors. Also, for now I'll leave all the configuration defined in pure Ruby. Maybe eventually I'll consider using ENV var's to handle that.

Thanks for your feedback!