News for hackers. Stuff that matters.

Ask HN: How do you manage your long running processes?

I have an application with several moving parts (solr, rabbitmq, celeryd, plus a few more custom python scripts) and was wondering if there are good ways to manage these. By "manage", I mean a simple way to register processes, execute a start command that starts all of them up in a particular order, and then can shut them down gracefully, too.

Of course, I could write my own master script, create and store my own PIDs etc. just wondering if people had some tools they use that they could recommend.

( 5 comments | 5 points | on HN )
Re:Ask HN: How do you manage your long running processes? by ojbyrne on Wednesday June 29, 2011 @07:44PM (#2711846)
I've used daemontools (http://cr.yp.to/daemontools.html) and supervisor (http://supervisord.org/). Supervisor is somewhat more user-friendly, daemontools feels more solid (it has fewer prereqs for starters).
Re:Ask HN: How do you manage your long running processes? by waterside81 on Wednesday June 29, 2011 @08:22PM (#2711945)
Thanks for the recommendation. Took a look at supervisord, got it up & running in < 2 mins.
Re:Ask HN: How do you manage your long running processes? by bartonfink on Wednesday June 29, 2011 @07:29PM (#2711793)
Solaris 10 has a service management system that's basically the master script that you alluded to with a lot of metadata. It's one of the nicer parts of the OS. Of course, it's also on its way out (even the DoD is planning to leave it behind). For your purposes, I'd just write start and stop scripts that read a list of processes from a file. That way, you don't have to worry about changing the order in all your scripts whenever you add or remove a process from your system - just edit your process.order file and you're set.

Also, there are several ways to kill a process by name on a Unix box, and that's preferable to storing your PID's.

Re:Ask HN: How do you manage your long running processes? by sg2342 on Wednesday June 29, 2011 @08:41PM (#2711986)
When i was still doing such things i preferred to use the facilities provided by the Operating System. That is: svc on Solaris, rc on FreeBSD and services on Windows. I never did anything serious with Linux but i suspect there is a way a Linux distribution manages it's services. On Solaris and FreeBSD i found it helpful to always contain the deployed software in packages -- this takes some effort first but saves one from reinventing the wheel all over again (including service management).
Re:Ask HN: How do you manage your long running processes? by maxdemarzi on Tuesday July 05, 2011 @03:12PM (#2731135)
I started using foreman and upstart about two weeks ago.

I use it to run redis, solr, workers, schedulers with one shot.

http://michaelvanrooijen.com/articles/2011/06/08-managing-an...