Re:MongoDB's lead developer: Foursquare outage post mortem
by chewbranca on Thursday October 07, 2010 @07:52PM
(#1770008)
I work on a team of 2 where I'm responsible for a handful of servers. I chimed in because I'm in a similar position, I've been looking for a monitoring solution for a while now. Things like nagios and zenoss are over kill, but lack of time has prevented me from finding an ideal solution. That said, I keep htop open and running at all times, and its saved my ass on more than one occasion. I say htop because of the color coding it provides, if things start going red it attracts my attention.
[ Parent
| on HN ]
Re:MongoDB's lead developer: Foursquare outage post mortem
by sjs on Thursday October 07, 2010 @08:30PM
(#1770083)
Nagios is pretty nice. It's dead easy to write custom monitors and clients are everywhere, there's even a Firefox extension. It requires a bit of learning to get going with but it's not so bad and the pay-off is big.
That said I'm looking at monit too. I hear it's quite nice and has less of a learning curve.
Re:MongoDB's lead developer: Foursquare outage post mortem
by hartror on Thursday October 07, 2010 @11:02PM
(#1770417)
I'll put a vote in for Zabbix as a good option, it has saved us more times than I can count.
Re:MongoDB's lead developer: Foursquare outage post mortem
by riffraff on Friday October 08, 2010 @02:42AM
(#1770860)
zabbix user here, and it works nicely. But man, it is painful to use.
Re:MongoDB's lead developer: Foursquare outage post mortem
by dedward on Saturday October 09, 2010 @01:21PM
(#1775359)
Stop thinking about it and set it up. There isn't anything to think about. Nagios (I like using it through groundwork) are certainly not overkill... it's thinking like that that is what gets you in trouble in the first place.
Get that tool in place NOW while you are small, so it's there when you get bigger... so you can hire people to watch it while you move on to other things.
You will always have a lack of time, and always be busy. Spend a weekend putting up Nagios, setting up alerts over SMS or XMPP or Twitter or whatever you want, and then move on. This is fundamental to systems management.
Dont' over think it - just get nagios/cacti (again, try groundwork) up and running and graph EVERYTHING you can... yuo'll be very, very glad you did.
Re:MongoDB's lead developer: Foursquare outage post mortem
by rhizome on Friday October 08, 2010 @02:04AM
(#1770786)
Nagios/Cacti/etc. are indeed overkill for a few servers, but Munin is dead-simple and has notifications:
http://munin-monitoring.org/wiki/HowToContact
Re:MongoDB's lead developer: Foursquare outage post mortem
by mootothemax on Friday October 08, 2010 @06:28AM
(#1771158)
Second vote for Munin, is simple, quick to install and set up and covers all the basics.