Re:Watchy: a distributed system monitoring solution
by dkhenry on Monday July 14, 2014 @08:06AM
(#8030489)
I really like that people are valuing monitoring and making tools to do it, but I mean how is this better then SNMP? It doesn't look like it gives any more information then the prTable of the UCd-SNMP-MIB (http://www.net-snmp.org/docs/mibs/ucdavis.html)
Also if you used snmp instead of rolling your own agent you would get security and encryption (SNMPv3) and the ability to use one of any number of existing monitoring tools.
Re:Watchy: a distributed system monitoring solution
by spo81rty on Saturday July 12, 2014 @06:33PM
(#8025981)
Monitoring background services can be difficult due to changing process ids. If anyone is looking for a commercial product that does this, I would recommend checking out Stackify. We do server monitoring, app monitoring, errors, logs, metrics, etc all in one app. http://www.stackify.com
Re:Watchy: a distributed system monitoring solution
by buro9 on Sunday July 13, 2014 @06:54AM
(#8027087)
Sounds good, so I checked it out. The one thing I'm looking for I cannot see though.
For application monitoring, and to be able to answer questions like "Why is my app slow?" I only want one thing.
I want a GANTT chart showing time spent by the application handling an individual request/job, highlighting the critical path and showing concurrency when it occurs (across systems).
i.e. give me the Network pane in Firebug or Chrome Dev Tools, but where each row on the chart is a visit to a service and/or machine (tell me which machine and which service).
Re:Watchy: a distributed system monitoring solution
by xorcist on Saturday July 12, 2014 @06:55PM
(#8026029)
That's a very strange comment. I've never had that problem across several years, platforms and monitoring systems.
Perhaps for a fast respawning process I can see potential problems monitoring it, but that is usually indicative of a problem in itself and something you generally watch for.
Re:Watchy: a distributed system monitoring solution
by spo81rty on Saturday July 12, 2014 @07:39PM
(#8026121)
A good example of this is Windows performance counters since they are process ID based. That is more complicated than knowing if a service named X is running. It's tough if you want to know more than just if the app is running.
Re:Watchy: a distributed system monitoring solution
by gk1 on Saturday July 12, 2014 @10:03PM
(#8026398)
While we're mentioning paid alternatives, I'd add Scalyr (http://www.scalyr.com) to the list. Does all of the above, with plans based on log volume instead of number of servers.
Re:Watchy: a distributed system monitoring solution
by cbsmith on Saturday July 12, 2014 @08:02PM
(#8026158)
TCP's spoof protections are fairly limited and weak. If you are worried about being spoofed, address it at layer-7. I wouldn't let spoofing concerns drive my choice of layer-3 protocol.
Re:Watchy: a distributed system monitoring solution
by mobiplayer on Sunday July 13, 2014 @10:22AM
(#8027391)
I personally hope you chose IP as a layer-3 protocol.
You might want to decide between TCP and UDP for layer-4, though, but as gingerlime pointed out it is stupidly easy to successfully spoof UDP comms, there's not even need to have a MITM situation. Same goes for ICMP.
Re:Watchy: a distributed system monitoring solution
by cbsmith on Sunday July 13, 2014 @11:30AM
(#8027596)
> I personally hope you chose IP as a layer-3 protocol.
ROTFL! Whoopsie. ;-)
> You might want to decide between TCP and UDP for layer-4, though, but as gingerlime pointed out it is stupidly easy to successfully spoof UDP comms, there's not even need to have a MITM situation. Same goes for ICMP.
Yes, but TCP's protections are quite weak. If you are really concerned about spoofing, you are much better off handling that with cryptographic solutions.
Re:Watchy: a distributed system monitoring solution
by gingerlime on Sunday July 13, 2014 @06:30AM
(#8027056)
Spoofing a TCP source address is still pretty hard compared to UDP. And security should always be applied in layers, so if your risk profile is such, then relying on source IP protection alone might not be enough.
Re:Watchy: a distributed system monitoring solution
by gingerlime on Sunday July 13, 2014 @06:35AM
(#8027063)
encryption != authentication. Even if you encrypt the datagrams, what stops someone from replaying them? and there are other attack vectors. You probably want to use something like HMAC in this case. But it all depends on your risk profile and environment.
I just highlighted one potential attack vector that I didn't see being addressed explicitly, and which might have implications.
If the tool explicitly provided message authentication for the UDP datagrams, I would probably not have made the same comment. I think the security fundamentals were missing, let alone more advanced layers of protection.
Re:Watchy: a distributed system monitoring solution
by graycat on Saturday July 12, 2014 @04:08PM
(#8025655)
What about false alarm rate? Do we know what it is? Can we adjust it? Since Watchy is for distributed systems, can it do well detecting problems that have 'distributed' causes?
Re:Watchy: a distributed system monitoring solution
by marykaichini on Wednesday July 16, 2014 @09:32AM
(#8041878)
Is it only for smart phones? Do you have the usual version of the tool? I am in search of a website monitoring tool and I am ready to listen to any options. At the moment I like a lot Anturis, which is all in one monitoring tool, but they have just developed iOS app and it is still fresh.
I am ready to consider anything else.
Re:Watchy: a distributed system monitoring solution
by kbaker on Saturday July 12, 2014 @03:10PM
(#8025484)
At least looking at its current featureset it looks much closer like it wants to be Logstash or Apache Flume, but with an included frontend. They note the deep inspiration from statsd as well.
Also all communication is over UDP at the moment, which could be bad for poor quality links, but probably OK for most servers.
The main difference from just a log shipper is including a frontend web service to look at data, and a library to link in the app to deliver stats instead of using a more standard protocol like syslog / collectd / ganglia, etc. and having the server parse / ship it.
So it's more of an integrated solution than combining different logging pieces.
Also if you used snmp instead of rolling your own agent you would get security and encryption (SNMPv3) and the ability to use one of any number of existing monitoring tools.
For application monitoring, and to be able to answer questions like "Why is my app slow?" I only want one thing.
I want a GANTT chart showing time spent by the application handling an individual request/job, highlighting the critical path and showing concurrency when it occurs (across systems).
i.e. give me the Network pane in Firebug or Chrome Dev Tools, but where each row on the chart is a visit to a service and/or machine (tell me which machine and which service).
Perhaps for a fast respawning process I can see potential problems monitoring it, but that is usually indicative of a problem in itself and something you generally watch for.
It's much easier to spoof a UDP packet from an arbitrary source-IP address than it is with TCP.
It gives a fairly easy opportunity for attackers to fake all your service metrics, make it look like everything is Up/Down etc...
You might want to decide between TCP and UDP for layer-4, though, but as gingerlime pointed out it is stupidly easy to successfully spoof UDP comms, there's not even need to have a MITM situation. Same goes for ICMP.
ROTFL! Whoopsie. ;-)
> You might want to decide between TCP and UDP for layer-4, though, but as gingerlime pointed out it is stupidly easy to successfully spoof UDP comms, there's not even need to have a MITM situation. Same goes for ICMP.
Yes, but TCP's protections are quite weak. If you are really concerned about spoofing, you are much better off handling that with cryptographic solutions.
I just highlighted one potential attack vector that I didn't see being addressed explicitly, and which might have implications.
If the tool explicitly provided message authentication for the UDP datagrams, I would probably not have made the same comment. I think the security fundamentals were missing, let alone more advanced layers of protection.
http://vigil-app.com
Also all communication is over UDP at the moment, which could be bad for poor quality links, but probably OK for most servers.
The main difference from just a log shipper is including a frontend web service to look at data, and a library to link in the app to deliver stats instead of using a more standard protocol like syslog / collectd / ganglia, etc. and having the server parse / ship it.
So it's more of an integrated solution than combining different logging pieces.