Re:Riemann – A network monitoring system
by MayMuncher on Tuesday January 19, 2016 @09:27AM
(#10930911)
I don't like it since the master basically does a curl on each server to get the info, so it doesn't work behind a firewall without tons of issues
[ Parent
| on HN ]
Re:Riemann – A network monitoring system
by zimpenfish on Tuesday January 19, 2016 @09:35AM
(#10930958)
That's the reason I haven't bothered with Prometheus - pull systems make no sense to me since you have to configure a single place with perfect* knowledge of your system rather than just pushing local knowledge to a collector.
(Ok, you do need some knowledge at the parent if you want to raise alerts but you'd need that anyway.)
Re:Riemann – A network monitoring system
by bbrazil on Tuesday January 19, 2016 @10:01AM
(#10931151)
You need a place with all that information anyway, otherwise how do you alert on something being missing?
If you can't easily do that with your existing infrastructure, you should fix that first. I've written about this at http://www.robustperception.io/you-look-good-have-you-lost-m...
Re:Riemann – A network monitoring system
by zimpenfish on Tuesday January 19, 2016 @10:13AM
(#10931249)
If a stream of data stops appearing, then you can alert. You don't need to pre-configure the existence of that stream (although if you might never get data from an object, this obviously is a failure you won't catch.)
Your article is a good one but in my experience, many companies are still many years from being able to implement that kind of database:machine knowledge consistency.
Re:Riemann – A network monitoring system
by bbrazil on Tuesday January 19, 2016 @10:47AM
(#10931453)
You can only alert in that case if the data starts, and there's no transient issues preventing your monitoring working around the time it stops.
Alerting based on state changes is fragile, it's better to compare against what you expect to see.
You can leverage service discovery to make your configuration dynamic: http://prometheus.io/blog/2015/06/01/advanced-service-discov...
Re:Riemann – A network monitoring system
by robzienert on Tuesday January 19, 2016 @09:37AM
(#10930977)
That's what the Prometheus push gateway is for. While prom itself is poll-based, you can extract data from servers behind a firewall by pushing metrics from an agent into the gateway. No crazy issues.
Re:Riemann – A network monitoring system
by bbrazil on Tuesday January 19, 2016 @10:01AM
(#10931157)
The general advice is to run Prometheus behind the firewall, you want as few things between your monitoring and what you're monitoring as possible. This makes it more resilient.
Re:Riemann – A network monitoring system
by lobster_johnson on Tuesday January 19, 2016 @06:43PM
(#10935004)
It supports push, though.