News for hackers. Stuff that matters.
Re:Observability is becoming mission critical, but who watch... by simskij on Friday September 16, 2022 @08:06AM (#32864980)
Author of the post here. I definitely agree with that assessment. Having multiple stacks monitor each other is a really good, albeit somewhat resource intensive, way to cover all of the *likely* cases. The cost of covering that last percent very quickly approaches the point of diminishing returns.
Re:Observability is becoming mission critical, but who watch... by ethbr0 on Friday September 16, 2022 @09:29AM (#32865815)
You start towards it in your post, but don't explicitly call it out: the central task is decreasing complexity.

A "dumb" heartbeat message, and a check for its absence, is reliable precisely because of its simplicity.

So if the question is "How do I monitor my complicated observability stack?" then the answer, to me, starts with "What is the simplest method of accomplishing the minimal version of my goals?"