News for hackers. Stuff that matters.

A comment on All you need is Wide Events, not "Metrics, Logs and Traces". Parent comment

Re:All you need is Wide Events, not "Metrics, Logs and Traces" by growse on Tuesday February 27, 2024 @05:54PM (#39531319)
The people feeling the pain of (and paying for) the expensive data swamp are often not the same people who are yolo'ing the sample rate to 100% in their apps, because why wouldn't you want to store every event?

Put another way, you're in charge of a large telemetry event sink. How do you incentivise the correct sampling behaviour by your users?

[ Parent | on HN ]
Re:All you need is Wide Events, not "Metrics, Logs and Traces" by Spivak on Tuesday February 27, 2024 @06:48PM (#39531866)
Don't let the user pick the sampling rate. In Honeycomb land this is called the EMA Dynamic Sampler.

https://docs.honeycomb.io/manage-data-volume/refinery/sampli...

[ Parent | on HN ]
Re:All you need is Wide Events, not "Metrics, Logs and Traces" by kiitos on Tuesday February 27, 2024 @09:20PM (#39533116)
You should never need to sample telemetry data.
[ Parent | on HN ]