Re:Tokio Console
by hinkley on Friday December 17, 2021 @05:58PM
(#29599376)
> we probably want two digits of precision maximum
I think it depends a lot on jitter in the system. Sigfigs are one kind of error bar, one where I often have to haul my coworkers or myself out of trying to read things into the data that aren't there.
There are times where a 10ms change in a 2 second response actually matter to me, because that's half a percent and not all improvements which are easy are also straightforward. Sometimes you're scrambling for 3% here and 2.2% there.
But if the noise in the system is ±50ms then people declaring that they've shaved 15ms off of response time are likely deluding themselves and then deluding the rest of us.
I know how to do some of these things by hand, I'm not sure how you automate them, or in the case of a dashboard, typeset them.
I think it depends a lot on jitter in the system. Sigfigs are one kind of error bar, one where I often have to haul my coworkers or myself out of trying to read things into the data that aren't there.
There are times where a 10ms change in a 2 second response actually matter to me, because that's half a percent and not all improvements which are easy are also straightforward. Sometimes you're scrambling for 3% here and 2.2% there. But if the noise in the system is ±50ms then people declaring that they've shaved 15ms off of response time are likely deluding themselves and then deluding the rest of us.
I know how to do some of these things by hand, I'm not sure how you automate them, or in the case of a dashboard, typeset them.