How percentile approximation works and why it's more useful than averages

Re:How percentile approximation works and why it's more usef... by djk447 on Tuesday September 14, 2021 @01:06PM (#28527673)

NB: Post author here.

Yeah, that was one of the reasons we chose it as one of the ones to implement, seemed like that was a really interesting tradeoff, we also used uddsketch[1] which provides relative error guarantees, which is pretty nifty. We thought they provided different enough tradeoffs that we wanted to implement both.

[1]: https://arxiv.org/abs/1908.10693

[ Parent | on HN ]

Re:How percentile approximation works and why it's more usef... by vvern on Tuesday September 14, 2021 @01:41PM (#28528130)

Is it using https://github.com/tvondra/tdigest under the hood, or a separate implementation?

[ Parent | on HN ]

Re:How percentile approximation works and why it's more usef... by mfreed on Tuesday September 14, 2021 @02:19PM (#28528617)

If folks are interested:

https://github.com/timescale/timescaledb-toolkit/blob/main/e...

(The TimescaleDB Toolkit is also implemented in Rust)

[ Parent | on HN ]

Re:How percentile approximation works and why it's more usef... by rogers18445 on Wednesday September 15, 2021 @03:41PM (#28543460)

Facebook seems to have an even better performance implementation using sqrt. Might make sense to port that over to Rust. https://github.com/facebook/folly/blob/master/folly/stats/TD...

[ Parent | on HN ]

Re:How percentile approximation works and why it's more usef... by Lockerman on Tuesday September 14, 2021 @01:43PM (#28528142)

a separate implementation

[ Parent | on HN ]

Re:How percentile approximation works and why it's more usef... by _0ffh on Wednesday September 15, 2021 @10:30AM (#28539515)

Hi, in an unrelated nitpick: The relative error should be calculated by dividing the error by the true value, not by it's approximation. Still, very nice writeup!

[ Parent | on HN ]

Re:How percentile approximation works and why it's more usef... by djk447 on Wednesday September 15, 2021 @12:14PM (#28540946)

Thanks! messed up the formula but had it right in the text :( Fixed now.

[ Parent | on HN ]