How Discord supercharges network disks for extreme low latency

Re:How Discord supercharges network disks for extreme low la... by seastarer on Tuesday August 16, 2022 @09:23AM (#32482380)

Persistent Disk read latency is usually good, but jitters up to 100ms. The scheduler can compensate for high latency (essentially applying Little's Law) but cannot compensate for jitter without saturating the disk.

[ Parent | on HN ]

Re:How Discord supercharges network disks for extreme low la... by ec109685 on Tuesday August 16, 2022 @12:07PM (#32484766)

Thanks. Do you see the same spikes with ebs volumes on AWS?

[ Parent | on HN ]

Re:How Discord supercharges network disks for extreme low la... by seastarer on Tuesday August 16, 2022 @01:38PM (#32486116)

Some more data on Google PD: https://raw.githubusercontent.com/scylladb/diskplorer/master...

Very odd - notice how the latency is high when the read IOPS are low. When the read IOPS climb, 95th percentile latency drops.

Looks like there is a constant rate of high latency requests, and when the read IOPS climb, that constant rate moves to a higher quantile. I'd inspect the raw results but they're quite big: https://github.com/scylladb/diskplorer/blob/master/latency-m...

Background: https://github.com/scylladb/diskplorer/

[ Parent | on HN ]

Re:How Discord supercharges network disks for extreme low la... by ec109685 on Wednesday August 17, 2022 @12:15AM (#32492269)

Very interesting. I wonder if higher iops involves reading from more servers and with request hedging it can smooth latency a lot better.

[ Parent | on HN ]

Re:How Discord supercharges network disks for extreme low la... by seastarer on Tuesday August 16, 2022 @01:30PM (#32486015)

No.

[ Parent | on HN ]