Monitoring latency: Cloudflare Workers vs Fly vs Koyeb vs Railway vs Render

Re:Monitoring latency: Cloudflare Workers vs Fly vs Koyeb vs... by mtlynch on Monday February 19, 2024 @04:45PM (#39435200)

I was surprised to see such miserable measured latency to Fly, but then I saw this note:

>The primary region of our server is Amsterdam, and the fly instances is getting paused after a period of inactivity.

After they configured Fly to run nonstop, it outperformed everyone by 3x. But it seems like they're running the measurement from Fly's infrastructure, which biases the results in Fly's favor.

Also weird that they report p75, p90, p95, p99, but not median.

[ Parent | on HN ]

Re:Monitoring latency: Cloudflare Workers vs Fly vs Koyeb vs... by catlifeonmars on Monday February 19, 2024 @07:45PM (#39436699)

I’m not sure what’s customary in most places, but in my experience you base most things off of avg and p99 and then you can use other percentiles to interpolate the shape of the distribution in the event you need a better model (you usually don’t). Of course I’m sure this sort of thing varies wildly by use case.

[ Parent | on HN ]

Re:Monitoring latency: Cloudflare Workers vs Fly vs Koyeb vs... by mtlynch on Tuesday February 20, 2024 @04:24PM (#39447109)

I admit that I don't deal a lot with latency measurements, but my memory from working at Google was that we focused on p50, p95, and p99. That was six years ago, and it wasn't my focus there, so it's possible I'm misremembering (especially based on all the responses saying I'm the weirdo for expecting p50).

Looking at Google's SRE book, they use p50, p85, p95, and p99, so it's possible I'm misremembering or that Google uses unusual metrics:

https://sre.google/sre-book/service-level-objectives/#fig_sl...

[ Parent | on HN ]

Re:Monitoring latency: Cloudflare Workers vs Fly vs Koyeb vs... by catlifeonmars on Tuesday February 20, 2024 @09:07PM (#39449397)

I wasn’t aware of the Google SRE book. Thanks, I’ve bookmarked it!

[ Parent | on HN ]

Re:Monitoring latency: Cloudflare Workers vs Fly vs Koyeb vs... by foofie on Monday February 19, 2024 @11:57PM (#39438202)

> Also weird that they report p75, p90, p95, p99, but not median.

I'm not aware of P50s having ever been a relevant performance metric in latency. The focus of these latency measurements were always the expected value for most customers, and that means P90-ish.

[ Parent | on HN ]

Re:Monitoring latency: Cloudflare Workers vs Fly vs Koyeb vs... by cheviethai123 on Tuesday February 20, 2024 @04:12AM (#39439310)

Using median to report latency is very weird. I haven't seen any tech report using median though and curious to see one, can you provide?

[ Parent | on HN ]