Re:Investigating the impact of HTTP3 on network latency for ...
by blerb795 on Monday May 22, 2023 @10:50PM
(#36039228)
Former Dropbox employee, just correcting one assumption:
> Dropbox's peak traffic is 1,500 queries per second (QPS).
I can't speak to search QPS directly, but most individual serving hosts for file sync/retrieval were receiving tens of thousands of QPS. The overall edge QPS peaked at several hundreds of thousands QPS every day across all the hosts. So I'd guess that even just search is an order of magnitude higher than 1,500 :)
Re:Investigating the impact of HTTP3 on network latency for ...
by asymptotic on Tuesday May 23, 2023 @10:19PM
(#36053304)
I misread the article when it stated:
> Traffic regularly exceeded 1,500 queries per second (QPS) at peak times
It may be quite cost prohibitive to use a managed service like AWS Global Accelerator.
One potential solution that hasn't been discussed much here is leveraging AWS managed services like AWS CloudFront or AWS Global Accelerator. It's worth noting that Dropbox's website already uses AWS CloudFront, so they are already leveraging AWS in some capacity. Based on my cost calculations, using AWS CloudFront would cost around $40k a month, while AWS Global Accelerator would be around $22k a month.
As of August 22, 2022, AWS CloudFront supports terminating HTTP/3 in a Point of Presence (POP), which could potentially help with the latency issues Dropbox is facing. AWS Global Accelerator, on the other hand, is designed to improve the performance of applications by terminating UDP/TCP as close to users as possible then routing user traffic through the AWS global network infrastructure. This could help reduce latency by ensuring that user traffic is routed through the most optimal path, even if the user is located far from a Dropbox data center.
It's hard to estimate what the potential latency reduction of using e.g. AWS Global Accelerator is, especially at higher percentiles. However, using https://speedtest.globalaccelerator.aws/, and assuming symmetry, my connections to Asia are 35-40% lower latency.
Of course, there are trade-offs to consider when using managed services like AWS CloudFront and AWS Global Accelerator. While they can provide significant performance improvements, they also come with additional costs and potential vendor lock-in. However, given the scale of Dropbox's operations and the importance of providing a fast, reliable search experience for their users, it may be worth exploring these options further.
---
Cost estimates
Assumptions:
---1) AWS CloudFront cost estimation:
Total estimated monthly cost for AWS CloudFront: $36,200 (data transfer) + $2,916 (HTTP requests) = $40k---
2) AWS Global Accelerator cost estimation:
Total estimated monthly cost for AWS Global Accelerator: $21,384 (data transfer) + $540 (accelerator) = $22k