News for hackers. Stuff that matters.
Re:Improving distributed caching performance and efficiency ... by mandeepj on Sunday July 03, 2022 @02:09AM (#31965489)
> the fleet serves up to ~180 million requests per second

For comparison, Google serves about 63k queries per second. I hope there's not a typo in the above line in Pinterest's blog

Re:Improving distributed caching performance and efficiency ... by LINKIWI on Sunday July 03, 2022 @04:40PM (#31970811)
Hi, I'm the original author of this article, though I have since left Pinterest.

180M/s is the peak throughput I've observed from the entire fleet, and is an accurate figure.

It's worth noting that:

(1) When it comes to caching workloads, there is often a wide spread of request amplification factor from a single inbound user request to the site. For example, a search query on Pinterest might internally fan out to an order of magnitude more requests to memcached across several different services along the request path used for servicing that query.

(2) There are many systems outside of the online critical path that use caching, which add load on the system independent of the rate at which users are posting or viewing content on the Pinterest site.

As another reference point, Facebook, who developed mcrouter, shared that their memcached deployment serves on the order of billions of requests per second [0]. And this figure is from 2014; I imagine it's grown a lot since then.

[0] https://www.usenix.org/system/files/conference/nsdi13/nsdi13...

Re:Improving distributed caching performance and efficiency ... by divyekapoor on Sunday July 03, 2022 @02:37AM (#31965610)
Google serves up way way more than 63k queries per second (several million requests/s is common for the critical services around search). Source: I worked there.

However, your main point is valid (180 million/s seems way too high). I've started a thread internally to double check these numbers. Please wait for an update.

Re:Improving distributed caching performance and efficiency ... by ddorian43 on Sunday July 03, 2022 @03:38AM (#31965871)
It's 180M/s memcached requests. A single page view probably hits 10s of requests. Just like a google search may hit 1K+ servers
Re:Improving distributed caching performance and efficiency ... by mandeepj on Sunday July 03, 2022 @10:43AM (#31968056)
> Google serves up way way more than 63k queries per second (several million requests/s is common for the critical services around search). Source: I worked there.

All right. But, here's my source - https://www.google.com/search?q=how+many+requests+does+googl... . Maybe, it's flawed

Re:Improving distributed caching performance and efficiency ... by mutreta on Sunday July 03, 2022 @08:14AM (#31967124)
Their numbers seem reasonable.

As an anecdote, I've worked at a scale up that handled ~20M HTTP(S) req/min on their service mesh.

The rule of thumb was that our Redis cache layer would have 10x the number of HTTP request, so 200M req/min.

Note that I'm using minutes, not seconds on the units. But it wouldn't surprise me that Pinterest handles 60x the load that we had back then.

Re:Improving distributed caching performance and efficiency ... by ksec on Sunday July 03, 2022 @12:36PM (#31968964)
It is 180 Million divided by 5000 instances, or 36K request per instance.

Honestly not that high of a number from memcached [1] prospective. It could easily handle 10x that even with SSD extstore.

[1] https://memcached.org/blog/nvm-multidisk/

Re:Improving distributed caching performance and efficiency ... by LINKIWI on Sunday July 03, 2022 @04:32PM (#31970762)
Yeah, 36k per instance (especially on an xlarge or 2xlarge EC2 instance) is well within the serving capacity of memcached. While it depends a lot on the workload profile for a specific cluster, some clusters serve on the order of ~5k/instance while others are as high as ~100k/instance. We've done a lot of experimentation with extstore as well; it certainly eats up more compute cycles on average than an equivalent in-memory only cluster, but is still quite efficient.
Re:Improving distributed caching performance and efficiency ... by aparsons on Sunday July 03, 2022 @02:24AM (#31965554)
180 million a day sounds about right
Re:Improving distributed caching performance and efficiency ... by divyekapoor on Sunday July 03, 2022 @03:14AM (#31965757)
I've started a thread internally to double check these numbers. Please wait for an update.