News for hackers. Stuff that matters.
Re:Thundering Herds and Promises by fwip on Wednesday April 17, 2019 @12:26PM (#19683984)
My reading is that the cache is just another JavaScript service - a proxy that says "on: URL, if (in cache) return val, else fetch, add val to cache, return val."

Changing the cache to store promises is indeed an easy way to deduplicate most requests that occur roughly at the same time.

The logic now is: "on URL: if (in cache) return promise.resolve(), else add promise to cache, fetch, return promise.resolve()"

Re:Thundering Herds and Promises by mason55 on Wednesday April 17, 2019 @02:57PM (#19685403)
But what does it mean for a promise to be in the cache? If you're using an external cache, like redis or memcached, you can't just put a JavaScript promise in there and expect everything to work. Neither of those services understands what it means for the promise to resolve so that all the clients waiting on it can continue.

At best you could serialize the promise and then have everyone who was waiting on it poll the cache to see if the promise was resolved yet, but that is pretty inefficient.

You probably want another layer in there that handles the conversion of a JavaScript promise to something that can be awaited on from your cache, like a redis pub/sub. Then in JavaScript land your services just await on a promise as normal but under the covers your service is doing something else which is specific to the cache you're using.

Re:Thundering Herds and Promises by fwip on Wednesday April 17, 2019 @03:09PM (#19685489)
I don't think they are using an external cache.
Re:Thundering Herds and Promises by coding123 on Wednesday April 17, 2019 @03:11PM (#19685502)
even if you have 1000 servers that all have a local promise you may still be helping the backend service by reducing 10000000 requests to 1000.