Re:Staying Out of TTL Hell
by thaumasiotes on Wednesday March 10, 2021 @02:07PM
(#26414838)
> Another common problem in caching is the "thundering herd".
> you have a fleet of web nodes going at full bore, and the cache expires. If the data takes 10 seconds to compute, then for the whole next 10 seconds, every new request will find the data missing from the cache since it has expired and not yet been replaced, and so each of those requests will take on the expensive and redundant work of repopulating that data.
Wait, is the problem that as soon as the cache entry expires everyone requests the new data all at once, overwhelming the backend ("thundering herd"), or is it that requests follow whatever the normal pattern is but the backend doesn't know how to queue them?
> you have a fleet of web nodes going at full bore, and the cache expires. If the data takes 10 seconds to compute, then for the whole next 10 seconds, every new request will find the data missing from the cache since it has expired and not yet been replaced, and so each of those requests will take on the expensive and redundant work of repopulating that data.
Wait, is the problem that as soon as the cache entry expires everyone requests the new data all at once, overwhelming the backend ("thundering herd"), or is it that requests follow whatever the normal pattern is but the backend doesn't know how to queue them?