Load balancing based on consistent hashing is the better way to implement this.
Short of something like cassandra's ring topology, how would you use consistent hashing add new servers and assign them requests?
Better consistent hashing means that existing servers don't have their caches invalidated, but the new servers that were just added start with empty caches anyway so are fielding all uncached requests. Hopefully the bottleneck is actually with some shared layer behind it (a database or something) otherwise I guess you'd need to come up with a more complex way to slowly distribute more traffic to the new nodes.
user_id%numb_server may work early on when user activity and uptake are consistent,
but what happens when user activity becomes more complex: increase in users, some users abandoning the platform, others using it more; and that complexity lacks homogeneous distribution through this only concerned property: 'user id';
what if over time you gain more users but the majority of people who drop the platform have a user_id%numb_servers==2|11|13|17
in this case you would have some servers working hard while others sitting dormant
what is the real distribution of the relation between activity and user_id over time? asymptotic(o)? similar to the prime numbers(i)? a gaussian distribution(ii)? a benford distribution(iii)?
whichever future dada will show to be the best fit, most distributions show a strong trend toward eventual favoring of values
which i think implies, to ensure an even distribution of work across servers, the problem requires something with greater dimensionality than modulo on an immutable value that is defined serially
(o) https://en.wikipedia.org/wiki/Asymptotic_analysis
(i) https://en.wikipedia.org/wiki/Prime_number_theorem
(ii) https://en.wikipedia.org/wiki/Probability_density_function
if you are introducing monitoring on a per user precision why use modulo? with a per user scheduled monitoring moving users based on user ids works as is
maybe i was unclear in the above but i like the gp's simple solution.. especially because i personally have an affection for the modulo operator, but also because.. it only requires an operator that performs in a scale dependent finitely specific number of cycles and works as designed without any monitoring
the above was intended to bring attention to shortcomings and probable failures in an otherwise elegant attempt
the method is flawed but the direction is superb
I did not want to use just any non random rebalancing mechanics to avoid advesaries attacking that implementation. With a hash the output is deterministic, but unpredictable.
Anyway, I do have a point beyond being pedantic: this offers two advantages that a fixed sharding scheme doesn't. #1: it doesn't need to identify a piece of data on the request to shard off of. #2: it actively (though imperfectly) attempts to achieve similar utilization on every server.
In fact, we use consistent hashing when we accept requests, and two random choices when we deliver them to the apps. This works much better for _most_ of the apps we see. We're typically worried about cache data for a particular app. The app instances themselves, though, tend to be mostly stateless and disposable.
The other main problem is that it's not a consistent hash: if you grow the server pool, you typically need to reshard a lot of content.
(It's still useful in a pinch, but it helps to be aware of the tradeoffs.)
This solves caching too since you are only ever receiving and caching user data on a single server. No cache communication required. You can enforce it on the server side for security as well.
Doesn't require a load balance server - just an extra line of code.
Keep it simple.