What's the use case of sharding on a technical key? I thought most sharding is done on business keys (e.g. per customer, region or year)?
Re:ULIDs and Primary Keys
by crazygringo on Thursday December 29, 2022 @07:04PM
(#34179851)
In my experience sharing is generally done on a "technical" key because it's easy to arbitrarily partition into any desired number of shards.
Something like per-year could result in different years having vastly different disk usage requirements, which would be a headache.
While per-region or per-customer isn't really sharding at all in my understanding. It's just giving each customer (or region) their own dedicated/private database.
Sharding is generally about dividing a single table (for business logic purposes) among (usually) equally-sized servers for performance/capacity reasons, according to some function of the primary key (modulus, hash, etc.).
A customer ID could well be a UUID. And sharding by year is similar to sharding by a UUID or ULID with a timestamp prefix.
But yes, in what dimension you shard will usually depend on the application.
Maybe the best option is something like a UUIDv7 (time + random) ID and you can tune your shard keys by shuffling the bits. Maybe you want something like moving the last 4 bits to the front of the shard key so that you have 16 hot spots, which is good for performance without any being too hot.