News for hackers. Stuff that matters.
Re:Sharding and IDs at Instagram by jconley on Friday September 30, 2011 @06:30PM (#3058787)
"What does matter is that the timestamp is new and increasing every second."

Right. I'm just sayin' that you have to be careful when you move data with a caveat like that. Moving the shard keyspace (the 13 bits) to a new machine that started generating ID's even one second behind (the first 4 bytes) would be troublesome, no?

Re:Sharding and IDs at Instagram by mikeyk on Friday September 30, 2011 @06:32PM (#3058796)
Yep--definitely something to watch out for. At worst, though, you'd have a duplicate key when trying to insert, and can re-try without the risk of having a duplicate ID floating around your system.
Re:Sharding and IDs at Instagram by andrewf on Saturday October 01, 2011 @12:53AM (#3059374)
The lower rotating 10 bits should give them a reasonable safety margin. If they're creating less than 128 entries in a particular shard per second (right now they're doing that across their entire datastore), their clocks would need to be out by 8 seconds to cause a problem.

They should definitely be monitoring their clocks though :)