I'd assume the correct thing to do is a vector clock, where each blocks time stamp is nothing more than a offset from the previous blocks.

It's easy to get most machines in agreement with how long a second is, but it's very very hard to get all nodes in agreement to what the exact time is.

So skip it entirely.