Re:Spanner: Becoming a SQL System [pdf]
by dgacmu on Sunday May 14, 2017 @10:12PM
(#14338690)
CSACs seem unnecessary. Within a few racks, PTP (IEEE 1588) can do a pretty decent job of getting things synchronized to tens of microseconds. Requires that your ToRs and NICs support it, but that's not a very onerous requirement, particularly if you're GoogleBookSoft.
This means you only need a few time receivers for a datacenter, along with careful monitoring and implementation of your time distribution, but that happens through your Ethernet switches.
But you need your datacenters to stay synced even if you lose GPS. You can use an atomic clock for this (Cs or Rb). But, for the rest of us, a good GPS-disciplined double-oven crystal oscillator (XO) can get you within the range needed for Spanner, IIRC spanner's time sync requirements. For example, this little one: https://www.microsemi.com/document-portal/doc_download/13341...
will do +- 7 microseconds / 24h holdover. ("holdover" == operating when it's lost GPS).
Re:Spanner: Becoming a SQL System [pdf]
by tyingq on Sunday May 14, 2017 @10:33PM
(#14338762)
Sounds like spanner can't tolerate much drift: "The most serious problem would be if a local clock’s drift were greater than 200us/sec: that would break assumptions made by TrueTime."[1]
This means you only need a few time receivers for a datacenter, along with careful monitoring and implementation of your time distribution, but that happens through your Ethernet switches.
More: http://www.ni.com/newsletter/50130/en/
But you need your datacenters to stay synced even if you lose GPS. You can use an atomic clock for this (Cs or Rb). But, for the rest of us, a good GPS-disciplined double-oven crystal oscillator (XO) can get you within the range needed for Spanner, IIRC spanner's time sync requirements. For example, this little one: https://www.microsemi.com/document-portal/doc_download/13341...
will do +- 7 microseconds / 24h holdover. ("holdover" == operating when it's lost GPS).