Re:Proposed server purchase for GitLab.com
by late2part on Sunday December 11, 2016 @08:09PM
(#13154708)
The latency is almost certainly immaterial, 0.3uSec v. 2uSec. That's MICROseconds not MILLIseconds. Power draw used to be an issue, but not anymore.
Consider the power pull from copper v fiber listed here [2]. The Arista 7050TX 128 port pulls 507W while the 7056SX 128 port pulls 235W. Yes, copper is more but we're talking half a kW for 2 TOR switches. And for this you get much cheaper cabling, as the SFP+ are much more expensive (go AOC if you do go fiber, BTW) and you have to worry about clean fiber, etc.
Where fiber is REALLY nice is with the density, although the 28AWG [3] almost makes that moot.
There's few DC builds that can't do end-to-end copper any more, at least for the initial 50 racks.
[1] - http://www.datacenterknowledge.com/archives/2012/11/27/data-... [2] - https://www.arista.com/en/products/7050x-series [3] - https://www.monoprice.com/product?p_id=13510
The PHY has to do a lot of forward error correction and filtering, so it adds latency (for the FEC), power (for all the DSP) and cost (for the silicon area to do all of the above).