News for hackers. Stuff that matters.
Re:Going With The Flow: Google’s Secret Switch To The Next W... by noobface on Tuesday April 17, 2012 @02:40PM (#3854283)
Fuck Spanning Tree

edit: down voters, have you configured STP in a data center? have you had a single vmware esx instance shut down the root VLAN on a DC? Spanning tree is being addressed with solutions like this.

Re:Going With The Flow: Google’s Secret Switch To The Next W... by DanBC on Tuesday April 17, 2012 @05:02PM (#3854972)
There's a big difference between saying "This is good / bad; I've configured X and you have to put up with Y." and an empty "Fuck spanning tree".

You have knowledge. I could learn stuff from you. I learn almost nothing from your comment "fuck spanning tree".

This kind of behind the curtain stuff is mysterious to many people. I would welcome something that taught me more about it. I'd especially welcome informed insights from someone who works with the technology.

Re:Going With The Flow: Google’s Secret Switch To The Next W... by noobface on Tuesday April 17, 2012 @06:25PM (#3855356)
Sorry, I forget my audience when I'm on HN. I professionally teach classes regarding L2 networking and servers, so I do hope I have some insight to share. I was a Cisco systems engineer at one point in my life.

Spanning Tree Protocol: http://en.wikipedia.org/wiki/Spanning_Tree_Protocol

STP as it's called, builds linear networks. Simple, single paths, through layer 2 (see: ethernet) networks. Think of spanning tree as a large state table tracking all MAC addresses on a network. If the state table realizes there a duplicate entries (ie: duplicate paths) for a single MAC address, it literally brings down the entire network to recreate path without duplicate entries.

Most managed (commercial) ethernet switches, speak the spanning tree protocol; this protocol allows synchronization of MAC tables between switches. However, by default, the vmware vswitch does not speak this protocol. This creates problems when you multi-home servers (connect a single server to multiple siwtches). The vswitch does not participate in spanning tree, and the default vswitch "load-balances" by transmitting frames from the various ports it has accessible. This, in the traditional switches' eyes, constitutes a loop in the network and can bring an entire ethernet domain down. This is a horrific scenario during which all participating hosts lose network access for 15 seconds or more, depending on the configuration (STP vs. RVSTP). If the vswitch remains active with it's default settings, the network may be down until a network engineer realizes the problem or the server is taken offline.

The reason I say "fuck spanning tree" is as a network engineer, I've taken entire data center's off-line due to a mistaken configuration on a ESX host (which I did not have visibility into at the time). This is obviously not a good way to go about production practices.

Network coordination services, like the one developed by Google, stand a good chance of replacing this antiquated protocol. Everyone in the ethernet networking world has been plagued by STP and its related quirks. I'm, for one, very happy to see its demise and hope for a future clear of such, potentially, disruptive technology with data centers.

Re:Going With The Flow: Google’s Secret Switch To The Next W... by sounds on Tuesday April 17, 2012 @08:38PM (#3855793)
More info about switching loops in case anyone wonders, "why not turn off STP?"

http://en.wikipedia.org/wiki/Switching_loop

Re:Going With The Flow: Google’s Secret Switch To The Next W... by wmf on Tuesday April 17, 2012 @02:56PM (#3854359)
As much as I agree with you, WAN backbones weren't running spanning tree to begin with.
Re:Going With The Flow: Google’s Secret Switch To The Next W... by noobface on Tuesday April 17, 2012 @04:56PM (#3854946)
Nope, they still link the instances. L2 domains traverse WANs in DCs now a days.
Re:Going With The Flow: Google’s Secret Switch To The Next W... by packetslave on Tuesday April 17, 2012 @06:57PM (#3855477)
Only in some (read: insane) datacenter designs. You don't need spanned layer 2 domains unless you're doing crazy things like long-distance Vmotion. There's almost always a better way that invites less pain.
Re:Going With The Flow: Google’s Secret Switch To The Next W... by noobface on Tuesday April 17, 2012 @09:22PM (#3855965)
Totally agree, but there are protocols in place to tunnel layer 2 domains to encompass multiple data centers. VMotion between DCs is not as crazy as it sounds, it's just prohibitively expensive.