I am a little confused how the geo encoded addresses and private addresses work. It seems like the network will be overwhelmed with keeping track of switch labels?

Switch labels are effectively interface IDs on the servers, there is no data to be stored.

Geo encoding simply improves routing to unknown routers, kind of as a baseline structure to the whole network.