So something like an optical FPGA?

not really, only the interconnects between nodes are optical

Multiplexer for optical paths, which allows you to change the point-to-point wiring. Much like physical circuit switching in a 50s telephone exchange, actually.

As the paper explains, optical circuit switches are not new in TPU v4 and not the main topic of this paper. Google was already using it for networking and published about it last year. For details, see https://arxiv.org/abs/2208.10041.

The future of the 90's. Optical matrix switches like this have been around for a long time. These aren't doing packet switching (which honestly would be the future if done optically), it's more of a layer 1 thing - the switch replaces you plugging and unplugging a cable. Bell labs was building these kinds of switches back in the day.

Like other Google datacenter technologies the innovation of their OCS is that it's a cheap hack that nobody would tolerate outside of Google. Their whole secret sauce is that everything just barely works.

You think reconfigurable MEMS optical switches are cheap hacks? Um, no.

The whole paper is focused on cost efficiency. I think that's in keeping with Google's history, where for example they were the first to use ultra short range optics, because that is an OK way to save a dime.

There's a difference between cost efficiency and "cheap". In this case, we're talking about advanced photonic systems running in a data center to tie together world-class computing equipment; they didn't go for "cheap". The total investment is in the tens of billions of dollars.

They are "cheap hacks" compared to a packet switched network that runs at that speed. Also, this is another reason why Google can't sell TPUs to other people - nobody would put up with managing this sort of switching network for something they bought. The equivalent for NVidia is to use HDR/NDR Infiniband, and it allows you to run a multi-tenant cluster a lot more efficiently, at no practical loss of performance (due to the marginally higher latency).

I don;'t think you can directly compare a reconfigurable optical switch with a packet switched network. A packet switched networks receives packets electrically, sends them through a processor, and then outputs the results on another port. This is a device that creates static paths between endpoints that can then be dynamically changed later.

It also has the advantage that when they move to multi-wavelenght, its performance will greatly exceed electrical packet-switching networks.

I actually kind of disagree with you about comparing the two - I have some experience from trading firms which convinced me that they are not actually that different (for context, trading firms use a lot of layer 1 switching, and hacks that take place between layers 1 and 2).

If you think of packets like snakes going through a network, a layer 1 switching network creates tunnels for the snakes that you choose ahead of time (and can reconfigure whenever you want). A packet switched network creates tunnels that are chosen by the snakes. If you run a packet switched network, you can do everything you do with a layer 1 switched network by simply restricting which peers you send data to. On a hardware level, you need to convert from optics to electricity to do this, but you don't strictly need to do any buffering (the use of large switch buffers on Ethernet switches is because of Ethernet, not because of packet switching). Low-latency switches don't buffer unless they need to, and basically just read the header as it's coming in to choose a route for the packet.

EDR Infiniband networks could certainly handle TPU v4 levels of bandwidth in a packet-switched fashion (at the time when TPU v4 was being built and deployed), particularly when the packets are doing something as tame as going around a torus. It also gives you the flexibility to do other things, though.

It certainly raises the complexity of the system, but I assume sometime around TPU v6 or v7, Google will rediscover packet switching for inter-TPU links.

"Any idiot can build a bridge that stands, but it takes an engineer to build a bridge that barely stands."

Example history: https://patents.google.com/patent/US4580873

40 year old patent. But I'm thrilled to see it applied at scale to reconfigurable supercomputers.

On a related note, Google also uses optical circuit switches in their datacenter network. See the paper form SIGCOMM'22 [1].

[1] Jupiter Evolving: Transforming Google’s Datacenter Network via Optical Circuit Switches and Software-Defined Networking. https://research.google/pubs/pub51587/