I read this slightly differently, that specific machine types with Nvidia GPU hardware also have Nvidia networking for tying together those GPUs.

Google has its own TPUs and don’t really use GPUs except to sell them to end customers on cloud I think. So using Nvidia networking for Nvidia GPUs across many machines on cloud is really just a reflection of what external customers want to buy.

Disclaimer, I work at Google but have no non public info about this.

Having just worked with some of the Thread folks at M&S, thought I'd reach out and say hello. Seems like it was an awesome team! (=

You're lucky to be working with them, an amazing team.

That was their plan with trying to buy ARM...

Grace Hopper already includes Arm based CPUs (and reference motherboards)

Nvidia networking is what used to be called Mellanox networking, which was already dominant in datacenters.

Only within supercomputers (including the smaller GPU ones used to train AI). Normal data centers use Cisco or Juniper or similarly.well known Ethernet equipment, and they still do. The Mellanox/Nvidia Infiniband networks are specifically used for supercomputer-like clusters.

Mellanox Ethernet NIC got used a bunch of places due to better programmability.

You seem to have a narrow definition of “normal” for datacenters. Meta were using OCP mellanox NICs for common hardware platforms a decade ago and still are.

Mellanox IB were ubiquitous in storage networking. None of the storage systems I worked on would have been possible without mellanox tech.

Most people on the TrueNAS/FreeNAS forums use 10gb Mellanox nics that are sold on eBay. The sellers get them after server gear gets retired.

I have to wonder if Nvidia has reached a point where it hesitates to develop new products because it would hurt their margins. Sure they could probably release a profitable networking product but if they did their net margins would decrease even as profit increased. This may actually hurt their market cap as investors absolutely love high margins.

They can always release capital back to investors, and then those investors can put the money into different companies that eg produce networking equipment.

Why would they release money if they can invest it and return much more?

I was working under HDThoreaun's assumption that the margins would be lower.

If they have other opportunities for investment with higher margins, they should seize those, of course. And perhaps even call up investors for more capital, if required.

When the agents employed by investors would be harmed by releasing capital back, which is guaranteed since so many people’s compensation is in the form of stock and returning capital leads to decreasing stock value, why would those agents ever return the capital voluntarily?

Have you heard of stock buybacks?

Buybacks increase stock price. Nvidia has a tiny dividend but most shareholder returns are in the form of buybacks.

Yeah there’s a bit of industry worry about that very eventuality — hence the ultra Ethernet consortium trying to work on open source alternatives to the mellanox/nvidia lock-in.

https://ultraethernet.org/

Interesting Nvidia is on the steering committee

Cisco have sat on the steering committees for a lot of things where they had a proprietary initial version of something. It's not that unusual, and also, it's often frankly not actually that open; e.g., see the rent seeking racket for access to PCI documentation, or USB-IF actively seeking to prevent open source hardware from existing, etc.

Eh, the UEC effort is a standards org through the Linux Foundation so it won't be subject to any of the usual chicanery. And actually, it looks like Nvidia is jsut a general member and not one of the Steering Committee members;

https://ultraethernet.org/wp-content/uploads/sites/20/2023/0...

I believe this is what they plan on doing. See, for example:

https://www.youtube.com/live/Y2F8yisiS6E?si=GbyzzIG8w-mtS7s-...