100Gb links do take some thought and work to saturate, but that's improving at a good rate lately so I expect it'll become more common rather soon.
The main downside to 25Gb and 100Gb links still seems to be hardware pricing. At these speeds, PCIe network adapters and switches get rather expensive rather quick and will make you really evaluate if your situation really demands those speeds. 10Gb SFP+ and copper network adapters and switches are quite inexpensive now in 2022.
But that's tweaking the setup already, it requires changes, testing and verification, and can cause problems in downstream equipment. And for a lot of applications, a 9K MTU will not be enough to saturate the link because they'll need NUMA awareness, or the NIC queues will need tweaking to avoid imbalances, or the application is not ready to send at that speed...
I'm not saying it can't be done, of course it can. But it isn't "plug a bigger card and it'll go faster".
That doesn't seem right. When I got my first 10G server, it was running dual Xeon E5-2690 (either v1 or v2), and I don't recall needing to tweak much of anything. That was mostly a single large file downloaded over http, so not super hard to tweak anyway, but server chips are a lot better now than sandy/ivy bridge. It could only get 9gbps out with https, but the 2690v4 could do 2x10G with https because aes acceleration.
Well, my point is that most servers don't just download single large files over HTTP. Even if you only look at storage servers, going into multiple files and connections you can easily find issues and have downgraded performance if you don't prepare the system for the workload.
Unfortunately, I'm finding switches with 2.5 to still be overpriced.
Instead we run 10G signaling at half or quarter clock rates and get something that works on the majority of existing wiring.
AFAIK the IEEE was initially resisting supporting this, but enough vendors were just doing it anyways that it was better to standardize.
I'd rather have a 100Mbps home DIA circuit that was symmetric and seriously reliable, than the typical Comcast circuit.
I would also very much rather have a symmetric connection than what I have now. We are within range of a fiber tap for Comcast's 3Gbps symmetric ftth service but construction costs are too high right now for them to do it. I keep asking every six months regardless. There's no other game in town as far as I can tell.
Comcast Business will run a 100Mbps symmetric fiber for ~$750/mo, maybe that would be a thing to do to get fiber in the door and then switch back to residential after the contract earns out.
https://www.cablinginstall.com/sponsored/berk-tek/article/16...
... points out that 5GBASE-T needs 200 MHz channel bandwidth which is past what Cat 5e is specified for. So perhaps for runs approaching 100 meters or in noisy environments, Cat 5e won't be reliable for 5GBASE-T after all.
I think what's new is the prevalence now of 2.5 and 5GBaseT ethernet chips that are cheap enough companies are starting to build them into any $125+ ATX motherboard. At short lengths even old crappy cat5e has a good chance of working at 2.5 or 5.0 speeds.