News for hackers. Stuff that matters.
Re:Apple introduces M6 and M5 Ultra for a big leap in perfor... by Zagitta on Tuesday August 25, 2026 @08:45PM (#49442784)
There are kernel patches to enable P2P communication via PCIE on 3090 which is almost as fast as NVLink for vllm
Re:Apple introduces M6 and M5 Ultra for a big leap in perfor... by mft_ on Wednesday August 26, 2026 @07:48AM (#49447361)
Please correct me if I'm wrong, but:

* 3090 memory bandwidth: 936 GB/s

* Maximum NVLink bandwidth between two 3090s: ~56 GB/s one way

* Maximum PCIe v4 bandwidth between two 3090s: 31.5 GB/s one way

* M5 Ultra memory bandwidth: 1.2 TB/s

I know that memory bandwidth is only one factor influencing LLM performance, but this seems like a major problem if your goal is to run larger models that won't fit on one 3090 - and even those fitting on two 3090s with NVLink will be pretty restricted.