News for hackers. Stuff that matters.
Re:Apple introduces M6 and M5 Ultra for a big leap in perfor... by mft_ on Tuesday August 25, 2026 @03:21PM (#49439227)
AIUI you can only connect two 3090s at a time with NV link? So you’d only have 48GB of fast combined memory? That’s not tremendously interesting as you’re still limited to the smaller models which, while impressive in their own right, are still IMO too limited to use as your only model.
Re:Apple introduces M6 and M5 Ultra for a big leap in perfor... by Zagitta on Tuesday August 25, 2026 @08:45PM (#49442784)
There are kernel patches to enable P2P communication via PCIE on 3090 which is almost as fast as NVLink for vllm
Re:Apple introduces M6 and M5 Ultra for a big leap in perfor... by mft_ on Wednesday August 26, 2026 @07:48AM (#49447361)
Please correct me if I'm wrong, but:

* 3090 memory bandwidth: 936 GB/s

* Maximum NVLink bandwidth between two 3090s: ~56 GB/s one way

* Maximum PCIe v4 bandwidth between two 3090s: 31.5 GB/s one way

* M5 Ultra memory bandwidth: 1.2 TB/s

I know that memory bandwidth is only one factor influencing LLM performance, but this seems like a major problem if your goal is to run larger models that won't fit on one 3090 - and even those fitting on two 3090s with NVLink will be pretty restricted.