Re:Intel and ARM Performance Characteristics for S3 Compatib...
by kllrnohj on Monday June 08, 2020 @01:51PM
(#23458847)
> The unidirectional bandwidth across a two socket system is roughly equal to a single DDR channel
Depends on the implementation. Epyc Rome's 2P socket design is 64 PCI-E 4.0 lanes between the two CPUs. That's 128 GB/s bandwidth, which is roughly equivalent to 6-channel DDR4 2666.
Re:Intel and ARM Performance Characteristics for S3 Compatib...
by StillBored on Monday June 08, 2020 @02:34PM
(#23459332)
Yes, I should have been clearer that I was speaking of intel products (which is what the article was talking about) not NUMA in general.
The unidirectional bandwidth across a two socket system is roughly equal to a single DDR channel. Which is why you see postings on the intel forums about people complaining about their perf falling off a cliff if the hardware decides to flag the directory as requiring cross socket snoop (or whatever its actually doing). So even if you have done a reasonable job keeping your thread+data locally, doing a remote reference, or accidentally running the thread for a short while on the wrong socket can permanently lower the performance. See https://software.intel.com/en-us/forums/software-tuning-perf... for an example about a contented cache line.
Thankfully the latest products are mostly 10.4GT (gold and above) as it appears intel isn't doing as much product segmentation based on UPI link speeds anymore (silver/bronze are still slower though).
https://www.microway.com/knowledge-center-articles/performan...