News for hackers. Stuff that matters.
Re:Intel and ARM Performance Characteristics for S3 Compatib... by redis_mlc on Monday June 08, 2020 @01:09AM (#23453744)
> The cross socket NUMA bandwidth is like a tiny straw.

And the Internet is a series of tubes!

Re:Intel and ARM Performance Characteristics for S3 Compatib... by StillBored on Monday June 08, 2020 @09:54AM (#23456482)
How about a ballpark.

The unidirectional bandwidth across a two socket system is roughly equal to a single DDR channel. Which is why you see postings on the intel forums about people complaining about their perf falling off a cliff if the hardware decides to flag the directory as requiring cross socket snoop (or whatever its actually doing). So even if you have done a reasonable job keeping your thread+data locally, doing a remote reference, or accidentally running the thread for a short while on the wrong socket can permanently lower the performance. See https://software.intel.com/en-us/forums/software-tuning-perf... for an example about a contented cache line.

Thankfully the latest products are mostly 10.4GT (gold and above) as it appears intel isn't doing as much product segmentation based on UPI link speeds anymore (silver/bronze are still slower though).

https://www.microway.com/knowledge-center-articles/performan...

Re:Intel and ARM Performance Characteristics for S3 Compatib... by kllrnohj on Monday June 08, 2020 @01:51PM (#23458847)
> The unidirectional bandwidth across a two socket system is roughly equal to a single DDR channel

Depends on the implementation. Epyc Rome's 2P socket design is 64 PCI-E 4.0 lanes between the two CPUs. That's 128 GB/s bandwidth, which is roughly equivalent to 6-channel DDR4 2666.

Re:Intel and ARM Performance Characteristics for S3 Compatib... by StillBored on Monday June 08, 2020 @02:34PM (#23459332)
Yes, I should have been clearer that I was speaking of intel products (which is what the article was talking about) not NUMA in general.