News for hackers. Stuff that matters.
Re:Core to Core Latency Data on Large Systems by bee_rider on Tuesday November 07, 2023 @05:28PM (#38183898)
The NUMA nature of recent* chips has made me wonder if there’s ever going to be a movement to start using message passing libraries (like MPI) on shared memory machines.

* actually, not even that recent, Zen planted this hope in my brain.

Re:Core to Core Latency Data on Large Systems by nvartolomei on Tuesday November 07, 2023 @05:31PM (#38183964)
Thread-per-core software architectures are doing this https://penberg.org/papers/tpc-ancs19.pdf

Real world examples are scylladb and Redpanda, both built on the seastar framework (C++ https://seastar.io/message-passing/).

And for rust there is glommio https://www.datadoghq.com/blog/engineering/introducing-glomm...

Re:Core to Core Latency Data on Large Systems by RedlineTriad on Wednesday November 08, 2023 @05:00AM (#38188463)
There is also another thread-per-core implementation by ByteDance (TikTok) for Rust called Monoio with benchmarks[0] comparing it to Tokio and Glommio.

[0] https://github.com/bytedance/monoio/blob/master/docs/en/benc...

Re:Core to Core Latency Data on Large Systems by sapiogram on Wednesday November 08, 2023 @06:36AM (#38189048)
Does thread per core necessarily imply message passing? I don't see why the two need to be related.
Re:Core to Core Latency Data on Large Systems by yencabulator on Wednesday November 08, 2023 @01:19PM (#38194576)
The thread-per-core manifesto has a goal of not sharing data between cores, and thus the communication inside the process becomes message passing, handing off ownership of a chunk of data to the recipient core. This lack of sharing is what enables the performance (no locks etc needed, outside of the message passing).

This is a good watch (first half is pure background, second half talks about the motivation): https://www.youtube.com/watch?v=PbgTyCSDPrs

Re:Core to Core Latency Data on Large Systems by senderista on Tuesday November 07, 2023 @06:07PM (#38184363)
A good recent paper on implementing message passing over shared memory:

"Message Passing or Shared Memory: Evaluating the Delegation Abstraction for Multicores"

https://cs.brown.edu/~irina/papers/2013-opodis.pdf

Re:Core to Core Latency Data on Large Systems by the_svd_doctor on Tuesday November 07, 2023 @06:26PM (#38184573)
In HPC it's common to do a mix of MPI (message-passing / distributed memory) and OpenMP (shared memory) parallelism when running on big multicore (and obviously multi-node) machines. It helps with locality, among other things.
Re:Core to Core Latency Data on Large Systems by bee_rider on Wednesday November 08, 2023 @11:10AM (#38192450)
This is what I do actually, and it works fairly well. Currently I do one MPI process per socket, but mostly just because the OpenMP code I’m calling is a library, and it doesn’t seem to scale well past one modern Xeon worth of cores.

I don’t know what I’d do if I had an old Zen machine, maybe map an MPI process to each chiplett.

My impression is that in the first generation Zen machines, the cost of communicating from one chiplett to another was really quite significant, but they’ve made good enough progress there that it is only something that the really hardcode folks care about.

Re:Core to Core Latency Data on Large Systems by adgjlsfhk1 on Tuesday November 07, 2023 @08:10PM (#38185458)
IMO, MPI is the wrong level to do this on. Most apps should either be using some form of mapreduce or not using parallelism beyond the numa node.
Re:Core to Core Latency Data on Large Systems by zozbot234 on Wednesday November 08, 2023 @06:58AM (#38189223)
The map-reduce programming model is overly simplified. It cannot express useful primitives such as prefix scan, which is used all the time in parallel algorithms.
Re:Core to Core Latency Data on Large Systems by hinkley on Tuesday November 07, 2023 @09:12PM (#38185912)
I keep thinking that Rust’s borrow semantics would be pretty good for hinting whether code should run on the same core or could be offloaded to another. Two modules that only communicate via small, read only messages could easily be on separate cores.

And on architectures where some cores share faster paths than others, gradations could be scheduled that way.

Re:Core to Core Latency Data on Large Systems by adapteva on Tuesday November 07, 2023 @09:37PM (#38186088)
Re:Core to Core Latency Data on Large Systems by wmf on Tuesday November 07, 2023 @10:52PM (#38186563)
Nope, Parallela was the wrong thing at the time and it's still wrong. Cache is good.
Re:Core to Core Latency Data on Large Systems by rewmie on Wednesday November 08, 2023 @06:51AM (#38189156)
> Nope, Parallela was the wrong thing at the time and it's still wrong.

Can you elaborate?

Re:Core to Core Latency Data on Large Systems by wmf on Wednesday November 08, 2023 @11:49AM (#38193096)
It didn't have DRAM or caches. Programming with scratchpads is so difficult that people just give up.
Re:Core to Core Latency Data on Large Systems by imtringued on Wednesday November 08, 2023 @01:18PM (#38194551)
Scratchpads are the memory equivalent of VLIW.

Processors can extract parallelism dynamically at runtime. They can also manage your memory automatically at run time. Better yet, they can utilize hardware resources instead of software resources. It is such an obvious win.

Re:Core to Core Latency Data on Large Systems by bee_rider on Wednesday November 08, 2023 @02:37PM (#38195793)
Are you saying that scratchpads were like VLIW in the sense that they seemed like a really cool idea, but failed because people didn’t want to manually code things well enough to take advantage of them?
Re:Core to Core Latency Data on Large Systems by adapteva on Wednesday November 08, 2023 @07:44AM (#38189630)
Absolute statements are bad.
Re:Core to Core Latency Data on Large Systems by imtringued on Wednesday November 08, 2023 @01:10PM (#38194423)
I don't know where you got that idea from. There is a movement in the complete opposite direction with CXL. Don't waste your time with silly libraries, serialisation or networking. Have a rack that is filled with nothing but memory pooled RAM and then connect your servers (which still retain RAM as a L4 cache). You now have a huge shared memory machine with distributed CPUs using CXL for cache coherence accross the entire system. There have been benchmarks that kept 75% of the memory outside the server and the performance degradation was only 10% compared to keeping the entire data set on a single server.
Re:Core to Core Latency Data on Large Systems by menaerus on Thursday November 09, 2023 @04:21AM (#38202793)
> There have been benchmarks that kept 75% of the memory outside the server and the performance degradation was only 10% compared to keeping the entire data set on a single server.

Performance degradation would greatly depend on how much data was actually touched by the workload outside the server and not solely by the fact that 75% of the memory was attached through CXL, no?

NUMA latency I measured last time on a dual-socket Xeon (Haswell) system was around 130ns for non-local memory access and 90ns for local memory access. OTOH some numbers I found seem to imply that the CXL latency is ~200ns.

This means that on average CXL latency is almost 100% larger than NUMA so I think it is not realistic to have only 10% performance degradation unless most of your workload fits into L1/L2/L3 cache plus that 25% of local memory or your workload is more CPU bound rather than memory bound.