> you're trusting software developers either way, whether it be at the app level, the language/runtime level, or the operating system level.

I trust systems to do better based on observed behavior rather than a software engineer's guess of how it will be scheduled. Who knows if, in a given use case, the program is a "small" part of the system or a "large" part that should get preferential placement and scheduling.

> If a threadpool manager is hinted that 4 threads are going to share a lot of memory, they can be allocated on the same l2 cache.

And so this is kind of a weird thing: we know we're going to be performance critical and we need things to be forced to be adjacent... but we don't know the exact details of the hardware we're running on. (Else, just numa_bind and be done...)

The beauty is that you don't care what hardware you run on, all you're annotating are very useful but generic properties such as which threads are sharing a lot of memory, or perhaps that a thread should have highest performance priority so that internally it stays on p cores instead of the more scalable e cores. Very simple optional hints.

> should have highest performance priority so that internally it stays on p cores

Everything will decide that it wants P cores; it's not punished for battery or energy impact, and wants to win over other applications for users to have a better experience with it.

And even if not made in bad faith, it doesn't know what else is running on the system.

Also these decisions tend to be unduly influenced by microbenchmarks and then don't apply to the real system.

> which threads are sharing a lot of memory

But if they're not super active, should the scheduler really change what it's doing? And doesn't the size of that L2 matter? It doesn't matter if e.g. the stuff is going to get churned out before there's a benefit from that sharing.

In the end, if you don't know pretty specific details of the environment you'll run on: what the hardware is like, what loading is like, what data set size is like, and what else will be running on the machine -- it is probably better to leave this decision to the scheduler.

If you do know all those things, and it's worth tuning this stuff in depth-- odds are you're HPC and you know what the machine is like.

To clarify, what gets scheduled is up to the OS or runtime, all you're doing is setting relative priority. If everything is all the same priority, then it's just as likely to all run on e cores.

And then, what's the point?

A system that encourage everyone to jack everything up is pointless.

A system to tell the OS that the developer anticipates that data is shared and super hot will be mostly lied to (on accident or purpose).

There's the edge cases: database servers, HPC, etc, where you believe that the system has a sole occupant that can predict loading.

But libnuma, and the underlying ACPI SRAT/SLIT/HMAT tables are a pretty good fit for these use cases.

If you lie about the nature of your application, you'll only hurt performance in this configuration. You're not telling the OS what cores to run on, you're simply giving hints as to how the program behaves. It's no different than telling the threadpool manager how many threads to create or if a thread is long lived. It's a platform agnostic hint to help performance. And remember, this is all optional, just like the threadpool example that already exists in most major languages. Are you going to argue that programs shouldn't have access to core count information on the cpu too? They'll just shoot their foots as you said.

Again, there's already explicit ways for programs to show fine control; this stuff is already declared in ACPI and libnuma and higher level shims exist over it. But generally you want to know both how the entire machine is being used and pretty detailed information about working set sizes before attempting this.

Most things that have tried to set affinities have ended up screwing it up.

There's no need to put an easier user interface on the footgun or to make the footgun cross-platform. These interfaces provide opportunities for small wins (generally <5%) and big losses. If you're in a supercomputing center or a hyperscaler running your own app, this is worth it; if you're writing a DBMS that will run on tens of thousands of dedicated machines, it may be worth it. But usually you don't understand the way you'll be employed well enough to know if this is a win.

In the context of the future of heterogeneous computing, where your standard pc will have thousands of cores of various capabilities and locality, I very much disagree.

> where your standard pc will have thousands of cores

Thousands of non-GPU cores, intended to run normal tasks? I doubt it.

Thousands of special purpose cores running different programs like managing power, managing networks, managing RGB lighting around? Maybe, but that doesn't really benefit from this.

Thousands of cores including GPU cores? What you're talking about in labelling locality isn't sufficient to address this problem, and isn't really even a significant step towards its solution.

CPUs are trending towards heterogenous many core implementations. 16 core was considered server exclusive a few decades ago, now we're at heterogenous 24 core on an Intel 14900k cpu. The biggest limit right now is on the software side, hence my original comment. I wouldn't be surprised if someday the cpu and gpu become combined to overcome the memory wall, with many different types of specialized cores depending on the use case.

The software side is limited, somewhat intrinsically (there tend to be a lot of things we want to do in order--- Amdahl's law wins).

And even when you aren't intrinsically limited by that, optimal placement doesn't reduce contention that much (assuming you're not ping-ponging a single cache line every operation or something dumb like that).

But the hardware side, too: we're not getting transistors that quickly anymore, and we don't want anything too much smaller than an Intel E-core. Even if we stack 3D, all that net wafer area is not cheap and isn't cheapening quickly.

NUMA aware threading is somewhat rare but it does exist.

Its just reaching into the high arts of high-performance that fewer-and-fewer programmers know about. I myself am not an HPC expert, I just like to study this stuff on the side as a hobby.

So NUMA-awareness is when your code knows that &variable1 is located in one physical location, while &variable2 is somewhere else.

This is possible because NUMA-aware allocators (numa_alloc in Linux, VirtualAlloc in Windows) can take parameters that guarantee an allocation within a particular NUMA zone.

Now that you know certain variables are tied together in physical locations, you can also tie threads together with affinity to those same NUMA locations. And with a bit of effort, you can ensure that threads that are in one workpool share the same NUMA zones.

---------

Now code-awareness of shared caches is less common. But following the same models of "abstracted work pools of thread-affinity + NUMA awareness of data", programmers have been able to ensure Zen1 cores to be working together with the same L3 cache.

L2 cache with E-cores is new, but not a new concept in general. (IE: the same mechanisms and abstractions we used for thread-affinity checks on Zen cores sharing L3 cache, or multi-socket CPUs being NUMA Aware... all would still work for L2 cache).

I don't know if the libraries support that. But I bet Intel's library (TBB) and their programmers are working on keeping their abstractions clean and efficient.

> I don't know if the libraries support that. But I bet Intel's library (TBB) and their programmers are working on keeping their abstractions clean and efficient.

Intel can declare in ACPI a set of nodes, the distances between nodes, and then Linux/libnuma/etc pick it up.

So, e.g. in AMD's SLIT tables, the local node is 10; within the same partition are 11; within the same socket are 12; distant sockets are >=20.

There's fancier, more detailed tables (e.g. HMAT) and some code out there that uses them, but it's kind of beyond the scope of libnuma.