Re:Low Latency Optimization: Understanding Pages (Part 1)
by deadcanard on Tuesday November 29, 2022 @12:33PM
(#33790543)
2) DDIO must write its data somewhere in the L3 cache. It ends up in the shareable slice. So either you're okay with sharing your cache or cannot use these slices if you want exclusive access for your processes. That was my point.
3) CAT does not recognize processes but resctl does. Feels we're kinda nitpicking here...
Last of your point: Yeah, that gives you 9ish usable slices which is not very much. Again that was my point.
Re:Low Latency Optimization: Understanding Pages (Part 1)
by bitcharmer on Tuesday November 29, 2022 @12:50PM
(#33790753)
3) resctl just uses COS under the hood. The same limitation applies
> Yeah, that gives you 9ish usable slices which is not very much. Again that was my point
This is 9 ways that you can use for your latency-sensitive workloads exclusively. This is MUCH better than letting all that LLC get trashed by non-critical processes/threads. Typically after applying such partitioning we've observed a 15-20% speed up in our apps.
In my area shaving off a few micros that way is a huge deal and definitely worth spending a couple of minutes implementing.
2) mutual cache eviction by hash collisions is solvable with a number of tricks (although those methods are not easy and often wasteful). The "DDIO slice" issue used to be a problem back when Intel used ring topology for LLC. These days they are built as a mesh thus minimizing this effect.
3) CAT doesn't recognize threads or processes. COS (class of service) uses CPU cores for way-of-cache assignments
Recent micro-architectures like SKX or CLX have 11 ways of L3 and what often happens is 1-2 ways get assigned to cpu0 for non latency-critical workloads while the rest are assigned to latency-sensitive, isolated cores usually running a single user space thread each.