It's important to note that if you want to use the blkio scheduler, if you're using the CFQ io scheduler[1] (which is the defualt on quite a lot of distributions) then using the blkio cgroup (even if you just join a non-root cgroup and don't have any limits set) can have massive performance penalties for O_DSYNC (and even some "regular") operations. It's caused by an implementation detail of blkio weighting within CFQ. We have a bug open in runc that details it[2].

Also, while this article is quite rosy-eyed about cgroupv2, there are many many issues that mean that cgroupv2 is not really usable by container runtimes today -- despite what systemd might be trying to do with their "hybrid mode". A friend of mine from the LXC community gave a talk about the wide variety of issues that occur due to cgroupv2's weirdly restrictive API[3]. The blkio writeback cache limits are very useful, but they're really not worth the pain of breaking how container runtimes have to work.

Also this comment is not _entirely_ accurate:

> Since in cgroups v1, different resources/controllers (memory, blkio) live in different hierarchies on the filesystem, even when those cgroups have the same name, they are completely independent. So, when the memory page is finally being flushed to disk, there is no way that the memory controller can know what blkio cgroup wrote that page.

You can mount a single cgroupv1 hierarchy with multiple controllers attached to it (that's what 'cpu,cpuset' are), which then could (in principle) know about it each other. But they don't, since a lot of the cgroupv2 code is locked behind cgroup_on_dfl(...) checks.

[1]: https://www.kernel.org/doc/Documentation/block/cfq-iosched.t... [2]: https://github.com/opencontainers/runc/issues/861 [3]: https://www.youtube.com/watch?v=P6Xnm0IhiSo