This comes into play when you copy or access huge files that are going to be read exactly once, they will start pushing out untouched program pages to disk, in exchange for disk cache that is completely 100% useless, even to the tune of hundreds of gigabytes of it.
Programs can reduce the problem with madvise(MADV_DONT_NEED), but that only applies to files you are mmap()ing, and every single program under the sun needs to be patched to issue these calls.
You can adjust vm.swapiness systctl to make X larger, but no matter what, programs will start to get pushed out to disk eventually, and cause unresponsiveness when activated. You can reduce vm.swapiness to 1, but if you do, the system only starts swapping in an absolute critical low ram situation and you encounter anywhere from 5 minutes, to 1+ hour of total, complete unresponsiveness in a low ram situation.
There _NEEDS_ to be a setting where program pages don't get pushed out for disk cache, peroid, unless approaching a low ram situation, but BEFORE it causes long periods of total crushing unresponsiveness.
Here's the thing: a mapped program page is just another page in the page cache. Now, you could maybe say that "any page cache page that is mapped into at least one process will be pinned", but the problem there is that means that any unprivileged process can then pin an unlimited amount of memory, which is an obvious non-starter.
A workable alternative might be to add an extended file attribute like 'privileged.pinned_mapping', which if set indicates that any pages of the file that have active shared mappings are pinned. That means the superuser can go along and mark all the normal executables in this way, and the worst-case memory consumption a user can cause is limited by the total size of all the executables marked in this way that the user has access to.
https://www.suse.com/documentation/sles-for-sap-12/book_s4s/...
It is quite effective, although historically there have been issues with bugs causing server lockups in the kernel code around this tunable. It seems to be quite stable in SLES 15, however.
While the tunable is available in their regular SLES product, it is only supported in the "SLES for SAP". The two share the same kernel, that is probably why.
Nobody is suggesting these pages be pinned which is an extreme measure.
That might be OK on a single-user system but it doesn't fly on a multi-user one. That's why I suggested you could gate that kind of thing behind some kind of superuser control.
The kernel could still “fairly” evict pages across users - just letting them choose which N pages they prefer to go first.
Because then you just get everyone asking for maximum QOS / don't-page-me-out on everything they can.
The pages in the page cache are not owned by a particular user, they're shared. If there's three users running /usr/bin/firefox, they'll all have have shared read-only executable mappings of the same page cache pages. If you do a buffered read of a file immediately after I do the same, we both get our data copied from the same page cache page. So it's not at all clear how you'd do the accounting on this to implement that user-based fairness criterion.
fadvise provides the same for file descriptors. some tools such as rsync make use of it to prevent clobbering the page cache when streaming files.
heck, if I were still a phd student, I'd want to run performance numbers on this in many different scenarios and see how performance behaves. feel like there could be a paper here.
And if you only do it in a container you can also limit the page cache size of the container to avoid impacting the other workloads.
i.e. I really wonder for desktop workloads if one only caches "executable" data, how would that negatively impact perceived performance. I'd imagine it have some impact, but I'd be interested in seeing it quantified.
[1]: https://wiki.freebsd.org/ZFSTuningGuide#L2ARC_discussion
Whenever I take a backup of my computer it winds up swapping everything else out to disk. Normally I'm perfectly happy letting unused pages get evicted in favor of more cache, but for this specific program this behavior is very much not ideal. I'm asking here since I've done some searching in the past and not found anything, but I'm not sure if I was using the right keywords.
THIS. I ended up disabling swap because my kernel insisted on essentially reserving 50% of RAM for disk buffers; meaning even with 16GiB of RAM, I'd have processes getting swapped out and taking forever to run, because everything was stuck in 8GiB of RAM, and Firefox was taking 6GiB of that. I couldn't for the life of me figure out a way to get Linux to make that something more reasonable, like 20%. (And yes, I tried playing with `vm.swapiness`.)
> This comes into play when you copy or access huge files that are going to be read exactly once
This is a good reason for programs to use buffered IO even when they are reading a large file once, so yes my comment was entirely on point.
- "Readahead... needs to be buffered"
- "reading unbuffered (ie. O_DIRECT)... you cannot read ahead"
Good application code needs to work around kernel limitations all the time. Your suggestion of fadvise is reasonable, except in practice fadvise rarely makes a measurable difference. The only way to get what you really want is to code it the way you want it, and fortunately, there are multiple ways to do that.
Did you try different vm.vfs_cache_pressure values?
TBH, there are very few OSes that get high-pressure resource scheduling and prioritization right under nearly all normal circumstances.
The hackaround for decades on Linux is always adding a tiny swap device, say 64-256 MiB on a fast device in order to 0) detect average high memory pressure with monitoring tools 1) prevent oddities under load without swap (as in OP example).
I would have thought some of the IRIX scheduler made it into Linux by now.
https://en.wikipedia.org/wiki/Zram
https://wiki.gentoo.org/wiki/Zram
Or Armbian, a Debian derivative for many RaspberryPi-like ARM boards, where it helps avoiding trashing the usual sd-cards, and holds the logs, with different compression algorithms for /var/log, /tmp and swap.
Their script is here:
https://github.com/armbian/build/blob/master/packages/bsp/co...
I really like how this enables a tiny NanoPi Neo2 with one GB Ram booting from a 64GB SD-Card in an aluminum case with SATA-adapter and a 2,5" HDD mostly idling to draw only about 1W from the wall socket, while clocking up from about 400Mhz to 1,3Ghz if it needs to. It's not the fastest, but sufficient.
But if it solves problems, why doesn't the kernel automatically assign part of RAM as a virtual swap device, if the user allows it. It can help monitoring tools, and because the kernel knows it is not a real swap device, it can optimize its use.
Linux does unfortunately have serious issues to do with peaks of latency which make it behave horribly with realtime tasks such as pro audio. It's so bad that it's often perceiveable in desktop usage.
linux-rt does mitigate this considerably, but it's still not very good.
I'm hopeful for genode (with seL4 specifically), haiku, helenos and fuchsia.
And I agree that a small amount of swap can actually reduce the paging that occurs, if you assume that the amount of RAM required is independent of the amount of RAM available. However, as we all know, stuff grows to fill the available space, and if you do configure swap you just delay the inevitable, not prevent it.
Having said that, having swap available means that when memory pressure occurs, you have a more graceful degradation in service, because the first you know about it is when the kernel starts swapping out an idle process to free memory for the new memory hog that you are interacting with. This slows down your interactive session, but not as much as if you have no swap available - in that case, the system suddenly and drastically reduces in performance because it is trying to swap in your interactive process. The more graceful degradation of having some swap available gives you a chance to realise that you are doing more than your computer can cope with, and stop.
As far as I see it, there are three solutions:
1. Disable overcommit. This tends to not play very nicely with things that allocate loads of virtual memory and not use it, like Java. And if you do have a load of processes that actually use all the memory they allocated, then you can still have the same problem occur. The solution to that one is to get the kernel to say no early, before the system actually runs out of RAM.
2. Get the OOM killer to kill things much earlier, before the system starts throwing away clean pages that are actually being used. On my system with 384GB RAM, I have installed earlyoom, and instructed it to trigger the OOM killer if the free RAM falls below 10% (and remember that stuff you are actually using, but happens to be clean, counts as free RAM). This is the easiest and quickest solution right now. If your main objection to this is that you are inviting the system to kill things that might be important, remember that the kernel already does this, and if you don't like it you should use option 1 above (and really hope that all your software handles malloc failure correctly).
3. Introduce a new system in the kernel to mark pages that are actually being regularly used but are clean as "important", and no longer count as free RAM for the purposes of calculating memory pressure. This could either be as a new madvise (but it would be impractical to get all software developers to start using this), or by marking all binary text by default (which would neglect the large read-only databases that some programs hammer), or by some heuristics. This would then trigger the OOM killer (or the allocator to say no, depending on overcommit) when actual free RAM is low.
Many long-lived processes are completely idle (when was the last time that `getty ttyv6` woke up?) or at a minimum have pages of memory which are never used (e.g. the bottom page of main's stack). Evicting these "theoretically accessible but in practice never accessed" pages for memory frees up more memory for the things which matter.