News for hackers. Stuff that matters.
Re:Almost Always Add Swap Space by scottlamb on Friday July 19, 2019 @11:51AM (#20479622)
This is bad advice when the swap is backed by spinning disk rather than SSD, in large part because paging in once the problem is solved is not aggressive enough.

Let me make it concrete: say 1 GiB of your application's RAM gets swapped out then is later paged in only as needed (4 KiB pages) with no readahead. Now the application's VM space is a minefield: there are up to 262,144 times it can stall for a ~10 ms disk seek (for a total of ~40 minutes). Sequentially reading in 1 GiB of RAM from disk would take only ~5 seconds.

Hopefully OSs use some readahead, but I'm sure Linux and macOS don't use enough. I find a system that has ever swapped to be totally unusable until I do "sudo swapoff -a" (on Linux) to force everything to be paged in, or just reboot (on macOS, I haven't found any other way).

Some of this can happen even without swap: the OS will still drop clean file-backed pages, so unless you mlock() your executables after startup (my production binary at work does this), you can still have major page fault-induced latency spikes.

Swap backed by SSD (or compressed RAM) should be more reasonable, but I've had enough bad experiences with swap on spinning disk that authoritative-sounding articles that encourage swap without mentioning this problem piss me off.

In contrast, if you don't have swap and run out of RAM, something will die and get restarted. In many cases this is a much better failure mode than continuing to run slowly.

Re:Almost Always Add Swap Space by Johnny555 on Friday July 19, 2019 @12:48PM (#20480267)
Also bad when it's backed by an attached drive from the cloud provider, like an AWS EBS volume -- it's particularly nefarious if you swap to the root drive, swapping can exhaust the EBS burst balance, and instead of 3000 IOPS to the root drive, after swapping sets in you're down to 300 IOPS (for a 100GB drive), and performance really slows to a crawl.

I know I'm giving up some RAM that could be swapped out without impact but it's worth the stability of never swapping to an attached drive.

Re:Almost Always Add Swap Space by magicalhippo on Friday July 19, 2019 @12:06PM (#20479775)
I've had the opposite experience on Windows. I always put swap on HDD, because I've been bitten several times by memory leaking applications rendering the entire OS unusable until the SSD with the swap file ran out of space.

Windows would happily grow the pagefile beyond the limits I had set, and if the allocation loop was tight enough the OS chocked on handling all page faults. As in using literally minutes to respond to keystrokes.

By putting swap on a slow HDD, Windows has plenty of time to service mouse and keyboard input while waiting for the disk to respond, allowing me to kill the offending application before saturating my disk.

I assume this behavior was OK back when computers had very limited memory, and disks were slow anyway. But these days, if a single application wants more than the available memory (physical + pagefile), I'd rather the OS just said no to the application.

Re:Almost Always Add Swap Space by scottlamb on Friday July 19, 2019 @12:18PM (#20479908)
I don't use Windows, but it sounds like you can't you set limits on how large the swapfile can get? That's crazy!

> But these days, if a single application wants more than the available memory (physical + pagefile), I'd rather the OS just said no to the application.

I agree.

Re:Almost Always Add Swap Space by magicalhippo on Friday July 19, 2019 @12:23PM (#20479970)
Oh you can set limits. And then Windows will promptly ignore them, at least under certain circumstances. Why I have no idea.
Re:Almost Always Add Swap Space by raverbashing on Friday July 19, 2019 @12:06PM (#20479780)
Swap sucks, but OOM killer sucks more. It certainly isn't a great solution, but it's good for temporary spikes.

You can adjust "swappiness" if it's moving stuff to disk too aggressively.

Re:Almost Always Add Swap Space by scottlamb on Friday July 19, 2019 @12:20PM (#20479931)
> Swap sucks, but OOM killer sucks more. It certainly isn't a great solution, but it's good for temporary spikes.

I disagree. The OOM killer usually kills the right thing, and if it doesn't a quick restart is usually better than continuing to run slowly.

You can also just put things in containers and set per-container limits.

> You can adjust "swappiness" if it's moving stuff to disk too aggressively.

What I want is to adjust un-swappiness: aggressiveness of paging stuff back in. Once the memory spike is over, it should page everything back in automatically.

Re:Almost Always Add Swap Space by mchristen on Friday July 19, 2019 @12:27PM (#20480018)
The OOM killer is just fine for stateless server applications, but pretty much anything else and it's a disaster waiting to happen.

Containers don't really solve your problem, it just adds a layer of indirection between the processes hogging RAM and the OOM killer, which in my experience adds even more uncertainty into what the OOM killer is going to do.

Re:Almost Always Add Swap Space by scottlamb on Friday July 19, 2019 @01:36PM (#20480760)
> The OOM killer is just fine for stateless server applications, but pretty much anything else and it's a disaster waiting to happen.

I won't say there's no circumstance in which the OOM killer is worse, but in general I disagree with you.

A stateful server application: maybe a standard ACID DBMS like PostgreSQL. I'd much rather it get killed (failing whatever transactions were in progress) and restart in a healthy state than continue to run slowly. Here I'm relying on a correct implementation of Durability, but if you don't have that, you have other problems.

Even more so for a quorum-based database server in which the leader dying (and another taking over) is much better than continuing to run slowly.

And yes, even more so in stateless servers, particularly when there are several of them. Going down and restarting quickly (a brief loss of capacity, a brief spike of errors if the client doesn't retry) is a lot better than continuing to be slow (a sustained loss of capacity, possibly leading to cascading overload; and user-visible latency problems if there's no hedging).

For something like a desktop application, it's debatable, but personally I'd still rather endure the up-front nuisance of having to restart it than have the OS try to paper over it until I finally realize why it's so painful and do something about it. (Insert anecdote about frog boiling here; I hear frogs actually do jump out before they boil, but yet somehow the story still rings true, if you know what I mean.)

Re:Almost Always Add Swap Space by mchristen on Friday July 19, 2019 @04:26PM (#20482143)
Databases are just one kind of stateful server application though, and yes I would agree with you for that usecase relying on it's underlying durability is a good thing, because it's built with that in mind.

Instead I would like to point out the ubiquity of things like nodejs, which make it all to easy for under-experienced developers to shoot themselves in the foot w/ a chaingun with respect to data design and management. It's not really the internal state of the nodejs process itself, it's the external state that it is inevitably being manipulated in the node process.

Re:Almost Always Add Swap Space by scottlamb on Friday July 19, 2019 @05:53PM (#20482761)
I'd rather go full chaos monkey and shake out such bugs (/design flaws) than try to avoid them.
Re:Almost Always Add Swap Space by mchristen on Friday July 19, 2019 @06:22PM (#20482923)
Completely agree there.

Some times that luxury doesn't exist though and you just have to get shit done and hope playing fast and loose doesn't come back to bite you.

Re:Almost Always Add Swap Space by VectorLock on Friday July 19, 2019 @12:54PM (#20480347)
Tuning oomkiller's hitlist makes it much more managable and the new BPF tools to track it make it a ton nicer than a system grinding into swap.
Re:Almost Always Add Swap Space by mnw21cam on Friday July 19, 2019 @12:53PM (#20480327)
Like all things in life, it depends. If you have a truly massive server, then swap becomes infeasible, and the OOM killer is an absolute lifesaver. In fact, so much so that you really want the OOM killer to act earlier, to prevent the system getting too low on free RAM/cache. There's a program called earlyoom that I installed on our server with 384GB of RAM - it has been useful multiple times, because it prevents the system entering the thrashing state that would make us basically email IT support and get them to press the big red button on the front.

Secondly, the thing with swappiness is that it alters the balance between stuff that is cache and stuff that is live data. But "stuff that is cache" includes your running software text, libraries, essential operating system bits, etc. If you set swappiness too low, then in a low memory situation the OS will throw away all the code that it is trying to run instead of swapping out some of the live data. That is what many people don't realise.

Re:Almost Always Add Swap Space by uaygsfdbzf on Friday July 19, 2019 @11:59AM (#20479700)
Author said almost always. Nothing is absolute in cases like this. I agree with the article. We enable swap on our production servers precisely because the engineers don't rectify their memory leaks or whatever problems are internal to their software. Some of those engineers are our own and we can't convince them to fix their software. Others are for things we have no control over, such as Kafka, for example. When Kafka starts going out of control and you've got no swap, oom killer will kill it and you'll end up with corrupted index files, which is a bigger problem than the performance problem you'll see if it has to swap for a while until it gets itself sorted out.
Re:Almost Always Add Swap Space by mason55 on Friday July 19, 2019 @11:29AM (#20479414)
I was hoping this post was going to say something like turning on some small amount of swap will allow the OS to cache some things that normally it wouldn't cache at all, and so you get some improvement there. What's actually in this post doesn't make any sense to me. If I have enough RAM why do I want things to swap to begin with? The author never explained why putting rarely used pages in swap will improve performance if I have adequate RAM.

And using swap as some kind of "RAM emergency cover" does not make any sense to me. Personally, I have never had a case where I could gracefully recover a host that had started swapping.

First, my production servers are not doing many other things besides being production servers. It's not like they are running a bunch of unnecessary services, and if I kill some then my application can recover. If a process is out of control then it's almost assuredly something important that I'm going to have to kill anyway.

Second, I find it's much harder to detect degraded performance than it is to detect a dead process. It's very, very easy to have a health check that will detect a host who has stopped listening and drop that host from the LB. And alerting on that scenario is very easy as well. The alternative is a host that's operating in a degraded state, which I need to detect with more sophisticated health check + alerting, and in the end my resolution is just going to be to kill everything anyway.

In a properly designed HA environment the loss of a host should be no big deal. Architecture should be focused on making sure a host goes down ASAP if it's having problems, not letting it survive in some kind of zombie state.

Re:Almost Always Add Swap Space by deathanatos on Friday July 19, 2019 @12:48PM (#20480276)
> If I have enough RAM why do I want things to swap to begin with? The author never explained why putting rarely used pages in swap will improve performance if I have adequate RAM.

The memory freed from a page that got pushed to swap might be better used for serving disk cache, for example.

If you don't have enough RAM (even for spikes), a preemptive push when things aren't busy might mean that when they do become busy, you won't have to wait.

At least, that's the theory as I understand it. Whether or not it works in practice, IDK.

> And using swap as some kind of "RAM emergency cover" does not make any sense to me. Personally, I have never had a case where I could gracefully recover a host that had started swapping.

I have successfully recovered hosts that began swapping, and even hosts that filled their swap. But it's beyond painful, and in most of the cases where I have a host doing such swapping, an OOM kill would have been a welcome reprieve.

> Second, I find it's much harder to detect degraded performance than it is to detect a dead process.

Agreed, and restarting something that's been OOM'd is simple enough too. Our hosts tend to stop reporting metrics when they start swapping, simply b/c so little is actually getting done, so they get labelled as "down". Linux also exposes a metric called "major page faults" (IIRC, it's "pgmajfault" in /proc/vmstat) that records the number of major (required service from disk) page faults; if the rate of that value is too steep for too long, that ≅ swap thrashing.

Re:Almost Always Add Swap Space by mnw21cam on Friday July 19, 2019 @01:03PM (#20480445)
The key point is that if you have no swap, and you run low on memory, then the OS has no choice but to throw away stuff that is clean. This includes stuff that is being actively used right now, like the code that is in your processes. So your system will still thrash, even though it has no swap, because it is trying to load pages of running code in on-demand, while constantly throwing them away again because it doesn't have enough space. This will incapacitate a server even more thoroughly than if there is swap, and the working set is larger than physical memory. The more RAM a server has, the more severe this incapacitation will be.

For this reason, you want an OOM killer to kill stuff way earlier than when you actually run out of RAM, because if you ever reach that point it is too late. I use a program called earlyoom, which has saved my server from having to have the big red button pushed quite a few times, when fumble-fingered PhD students "accidentally" consume all the RAM in their pet projects.

Re:Almost Always Add Swap Space by scottlamb on Friday July 19, 2019 @02:06PM (#20481030)
> I use a program called earlyoom, which has saved my server from having to have the big red button pushed quite a few times, when fumble-fingered PhD students "accidentally" consume all the RAM in their pet projects.

Along those lines, take a look at this: [1]

> Pressure stall information were added to Linux 4.20 as a way to quantify resource pressure in the system in a better way than the traditional load average. PSI aggregates and reports the overall wallclock time in which the tasks in a system (or cgroup) wait for cpu, io or memory.

> This release [Linux 5.2] lets users to configure sensitive thresholds and use poll() and friends to be notified when a certain pressure thresold is breached within a user-defined time window. With this mechanism, Android can monitor for, and ward off, mounting memory shortages before they cause problems for the user. For example, using memory stall, monitors in userspace like the low memory killer daemon (lmkd) can detect mounting pressure and kill less important processes before device becomes visibly sluggish. In memory stress testing psi memory monitors produce roughly 10x less false positives compared to vmpressure.

I haven't tried it yet, but it sounds promising.

[1] https://kernelnewbies.org/Linux_5.2#Improved_Presure_Stall_I...

Re:Almost Always Add Swap Space by mason55 on Friday July 19, 2019 @01:11PM (#20480514)
> That page that got pushed to swap might be better used for serving disk cache, for example.

> If you don't have enough RAM (even for spikes), a preemptive push when things aren't busy might mean that when the do become busy, you won't have to wait.

> At least, that's the theory as I understand it. Whether or not it works in practice, IDK.

These are the ideas that I hoped the original post was going to be about.

Disk cache is a good one. I'm not knowledgeable enough about this, will the Linux kernel automatically start using RAM as disk cache if there's enough free space? Is it smart enough to prioritize disk cache over infrequently used pages and then know to discard the disk cache if it does need to page stuff back in from swap?

Re:Almost Always Add Swap Space by FBT on Friday July 19, 2019 @01:30PM (#20480702)
The original post does talk about it, but it skips the basics, presumably on the assumption that they're already known.

> will the Linux kernel automatically start using RAM as disk cache if there's enough free space?

Yes, exactly.

> Is it smart enough to prioritize disk cache over infrequently used pages and then know to discard the disk cache if it does need to page stuff back in from swap?

It is, and this is what's controlled by the vm.vfs_cache_pressure sysctl that the post discusses how best to configure.

Re:Almost Always Add Swap Space by gnode on Friday July 19, 2019 @01:02PM (#20480432)
> The author never explained why putting rarely used pages in swap will improve performance if I have adequate RAM.

Putting rarely used (or never to be used again) pages in swap frees up physical RAM to be used as disk cache, which can improve performance for file access. If you have enough RAM to cover all application memory, and cache your entire filesystem, there is no gain to be had.

I agree that slowing down low memory scenarios is silly though. In many cases killing a process is easier to deal with than unexpected poor performance. The slow death of out of memory systems doesn't seem to be unique to having a swapfile though; Linux seems to discard any recoverable pages (such as application / library code backed by files on disc) in the run-up to OOM, resulting in thrashing.

Ideally I'd want to have swap, but automatically kill processes when performance deteriorates due to swapping, rather than when the system is strictly out of memory.

Re:Almost Always Add Swap Space by wrycoder on Friday July 19, 2019 @12:50PM (#20480290)
From TFA: "The Linux Kernel will move memory pages which are hardly ever used into swap space to ensure that even more cachable space is made available in-memory for more frequently used memory pages (a page is a piece of memory). Swap usage becomes a performance problem when the Kernel is pressured to continuously move memory pages in and out of memory and swap space."
Re:Almost Always Add Swap Space by mason55 on Friday July 19, 2019 @12:57PM (#20480379)
I still don't get how that answers the question. If I have enough RAM for both the "hardly ever used" pages and the "more frequently used" pages then what good is the swap doing me?

The only way that makes sense is if there is some cache the kernel has which is faster than RAM. BUT the kernel cannot move things between this cache and RAM, it can only treat the cache + RAM as a single blob, so to free up space in the cache the kernel has to move pages to disk. So by having some swap space the kernel can move hardly ever used pages to disk which is the only way to free up space in this cache.

It's possible that what I just described really exists, I don't know that much about the internals of how the kernel manages memory. But if that is what's happening then THAT is the detail that would have been useful in this post.

Re:Almost Always Add Swap Space by wrycoder on Friday July 19, 2019 @01:47PM (#20480855)
It does work that way.

I had a small, very lightly loaded VPS that had no swap. Occasionally, it would not start all its services on boot, in particular the db, due to OOM.

Adding a small amount of swap allowed it to boot cleanly. The swap was not subsequently used.

The alternative was to upgrade the server RAM at double the monthly cost.

Re:Almost Always Add Swap Space by pfranz on Friday July 19, 2019 @01:17PM (#20480576)
> If I have enough RAM why do I want things to swap to begin with? The author never explained why putting rarely used pages in swap will improve performance if I have adequate RAM.

One thing I think the author was getting at, but never explicitly stated is that we spec most things for the average expected workload with a bit of headroom--not the peak workload since that is very difficult to assess. Peak workload might also be few and far between and not worth paying for upfront. Swap allows you to deal with peak workloads more gracefully for many applications.

Having your process randomly die isn't a big deal with stateless processes (which is a more and more common design), but really sucks for stateful things. Not just the possibility of lost/corrupt state, but process startup and loading of state can take a long time.

We have a few servers that don't have swap and I was hoping to find an excuse to add it in this article. Instead, it reenforced that decision because of type of things we run on it.

Re:Almost Always Add Swap Space by eikenberry on Friday July 19, 2019 @02:50PM (#20481431)
> Swap allows you to deal with peak workloads more gracefully for many applications.

This is almost always false in practice though. Peak workloads means more than just lots of memory, it means high CPU and high IO (disk and network). If you under-spec your servers such that they need to hit swap during peak loads they are going to generally cause more problems than if they just die. A server dying is something you have to prepare for and deal with anyways, and is much simpler to plan for than the weird latency issues you'll get with a server hits swap.

Think of it as similar to Erlang's fail-fast model. Better for things to just die and be recovered than to continue on in a bad state.

Re:Almost Always Add Swap Space by pfranz on Friday July 19, 2019 @04:20PM (#20482104)
I'm not really sure what you're getting at. You and I are both emphasizing a different behavior for different classes of applications. I feel like my comment and the article were pretty clear about that. I've had the opposite experience than "almost always false in practice" but my domain is fairly different than most here so I try and be very careful when generalizing.

If your hardware is underspeced for the workload, there's still a business case to be made before you buy new hardware. Sometimes the preferred failure mode is to slow down, other times it's preferred to die and restart. Hardware speced for peak loads can sometimes cost multiple times otherwise appropriate hardware...assuming you even know what peak loads will be.

I have one class of application that is fairly idempotent, we don't have hard drives on these machines, so we don't use swap and expect it to die if it exceeds the amount of memory requested. This article reenforced this.

I have another class of application that queues up things to process. The specs work on the current hardware 99% of the time, but on holiday weekends when large projects are delivering things slow down because work is submitted and people leave so nobody cares. Reloading the state from disk can take 20m+ and without swap if you were to exceed memory you're dead in the water until you purchase more hardware. Our contingency hardware has identical specs as the original.

Re:Almost Always Add Swap Space by erik_seaberg on Friday July 19, 2019 @12:57PM (#20480381)
Redundancy helps with uncorrelated failures, but if you start getting requests that cause memory pressure and OOM kills, you could lose the entire cluster. I agree that swap is often too slow to help, but "self-destruct and hope the rest of the cluster is healthy right now" is not a good failure mode. Don't overcommit in prod.
Re:Almost Always Add Swap Space by mason55 on Friday July 19, 2019 @01:07PM (#20480478)
Sure, I agree with that 100%. I guess if you had a scenario where you were operating near your limits and something happened that caused one or a few hosts to start swapping, and because they were up the LB kept sending them traffic instead of redirecting it to other hosts, then you could avoid a thundering herd-type problem.

But that still relies on uncorrelated failures.

Otherwise that seems like an orthogonal discussion to swap vs. OOM killer. It's something to be aware of and you need circuit breakers in place but I don't think it's an argument for letting hosts swap instead of having the OOM killer deal with it.

Re:Almost Always Add Swap Space by RoutinePlayer on Friday July 19, 2019 @11:44AM (#20479564)
Awesome comment! Especially as it relates to VMs in modern cloud environments. There really is no need for swap space, and letting them fail quickly is usually best.

Using swap space may make sense for a traditional physical, stand-alone server, where having it fail quickly doesn't really help. Or maybe for a workstation.

Re:Almost Always Add Swap Space by tutfbhuf on Friday July 19, 2019 @11:29AM (#20479412)
> The Linux Kernel will move memory pages which are hardly ever used into swap space to ensure that even more cachable space is made available in-memory for more frequently used memory pages (a page is a piece of memory).

Why should I care if I always have enough RAM? Gedankenexperiment: Let's say u'd have infinite RAM, would swap make any sense?

Re:Almost Always Add Swap Space by vectorEQ on Friday July 19, 2019 @11:33AM (#20479466)
what if an application contains a memory leak. it could hypothetically use up your infinite ram, and linux will then start randomly killing stuff to try and recover.... adding swap allows you to trigger on memory issues / overload without random shit getting killed. you could then kill the offending process yourself specifically being triggered by some memory usage monitor or w/e instead of running the risk of other critical processes being killed.
Re:Almost Always Add Swap Space by saalweachter on Friday July 19, 2019 @11:38AM (#20479514)
Well to start, slap default resource limits on your processes so they can't use all the RAM in the world. Then your leaky processes will crash when their mallocs fail and everyone will be as happy as they deserve.
Re:Almost Always Add Swap Space by scottlamb on Friday July 19, 2019 @01:47PM (#20480854)
I agree with the idea, but a nit: on 64-bit Linux with the default vm.overcommit_memory=0, their mallocs will likely continue to succeed. Those just expand the address space, which is probably not what your resource limits are controlling. [1] The actual limit [2] is hit when they try to use the virtual address space, causing a minor page fault (when it's backed by a zero-filled page of physical RAM).

[1] You can add in such a limit via RLIMIT_AS, but it'd have a lot of collateral damage like preventing LMDB from mmap()ing a big database. I don't recommend this.

[2] I'd recommend setting limits on a cgroup; with a systemd unit file, see MemoryMax= and friends in the systemd.resource-control(5) manpage.

Re:Almost Always Add Swap Space by xorcist on Friday July 19, 2019 @06:39PM (#20483032)
mallocs never fails. By design. That's the whole point of all this.

On Linux you can make them fail, but don't do that unless you know what you're doing.

Re:Almost Always Add Swap Space by zamadatix on Friday July 19, 2019 @01:42PM (#20480813)
Why do you need to wait for it to swap to trigger a memory usage alert?
Re:Almost Always Add Swap Space by brylie on Friday July 19, 2019 @11:31AM (#20479438)
I have manually added swap on SSD VPS nodes that come without it by default. I was under the impression that the hosting providers were trying to preserve the hardware. Is swap space still bad for SSD drives?
Re:Almost Always Add Swap Space by parliament32 on Friday July 19, 2019 @01:04PM (#20480450)
It's not quite as black and white as the author makes it out to be. I've worked on (specialized) systems with 512gb of RAM with a 180gb SSD installed... No, you're not going to see any performance improvement at all by adding swap to the already insane amount of RAM, and if you run into an OOM issue where swap would be your parachute, you're already fucked.
Re:Almost Always Add Swap Space by Waterluvian on Friday July 19, 2019 @11:44AM (#20479562)
I have 32GB Ram on Ubuntu 18.04. Can I just turn swap off? If I do, what's the worst that will happen?
Re:Almost Always Add Swap Space by blihp on Friday July 19, 2019 @01:01PM (#20480423)
You can absolutely turn it off. I've been running with swap turned off on all of my desktop systems (Linux, Unix and Windows), and some servers, for over 20 years.

The theoretical worst case scenario is that the system hits an OOM situation and you end up with data/filesystem corruption... I have not had this occur due to swap being turned off this century.[1] The typical worst case scenario is that either due to a runaway process or accidentally overloading the system for a indefinite period of time, the system goes completely unresponsive so you have to perform a hard reboot. This happens to me 1-2 times a year but I have yet to loose any data (and if it ever does, that's one of the many reasons to have backups) The more common scenario is that a runaway process will OOM briefly, when it can't allocate the memory it needs crash, and everything is back to normal within a couple of seconds.[2]

FWIW, mobile (Android and iOS) devices typically run without swap just fine. IMO, this 'always enable swap' is bad advice and dates from a time when very bad things could and did happen if you OOM'd a system. The marginal performance improvement from freeing up on the order of megabytes (or even tens of megabytes) of RAM on a system with double digit GB's of RAM makes absolutely no sense to me. It makes even less sense on SSD systems where you would be unlikely to notice a chronic swapping situation until you had severely shortened the life of your SSD.

[1] This would be a good reason to not turn off swap on a production database server where it is more likely to be an issue, for example.

[2] One of the dubious advantages to 'modern' programs not handling errors like OOM is that, since they don't check for it, they can't handle when it happens and just crash. This was actually more of a problem in the 90's and earlier when it was more common for applications to handle OOM situations which could actually make the problem worse since they would often hang around waiting for any memory to be freed and use it which resulted in a persistent OOM situation.

Re:Almost Always Add Swap Space by mixmastamyk on Friday July 19, 2019 @12:27PM (#20480013)
I haven't used swap for decades now. Does it make sense for you? Depends on whether you exceed the physical amount of RAM in your machines often. Personally only when a bug occurs, and I’d rather the process was killed.
Re:Almost Always Add Swap Space by stevekemp on Friday July 19, 2019 @12:28PM (#20480034)
"It depends".

I'd suggest trying it and seeing what happens, if you wish to learn rather than listen to people who aren't using your host - and don't know what you run..

Re:Almost Always Add Swap Space by serf on Friday July 19, 2019 @11:49AM (#20479608)
in a sane OS? Nothing until you run out of ram.

In a Microsoft OS? Depends on the app -- some are dependent (many games) on having swap available (and may err silently or under false pretenses without one).

I've found that a lot of MMO style games include cheat detection systems (see : rootkit) that rely on swap, and fail without it. As to why they do that, i'm unsure.

Re:Almost Always Add Swap Space by Waterluvian on Friday July 19, 2019 @11:53AM (#20479635)
Fascinating. Maybe just as a part of the fingerprinting mechanism?
Re:Almost Always Add Swap Space by devoply on Friday July 19, 2019 @11:45AM (#20479575)
Out of memory processes will be terminated automatically with only a message in the kernel log, i.e. dmesg.
Re:Almost Always Add Swap Space by Beldin on Friday July 19, 2019 @11:58AM (#20479697)
What happens in case you have swap space and run out of memory?
Re:Almost Always Add Swap Space by pjc50 on Friday July 19, 2019 @03:23PM (#20481682)
Your system thrashes to a halt, until eventually you run out of swap as well and the OOM killer is activated anyway. It just takes longer and is preceded by a period of dismal performance.
Re:Almost Always Add Swap Space by Dylan16807 on Saturday July 20, 2019 @01:08AM (#20484290)
Even without swap, you get a period of thrashing and dismal performance as your disk cache is squeezed to nothing.

If you have swap, this happens later and more gradually, and lasts longer before the OOM killer kicks in.

Re:Almost Always Add Swap Space by devoply on Friday July 19, 2019 @12:03PM (#20479745)
some stuff in your memory gets written to the swap page.

https://en.wikipedia.org/wiki/Paging

Re:Almost Always Add Swap Space by xorcist on Friday July 19, 2019 @06:46PM (#20483071)
Of course you can, but performance can be a bit lower since less memory will be available for file system buffers. Most mainstream applications isn't very careful with their memory usage. It will depend on your use case.
Re:Almost Always Add Swap Space by paulcarroty on Friday July 19, 2019 @12:10PM (#20479823)
Of course, in any case if you have enough RAM for your tasks.
Re:Almost Always Add Swap Space by JustSomeNobody on Friday July 19, 2019 @12:29PM (#20480052)
Dunno. Do you care about hibernating your system?
Re:Almost Always Add Swap Space by jschwartzi on Friday July 19, 2019 @01:36PM (#20480757)
Not OP, but I don't. Hibernating and Sleeping my systems always causes massive problems for me, and I'd rather just shut down and start up really fast.

For example on my current laptop if I hibernate or sleep the system the USB fails to initialize in Windows until I power-cycle. I think this is mainly due to me having about 10 or 15 USB devices plugged in via hubs and USB ports. But that's not the norm.

I've also got 64 GB of ram and a pretty decent pair of NVMe SSDs so read speed isn't really a limiting factor for startup time.

Re:Almost Always Add Swap Space by JustSomeNobody on Friday July 19, 2019 @02:14PM (#20481108)
I agree, hibernate isn't the most stable thing in world. My work machine is a Windows laptop with a regular ssd so I usually hibernate it and about 1 in 20 times it forgets either BT, network card or wifi. I have a NUC8 running ubuntu for one of my personal machines and have a NVMe and I shut that down because, as you mentioned, the speed is there.
Re:Almost Always Add Swap Space by pfranz on Friday July 19, 2019 @04:33PM (#20482212)
Recent Windows seems especially horrible about this. Personally, my workstation seems to forget which output to send video to and fails to properly wake up my USB wifi adapter. A tech I recently spoke to said, "Windows 10? Yeah, it seems to have way more problems. I just suggest disabling sleep."
Re:Almost Always Add Swap Space by LargoLasskhyfv on Friday July 19, 2019 @12:54PM (#20480337)
Swap is good. If it is in RAM. Just use zram. Like so:

https://github.com/armbian/build/blob/master/packages/bsp/co...

zramctl output from a tiny NanoPiNeo2 with 1GB Ram, running from a 64GB Sandisk SDXC card:

NAME ALGORITHM DISKSIZE DATA COMPR TOTAL STREAMS MOUNTPOINT

/dev/zram1 lzo 496.7M 4K 76B 12K 4 [SWAP]

/dev/zram0 zstd 50M 2.5M 449K 832K 4 var/log

Edit: Formatting (useless)

Re:Almost Always Add Swap Space by LinuxBender on Friday July 19, 2019 @04:17PM (#20482082)
Almost never add swap. [1]

[1] - https://github.com/ohdns/sysctl_and_thp_test

Re:Almost Always Add Swap Space by yellowapple on Friday July 19, 2019 @04:25PM (#20482133)
One thing the article doesn't mention for those of us who run Linux on laptops (There are dozens of us! Dozens!) is that having a swap partition at least as big as one's physical RAM (if not bigger) is mandatory for hibernation. Linux (like any OS kernel that supports hibernation) needs to save the current contents of RAM somewhere, and (last I checked) it'll only work with swap partitions (swap files don't work for some reason).
Re:Almost Always Add Swap Space by zzzcpan on Friday July 19, 2019 @12:59PM (#20480396)
What I agree is that these days you should still use swap, but swap should always be either on zram or on zswap if you can't use zram for some reason, have too little ram, can't load zram module or something. Raw swap on disk is just not acceptable anymore for performance, security, wearout reasons.
Re:Almost Always Add Swap Space by gnode on Friday July 19, 2019 @01:10PM (#20480505)
> performance, security, wearout reasons

If memory pressure is low, and the swap is used to page out rarely / never used pages in favour of more disk cache, then there is a performance gain to be had, and wear should be minimal.

You can encrypt swap with a randomly generated key to aid security.

Ideally memory compression and disk swap would be used together; there's a gain to be had by shrinking lesser used pages, but no sense in taking up any space in RAM for pages showing no sign of life.

Re:Almost Always Add Swap Space by zzzcpan on Friday July 19, 2019 @07:44PM (#20483319)
More vfs cache for swap space is not a trade off here, but rather a mostly incorrect oversimplification. The value of more cache is very non linear, it depends on the workload and usually only matters when nearly all of the disk I/O is already exhausted, the exact situation you want to prevent by using more of the same disk I/O for swap. Essentially trading more cache for abrupt performance degradation in terms of latency and responsiveness. I mean, sure, if you can't use zram, say you are on a system that's a bit behind on performance features, like freebsd, very light swappiness might be ok, might free up some of very unused memory and just not be aggressive enough to swap something that causes latency, responsiveness problems, although this is not certain. With memory compression you can swap and compress much more of it more aggressively and be certain that no abrupt performance degradation can happen. It feels like having more ram that's gradually affecting tail latency the more you use it.
Re:Almost Always Add Swap Space by Dylan16807 on Saturday July 20, 2019 @01:25AM (#20484342)
Zram is nice when you're barely using it, but every time I've seen it get low on memory it degrades quite abruptly, much worse than ssd swap.
Re:Almost Always Add Swap Space by Dylan16807 on Saturday July 20, 2019 @01:18AM (#20484317)
That really depends on what you're doing.

In my experience, when Chrome is being a memory hog I can swap about 2/3 of its memory out with zero performance impact. As long as I can swap that away, I never run out of memory. If I can't swap it to disk, I have problems. I don't think that qualifies as "too little ram".

So in this case performance is fine, security should be an issue of encryption not disabling, and the SSD is not going to wear out because of an extra <1% average utilization.

Re:Almost Always Add Swap Space by paulcarroty on Friday July 19, 2019 @12:16PM (#20479890)
Swap doesn't make difference, just a parachute for your apps: give some work guaranties, but make things slower & eats resources of SSDs.

I'm prefer add RAM as much my tasks needs it.

Re:Almost Always Add Swap Space by mnw21cam on Friday July 19, 2019 @01:06PM (#20480476)
Without swap, your server could potentially become suddenly unresponsive, as memory pressure forces all your running code out. The system will still be thrashing, and even worse than it you did have swap. If you do have swap, then you get a slightly more graceful degradation, and hopefully a chance to fix the problem first.

What you really want is to never have a severe out of memory situation, and you can do that either by getting enough RAM that you never use it all, or by killing stuff nice and early when memory starts getting low.

Re:Almost Always Add Swap Space by paulcarroty on Friday July 19, 2019 @02:07PM (#20481040)
> Without swap, your server could potentially become suddenly unresponsive

The server never be unresponsive 'cause lack of swap. Don't like the logic "we just add a swap and maybe it will be slower a bit, but still working".

Re:Almost Always Add Swap Space by IHLayman on Friday July 19, 2019 @11:35AM (#20479488)
For another perspective, the Kubernetes community has had a long discussion about using swap space over the course of the last 2 1/2 years, as you can see in this feature thread: https://github.com/kubernetes/kubernetes/issues/53533 TL;dr: As of version 1.8 any system running k8s has to disable swap space or formally mark in the kubelet that you are turning the check for it off. There are good arguments on both sides of this issue, and the discussion is still continuing.
Re:Almost Always Add Swap Space by everybodyknows on Friday July 19, 2019 @01:13PM (#20480529)
OP serves up animated ads directly from the origin URL -- unblockable by umatrix. Very sophisticated indeed.
Re:Almost Always Add Swap Space by lone_haxx0r on Friday July 19, 2019 @12:49PM (#20480282)
Is it true that swap reduces the life expectancy of SSDs by writing too frequently?
Re:Almost Always Add Swap Space by JustSomeNobody on Friday July 19, 2019 @02:19PM (#20481152)
I think this used to be the case with older SSDs, but newer ones are able to survive much longer.
Re:Almost Always Add Swap Space by sgt on Friday July 19, 2019 @11:29AM (#20479408)
Learned about atop. Wish it existed on macOS too.
Re:Almost Always Add Swap Space by IHLayman on Friday July 19, 2019 @12:52PM (#20480309)
Also check out glances: https://nicolargo.github.io/glances/ A good terminal display overview of processes, network, disk, and battery.
Re:Almost Always Add Swap Space by jihadjihad on Friday July 19, 2019 @11:57AM (#20479677)
I've been using htop for a while now, it is pretty great: https://hisham.hm/htop/
Re:Almost Always Add Swap Space by sys_64738 on Friday July 19, 2019 @12:20PM (#20479933)
SSD V Spinner? Metal V VM?