Probably lack of integration; if NT hits a memory issue, it can just pass notice to the tightly-coupled userland and GUI. If Linux runs out of memory, even if it internally knows what process to blame... What would it do that makes sense for a headless server, TiVo, and Android phone? Keeping in mind that the kernel folks don't even work that closely with many userspace vendors.

OSX handles this with a kqueue event that can notify userland when the system moves between various memory pressure states; this is hooked into by libdispatch and other userland libraries which will discard caches and so on.

I don't see why Linux couldn't do the same; open /sys/kernel/something and epoll on it.

This already exists: applications can receive memory pressure events (such as the system reaching "medium" level, where you may want to start freeing some caches) via /sys/fs/cgroup/memory/.../memory.pressure_level. See https://www.kernel.org/doc/Documentation/cgroup-v1/memory.tx....

The first two nota benes explicitly describe this document being outdated and not what most people expect when it comes to “memory controller”. I am not certain that citing this is a great example.

What about this? Seems to do what they want.

https://serverfault.com/a/949045

Windows (server and desktop versions) will throw up a message dialog on the screen. It will also start to kill off processes just enough to resolve the low memory situation.

During this - unlike Linux - you can actually use the mouse, CLI and close programs yourself.

On top of that server applications like IIS has built-in watchdogs. If an IIS process grows to use too much memory (60% IIRC) or excessive CPU, the watchdog will recycle the process.

I think Windows kernels do not use overcommit, so memory allocation will fail if you run out of memory.

You could use the message bus to post a message to the service that handles out of memory decisions, which in turn could either

1. Show a GUI with a choice

2. Show a message on the current terminal and ask what to do

3. Just return "kill it now" if there is no interactive session

And if there is no such service, just default to 3. The problem really is that the state cannot be captured and communicated to the user. I doubt the NT kernel itself shows a GUI window, it's probably a service that gets woken up by a kernel exception, which in turn shows the window. Basically, the Linux kernel needs more pluggable functionality for user interactions. It's absolutely fine and even recommended to not have an entire GUI in the kernel, it needs to just provide a mechanism for userspace to capture the event and decide what to do with it.

Throw a signal like it would do if the process were out of memory completely and about to be killed? (for clarification, no snark intended, actual question)

What signal is sent when a process is out of memory? I thought malloc would either start returning NULL or you’d fault when trying to access overcommitted memory.

Yes, but ideally you want to be throwing some ‘memory pressure’ signals before absolutely running out of memory, so that programs can take simple actions like emptying caches, etc.

Catching an otherwise-fatal out-of-memory fault and recovering would be too complicated / bug-prone.

Android sends low memory events and kills processes based on heuristics.

Then again, Android has a customized Linux kernel.

This describes my general Linux experience well: A very stable kernel, with which I never had serious issues on a headless server. But applications in the userspace (apart from the standard GNU packages) are usually a tossup anywhere between system-crashing garbage and perfectly working pieces of software.

I used to run into this problem all the time in grad school. Once a month or so I'd load a data set, do some dumb Python operation on it that took significantly more memory than I predicted, and BAM! I'd have to restart my laptop.

I just kinda assumed that's how computers worked until I got a Mac a couple of months ago...

The link suggests that there might be some default parameters you could change to protect against this behavior. Does anyone have any suggestions on what settings to change?

A Mac is certainly better at handling these kinds of issues but it's by no means totally safe. It tries to compress memory and dynamically allocate more swap, but there's still a limit and you can see that if you accidentally run programs with way higher RAM requirement than you have. I've had multiple occasions where my program used so much RAM that even moving the cursor is an exercise in patience, never mind switching to a terminal window and typing commands to kill the process.

A Mac will keep creating virtual memory swap up to some limit (some multiple of the amount of physical RAM — can't quite remember, possibly 5x) and then it will produce a kind of vague dialog box saying "You've run out of application memory" with a list of applications to force quit.

But at least you can recover rather cleanly from the issue.

If you've foolishly decided to run without swap (like the original post), then suspending the offending program does nothing.

This is because the offending program has allocated a lot of private dirty pages, which can't be dropped from memory because without swap space, there is nowhere for it to go.

Linux use cases tend to be servers where user interaction is unexpected at 3AM? No one around to make a choice, so automate a choice.

IMO despite the standout behavior, I prefer my software to deal with itself.

Systems designed to wait for user input end up having design choices intent on keeping a user using them.

Software is just a tool. Not a lifestyle. Set and forget this shit as much as possible

If the alternative is simply killing a process or crashing the kernel, then surely a better approach would be to suspend the offending process and call a handle that does something. If you want that something to restart the machine, fine. You want it to notify the administrator, fine.

There is the use case of android phones. One of the answers to the OP is about that case. It sends that Google developed a user space process to monitor those events https://lkml.org/lkml/2019/8/5/1121

From that reply it seems that Facebook implemented something similar, I guess for their servers.

> Linux use cases tend to be servers where user interaction is unexpected at 3AM? No one around to make a choice, so automate a choice.

Even if it's a small percentage of the overall "computing" population, there are still millions of people running Linux on the desktop (roughly 2% out of 3.2 billion people using internet makes for 64 million - a large european country). It's 64 million of people for which this behaviour is a pain in the arse.

Does FreeBSD handle this issue better?

And you've turned swap off on Windows?

Windows will not BSOD due to memory pressure.

I think it happens only if you have a broken device driver, applications cannot cause BSOD with memory allocation.

Well I beg to differ, using a unlimited swap file can quickly reach hard issues after 64gb of swap use. At that point mallocs in the windows ui fail (timouts or something?), that apparently are not meant to, eg fonts from shutdown menu missing, the system being unable to shutdown ect.

Does it BSOD if it runs out of swap space though?

I'm not sure but the assumption might be that there's generally no user to ask as the computer might be a server.

Right but if there is a user to ask then it should ask!!

Ehm, and how does the OS know which is the “offending” process?

I think you are confusing the issue raised here with your desktop experience.

Currently the Linux kernel computes a score for each process based on some heuristics. There's a good introductory article on LWN:

https://lwn.net/Articles/317814/

Yep and it's about as good as just picking a random process and killing it.

It's awesome when you run out of memory and you try to log in only to have it kill sshd.

A classic from [1]:

> An aircraft company discovered that it was cheaper to fly its planes with less fuel on board. The planes would be lighter and use less fuel and money was saved. On rare occasions however the amount of fuel was insufficient, and the plane would crash. This problem was solved by the engineers of the company by the development of a special OOF (out-of-fuel) mechanism. In emergency cases a passenger was selected and thrown out of the plane. (When necessary, the procedure was repeated.) A large body of theory was developed and many publications were devoted to the problem of properly selecting the victim to be ejected. Should the victim be chosen at random? Or should one choose the heaviest person? Or the oldest? Should passengers pay in order not to be ejected, so that the victim would be the poorest on board? And if for example the heaviest person was chosen, should there be a special exception in case that was the pilot? Should first class passengers be exempted? Now that the OOF mechanism existed, it would be activated every now and then, and eject passengers even when there was no fuel shortage. The engineers are still studying precisely how this malfunction is caused.

[1] https://lwn.net/Articles/104185/

Fortunately, engineers have invented a way to attach a Strolling Wheelbarrow After Plane, where you can stash the sleeping passengers without ejecting them out of the plane entirely. This has the unpleasant side effect of slowing down the journey for everyone when passengers wake up inorderly (and God forbid everyone wake up at the same time), though.

What I still do not understand is why people continue to turn a blind eye to this instead of switching to SmartOS. I just don't get it.

How does Solaris/SmartOS handles that situation?

It doesn't get in that situation, because malloc() can return null on Solaris (i.e. it never¹ overcommits).

While in general I think this is vastly better than the somewhat insane Linux OOM killer, you can get in awkward situations where you can't start any processes (including a root shell) because you're out of memory.

I rather like the FreeBSD solution to this, which is to not overcommit, but after a certain number of allocation failures it kills the process using the most memory. This prevents situations where you can't start any processes.

There's no one-size-fits-all solution to handling low memory conditions, but the Linux solution manages to almost never do what you want which is kind of impressive in a way.

¹ I seem to recall hearing somewhere that you can allow allocations to overcommit on a per-application basis on later versions of Solaris, but don't quote me on this.

> FreeBSD solution to this, which is to not overcommit

Where did this myth come from? Did y'all just assume that the vm.overcommit sysctl actually makes sense and zero means "no overcommit"? :)

https://news.ycombinator.com/item?id=20623919

But indeed, OOM killer kills the largest process, which makes more sense in most scenarios than Linux's "badness" scoring.

Huh, I had no idea it worked like that. That's bizarre.

Running sshd as an on-demand (Type=socket) service would probably work better, since then the sshd process would be new and thus have a better heuristic score - also not be tying up memory sitting unused in the meantime.

systemd still seems to run it (Type=notify) with the -D option all the time though, at least on the systems I can check.

Dropbear is configured by default as a Type=socket service though.

This is sort of just kicking the problem down the road. Your idea actually might work for (presumably) low-volume use ssh, but what about the next important service? When does the work-around to a papered-over work-around to a virtual problem that is supposed to just be RAM-backed or handled at

  ptr = malloc(42);
  if(!ptr) exit_error();

end?

Well, there probably needs to be a way to override the heuristic at least, sort of a 'this process is important, don't auto-kill it if trying to find memory'.

As for ssh specifically, I rarely ssh into my desktop machine, but I keep sshd running for just this kind of situation where I might need to try and rescue a swamped machine. So in most cases low-volume sshd use is exactly what is called for.

If you're running into the memory purge of doom on a server that's probably a whole different nightmare scenario.

malloc returning NULL has been a broken assumption for a long time though, and that isn't going to change afaik.

I blame the specific algorithm for that, not the basic concept. Nothing with less than 10MB of memory use should ever get killed unless you're in some kind of fork bomb.

Probably it would be good to include the amount of overcommit in the heuristic. Processes that overcommit should be killed first. What is a good way to measure/inspect overcommit of the process?