News for hackers. Stuff that matters.
Re:In C, how do you know if the dynamic allocation succeeded? by convolvatron on Friday November 05, 2021 @04:05PM (#29124134)
wouldn't you just account the COW pages against the parent until they are copied?

kicking the can down the road means there isn't any longer a reasonable correction (failing the allocation), but instead we get to drive around randomly trying to find something to kill.

this is particularly annoying if you are running a service. there is no hope for it to recover - for example by flushing a cache. instead the OS looks around - sees this fat process just sitting there, and .. good news, we have plenty of memory now.

Re:In C, how do you know if the dynamic allocation succeeded? by pcwalton on Friday November 05, 2021 @06:19PM (#29125319)
> wouldn't you just account the COW pages against the parent until they are copied?

That's what Linux does. But how do you return ENOMEM when the copy does happen and now the system is out of memory? Memory writes don't return error codes. The best you could do is send a signal, which is exactly what the OOM killer does.

Re:In C, how do you know if the dynamic allocation succeeded? by lokedhs on Saturday November 06, 2021 @04:42AM (#29128462)
That's why you ensure you have enough swap to cover the worst case scenario.

You as swap not because you need to actually use it, but on order to be able to guarantee there is enough memory available if the worst case scenario happens. In normal circumstances, swap should never really be utilised.

Re:In C, how do you know if the dynamic allocation succeeded? by Hendrikto on Saturday November 06, 2021 @05:51AM (#29128865)
> You as swap not because you need to actually use it […] In normal circumstances, swap should never really be utilised.

Swap is actually there so anonymous pages can be evicted, and is often used long before there is memory contention, during normal operation.

Not having swap means that only file-backed pages can be evicted. During memory contention, this can cause thrashing. During "normal" operation, it degrades performance.

Swap only being used when memory runs out, as kind of "emergency RAM", is a very widespread misunderstanding.

Re:In C, how do you know if the dynamic allocation succeeded? by vbezhenar on Saturday November 06, 2021 @05:47AM (#29128844)
> In normal circumstances, swap should never really be utilised.

I don't agree. There're plenty of dormant virtual memory pages which will never be used. Keeping them in RAM is wasting precious resources.

Re:In C, how do you know if the dynamic allocation succeeded? by pjc50 on Friday November 05, 2021 @04:31PM (#29124414)
But what happens when the kernel needs to copy one and has run out of memory? You still get a random process killed.

(I note that Windows has a different approach, with "reserve" vs "commit", but nobody regards that as a preferential reason for using Windows as a server OS)

Re:In C, how do you know if the dynamic allocation succeeded? by magicalhippo on Friday November 05, 2021 @08:09PM (#29126071)
> But what happens when the kernel needs to copy one and has run out of memory?

Don't allow it. Fork the process with read only pages except for the ranges passed to fork(). Count read-write pages as used memory by the child process.

If the forked process wants to write to a page that's read-only it'll have to do a system call to turn it read-write. That call can then fail if there's not enough free memory to copy the pages.

Of course, kinda hard to fix this now...

Re:In C, how do you know if the dynamic allocation succeeded? by tsimionescu on Saturday November 06, 2021 @01:06AM (#29127676)
> Don't allow it. Fork the process with read only pages except for the ranges passed to fork().

This doesn't work because of the horrible fork/exec design. If I am a huge process and I want to run `ls`, I will first have to clone myself using fork(), which may trigger an OOM, even if the first action of my clone would have been exec(), ignoring all of that memory.

I am always surprised that no one has added a sane 'spawn process' primitive to replace fork/exec. Especially since fork() without exec() only really works in single-threaded processes.

Re:In C, how do you know if the dynamic allocation succeeded? by naniwaduni on Saturday November 06, 2021 @01:22AM (#29127739)
We do have posix_spawn &c., which are expressly designed to be implementable even in situations where fork doesn't make sense.
Re:In C, how do you know if the dynamic allocation succeeded? by tsimionescu on Saturday November 06, 2021 @01:50AM (#29127857)
Interesting, I'd never learned about that. If I understand correctly, this actually used to be implemented as a fork()+exec() on many systems, though. It's actually vfork() that addressed the problem and allowed an actually useful implementation of posix_spawn in Linux at least.
Re:In C, how do you know if the dynamic allocation succeeded? by toast0 on Friday November 05, 2021 @08:49PM (#29126370)
You'd presumably need the parent to also do a system call to mark pages read-write after forking, and that would need to include the stack pages, and that sounds like lots of fun; especially since sometimes library functions fork (although most of that is to fork/exec and more specific apis for that exist now). Not that it isn't loads of fun anyway, but woe betide thee who forks and threads and makes the other threads have a read-only stack.

Probably you'd get people just marking as read-write and returning in sigsegv handlers, which I'm sure has great security properties... OTOH, at least there's an opportunity to deny the remap in the handler and get a decent crashdump from the program or a sliver of hope for managing the situation.

Re:In C, how do you know if the dynamic allocation succeeded? by pjc50 on Saturday November 06, 2021 @10:06AM (#29130302)
> Of course, kinda hard to fix this now..

Well, yes, if you can break backwards compatibility you can do anything. Except run all the existing software.

Re:In C, how do you know if the dynamic allocation succeeded? by enedil on Friday November 05, 2021 @08:35PM (#29126282)
That also means that if you used seccomp to restrict mprotect syscall, the child process may never ever write again, which is an unfortunate design choice.
Re:In C, how do you know if the dynamic allocation succeeded? by magicalhippo on Friday November 05, 2021 @09:19PM (#29126601)
Seccomp can check for the PROT_EXEC flag though, and let the others pass, no?
Re:In C, how do you know if the dynamic allocation succeeded? by trhway on Friday November 05, 2021 @04:31PM (#29124415)
>there is no hope for it to recover - for example by flushing a cache. instead the OS looks around - sees this fat process just sitting there, and .. good news, we have plenty of memory now.

Overcommit was godsent in the times of expensive memory and when people used virtual memory on disk (so it will spill low use memory pages there instead of the kill). Of course these days with abundance of cheap memory and people not configuring virtual memory any more we get the situation you describe.

Re:In C, how do you know if the dynamic allocation succeeded? by angry_octet on Friday November 05, 2021 @11:56PM (#29127384)
Virtual memory is always on... If you actually hit swap the system effectively deadlocks anyway because the swap daemon is too dumb by being too fair. Persistent memory might change this but it needs so much work.
Re:In C, how do you know if the dynamic allocation succeeded? by trhway on Saturday November 06, 2021 @02:45AM (#29128046)
it was a throwback down the memory lane and the years before my first Linux in 1995 i had been dealing with Windows where configuration of swap was "configuring virtual memory", so i automatically used that term.

> If you actually hit swap the system effectively deadlocks anyway

that depends. In many cases in the past the options would be either with swap and thus slow or pretty much not at all. And again these days there is so much memory that there is always a way to avoid the swapping. Though i've met funny situations in recent years like when a several terabyte sized database process would get killed by the OOMKiller on a machine with overcommit left on and no swap configured.

Re:In C, how do you know if the dynamic allocation succeeded? by angry_octet on Tuesday November 09, 2021 @09:14AM (#29161969)
OOM kill exists because swap is not being useful. A process failure is something you can deal with, livelock isn't. If you have an algorithm that works out of core then you'll never hit swap because you maintain the working set below memory limits.
Re:In C, how do you know if the dynamic allocation succeeded? by secondcoming on Friday November 05, 2021 @04:13PM (#29124245)
Indeed. It's totally dumb that the OS is allowed lie to you when you've asked for some resources. And because of this behaviour people no longer check for success from malloc(), etc, because of laziness. It's a bad situation.
Re:In C, how do you know if the dynamic allocation succeeded? by joosters on Friday November 05, 2021 @04:30PM (#29124404)
Please don’t blindly declare it ‘totally dumb’. If you disallow overcommit, you can end up with a system that can’t fork and run /bin/true, even if there are gigabytes of memory left.

Both styles of memory allocation have their uses, and their drawbacks, but please understand them before declaring many OS designers as stupid and dumb.

Re:In C, how do you know if the dynamic allocation succeeded? by chrsig on Friday November 05, 2021 @08:42PM (#29126333)
I agree with you on all counts, but it's worth highlighting that the parent did not call the designers dumb, but rather the situation and implementation.

I know it can seem like a distinction without a difference, but I think it's fair to critique work. I think they could have been more articulate and considerate towards the designers, and have some empathy that many people have worked very hard on it, and did their best in the problem/solution space they were working with.

I just think it's important that people stay objective on if the quality of the people themselves is in question. It's not ok if it is, but I don't think it was here.

now people that don't check how malloc() returns being blanketly labled as lazy is an example of it being about people. people don't ignore the possibility of a nullptr return from malloc because they're lazy. They ignore it because it's hard -- even if you do catch it, there's very little that you can actually do. You can't dynamically allocate...so I hope you've got enough stack space to do what you've gotta do. And even then, you have to be able to propagate that there's no memory all the way up the stack as it unwinds...and check every single allocation.

The cpp world is mildly better in that it can throw std::bad_alloc...but if you have something that winds up doing an allocation in a destructor, I imagine that's not a fun time.

Most of the time, there's not really a better thing to do than crash -- and there's not a lot of incentive to put any work into it. It's not something that should be happening on any sort of regular basis.

Re:In C, how do you know if the dynamic allocation succeeded? by andi999 on Wednesday November 17, 2021 @02:34AM (#29250123)
But don't you need to check the return value of malloc to then reliable crash/terminate the program? It is like if your car runs out of gas you need to stop, but you don't abandon the steering wheel at high speed and hope for the best.
Re:In C, how do you know if the dynamic allocation succeeded? by giomasce on Friday November 05, 2021 @05:33PM (#29124951)
I suppose vfork might help with that (though I don't understand why they don't directly add fork_and_execve, it would seem much easier).

Also, the problem with Linux is not having overcommit, but notv being able to choose when to overcommit and when not. Windows makes that easier, AFAIU.

Re:In C, how do you know if the dynamic allocation succeeded? by gpderetta on Friday November 05, 2021 @05:42PM (#29125025)
They do. Man posix_spawn. It comes with its own dsl to be able to support a small subset of all operations that are often performed between fork and execv.

Vfork is usually a better solution.

Edit: and yes, I would love to be able to disable overcommit per process.

Re:In C, how do you know if the dynamic allocation succeeded? by scottlamb on Friday November 05, 2021 @08:00PM (#29126023)
> Vfork is usually a better solution.

I think the biggest downside to vfork is (from the Linux manpage) "the behavior is undefined if the process created by vfork() [...] calls any other function before successfully calling _exit(2) or one of the exec(3) family of functions."

So strictly speaking I think doing any of those operations documented by posix_spawn yourself between vfork and execv is undefined behavior. In practice, I believe it's fine, and on glibc posix_spawn is apparently written in terms of vfork, but libc is allowed to make assumptions that portable programs shouldn't.

Story time: I once worked on a Unix-based system with no MMU. The fork implementation did something insane: it looked for things that were possibly pointers (four-byte aligned memory locations which can be interpreted as valid physical memory locations on this system allocated to that program) and adjusted them. It was kind of like a conservative GC in that it treated anything that looked like a pointer as if it were a pointer. But no reasonable person would call modifying memory that may or may not be pointers to be "conservative". Amazingly, it worked most of the time, but sometimes strings got corrupted, and as a workaround there were a bunch of places where string buffers were fully zeroed where just a NUL byte would otherwise do.

This was early in my career, and I didn't design the system anyway. If I were working on it today I'd remove the fork implementation and make everything use vfork and/or posix_spawn instead. Apparently that's what posix_spawn was made for.

Another weird problem with the lack of MMU: the compiler also didn't use register-based addressing (what's the term? like position-independent code but for the data segment?), so global variables were truly global, not just global to the process. A bunch of code needed to be "deglobalized" to deal with this. I wanted to improve the compiler, but long story short I got offered another job first.

Re:In C, how do you know if the dynamic allocation succeeded? by pcwalton on Friday November 05, 2021 @08:14PM (#29126118)
No-MMU Linux usually just disables fork and uses vfork instead. The really annoying part is that you can't use shared libraries without doubling the size of function pointers (FD-PIC ABI).
Re:In C, how do you know if the dynamic allocation succeeded? by scottlamb on Friday November 05, 2021 @09:10PM (#29126529)
> The really annoying part is that you can't use shared libraries without doubling the size of function pointers (FD-PIC ABI).

Interesting, thanks. That didn't come up on our system—not only did we not use shared libraries but we also linked the whole system into one binary, kernel and all. Link times were atrocious in combination with identical code folding and big VLIW sentences, but it worked.

Re:In C, how do you know if the dynamic allocation succeeded? by OldHand2018 on Friday November 05, 2021 @05:53PM (#29125108)
This paper [1] argues that fork needs to be deprecated in favor of posix_spawn() or other more modern solutions. It claims that, among many other reasons, that fork encourages overcommit and that programs such as Redis are extraordinarily constrained if overcommit is disabled - because of fork.

[1] https://dl.acm.org/doi/10.1145/3317550.3321435

Re:In C, how do you know if the dynamic allocation succeeded? by jart on Saturday November 06, 2021 @01:45AM (#29127835)
> They do. Man posix_spawn. It comes with its own dsl to be able to support a small subset of all operations that are often performed between fork and execv.

In other words, you write your C code inside a C string instead of in C.

Re:In C, how do you know if the dynamic allocation succeeded? by joosters on Friday November 05, 2021 @05:49PM (#29125077)
It’s a weakness of the fork()+exec() model, for sure. However, creating a fork_and_execve() API is extremely tricky. Just think of all the innumerable setup options you would need to give it, e.g. what file handles should be closed or left open? What directory should it start in? What environment variables should be set - or cleared? And on and on and on…

the flexibility of a separate fork() then exec() means you can set up the initial state of a new process exactly as you want, by doing whatever work is needed between the two calls. If you merge them into one, then you will never be able to encapsulate all of that.

Re:In C, how do you know if the dynamic allocation succeeded? by wmf on Friday November 05, 2021 @08:20PM (#29126165)
A third option would be the ability to create a blank process, configure it, then exec within it. I think Mach allows this.
Re:In C, how do you know if the dynamic allocation succeeded? by saagarjha on Friday November 05, 2021 @07:14PM (#29125744)
I mean, posix_spawn exists. It's a messy function, but its job is messy for exactly the reasons you describe. (FWIW, there are very few things you can legally perform between fork and exec.)
Re:In C, how do you know if the dynamic allocation succeeded? by marcan_42 on Friday November 05, 2021 @09:47PM (#29126793)
Are there any things that are illegal between fork and exec? It is perfectly legal to exec whenever you want, and it it perfectly legal to fork whenever you want. I'm not aware of any requirements around the fork/exec sequence.
Re:In C, how do you know if the dynamic allocation succeeded? by tsimionescu on Saturday November 06, 2021 @01:12AM (#29127702)
For a single-threaded process, sure. But in a multi threaded context, almost any operation done in the child after fork() can royally mess up the system. There are many versions of libcs where malloc() after fork() is likely to deadlock between the parent and child processes, as they share the internal malloc() locks.
Re:In C, how do you know if the dynamic allocation succeeded? by marcan_42 on Saturday November 06, 2021 @10:31PM (#29136001)
Ah, yes, multithreading. That always interacts poorly with old UNIX APIs...
Re:In C, how do you know if the dynamic allocation succeeded? by spacechild1 on Saturday November 06, 2021 @05:44AM (#29128828)
From the fork() man page:

“After a fork() in a multithreaded program, the child can safely call only async-signal-safe functions (see signal-safety(7)) until such time as it calls execve(2)“

For example, you can't do any dynamic memory allocation or call a function that may allocate.

Re:In C, how do you know if the dynamic allocation succeeded? by jart on Saturday November 06, 2021 @01:58AM (#29127874)
There's no restrictions in that sense. However the computer might not do what you intended if you were to do something like fputc('x',stdout);fork();fflush(0);
Re:In C, how do you know if the dynamic allocation succeeded? by nextaccountic on Friday November 05, 2021 @11:25PM (#29127238)
I think you mean between vfork and exec. The Linux manpage says:

Standard description

(From POSIX.1) The vfork() function has the same effect as fork(2), except that the behavior is undefined if the process created by vfork() either modifies any data other than a variable of type pid_t used to store the return value from vfork(), or returns from the function in which vfork() was called, or calls any other function before successfully calling _exit(2) or one of the exec(3) family of functions.

Re:In C, how do you know if the dynamic allocation succeeded? by jart on Saturday November 06, 2021 @02:00AM (#29127883)
Note that modern POSIX recommends just calling fork() because "[vfork] was previously under-specified". The glibc man pages shouldn't say undefined since that's too strong a word. There is so much you can actually do in practice during vfork(). Pretty much all system calls are just as safe. It's mostly a question of whether or not your userspace libraries might clobber some global, due to how vfork puts the process in a state where implementation details that are normally private become part of the public surface area of these apis, and not many library authors are disposed to offer those kinds of assurances.
Re:In C, how do you know if the dynamic allocation succeeded? by bitwize on Friday November 05, 2021 @09:05PM (#29126494)
That's just the thing. fork(2), like much of POSIX, was a mistake.

The proper semantics for starting a new process is something like posix_spawn or Win32 CreateProcess, i.e., you specify an executable image to start.

Re:In C, how do you know if the dynamic allocation succeeded? by gnubison on Saturday November 06, 2021 @11:10AM (#29130845)
It’s not like you can blame POSIX for this — they had to standardize existing behavior.
Re:In C, how do you know if the dynamic allocation succeeded? by jagrsw on Saturday November 06, 2021 @04:12AM (#29128337)
> The proper semantics for starting a new process is..

:) Ah.. I guess when "I was younger" (TM) I was also so dogmatic on most things computer-related.

Anyways, when we need to do something a bit more complex than equivalent of system(), then it quickly becomes evident, that in many cases we need to prepare ourselves for the future execve().

Here's the list of syscalls/c-funcs, which are called in two random projects I maintain, after fork() and before execve() (or execveat() or fexecve()).

  alarm(0); /* disable alarms */
  setenv(); /* A couple of required envs, like MALLOC_PERTURB_ or MALLOC_PERTURB_ */
  prctl(PR_SET_DUMPABLE, 1) /* regarding ptrace()-attach */
  syscall(__NR_personality, ADDR_NO_RANDOMIZE); /* disable ASLR for debugging, if needed */
  socketpair() /* reliable execve success detection, some form of witchcraft */
  setpriority()
  prctl(PR_SET_PDEATHSIG, SIGKILL); /* die upon parents death */
  setrlimit(); /* set of reset rlimits */
  lseek(fd, 0, SEEK_SET); /* rewind input file for this specific subprocess */
  /* prepare arguments (argv) for execve dynamically */
  sysconf(_SC_NPROCESSORS_ONLN); pthread_setaffinity_np(); /* pin subprocess to a list of CPUs */
  /*
  LOTS of functions here
  if we wanted to use net/process/mount namespacing
  e.g:
  assigning IP adddresses to interfaces
  creating custom views of the filesystem tree
  modifying capability sets
  */
  open("/proc/self/oom_score_adj"), write(), close(); /* adjustment of oom score */
  open("/proc/self/fd", O_DIRECTORY); getdents(); fcntl(F_GETFD); fcntl(F_SETFD, FD_CLOEXEC); close() /* closing fds upon exec */
  setsid(); /* new session */
  sigprocmask(empty_set); /* reset signal mask */
  open("/dev/null"); dup2(null, 0..1); /* close fd 0,1,2 */
  prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0)
  prctl(PR_SET_SECCOMP, SECCOMP_MODE_FILTER); /* application of sandboxing */
  and finally execv() or execveat()
Granted, the projects are maintain are probably more on the heavy side of things, when it comes to process manipulation, before execve, but putting all of that in some control structure, would be down to impossible for me. Such structure would have to be so extensible, that it'd have to be some form of VM I guess effectively. So.. having ability to simply call a couple of syscalls from the context of a regular new process, and before execve() is quite good here.

Sure.. maybe we should have some simple form of fork/execv, for those who want to call system() or popen() and not hit the memory overcommit related crashes.

But not as a replacement, rather a new syscall. Even so, debugging while a process creation/execution failed would be madness, given that you simply would get EINVAL, and the failure could be related to any of dozen parameters in a process creation control structure.

Re:In C, how do you know if the dynamic allocation succeeded? by Koromix on Saturday November 06, 2021 @04:51AM (#29128522)
I think that in a proper API, we could have:

  int process_handle = spawn("/bin/true", SP_PAUSED); // create process but don't execute, and return a handle
  // most API would take a process handle, thus you could do stuff (such as prctl) on the new process
  unpause(process_handle);
There's a bit of a move in this direction in the Linux API with the PIDFD stuff.
Re:In C, how do you know if the dynamic allocation succeeded? by jagrsw on Saturday November 06, 2021 @05:05AM (#29128616)
Sounds reasonable. I guess most process controlling syscalls (ptrace, prctl, seccomp, bpf, remotely operating on pid's FDs, setsid, signal masks) will have to be replaced to be able to operate on remote process or on a fd, but yeah, this can work, potentially.
Re:In C, how do you know if the dynamic allocation succeeded? by foxfluff on Saturday November 06, 2021 @05:16AM (#29128685)
If it's all syscalls (i.e. no libraries messing with hidden state) and you don't mess with parent's memory, you might be able to do this all with clone(CLONE_VM|CLONE_VFORK) instead of fork(). This is what musl uses to implement posix_spawn().
Re:In C, how do you know if the dynamic allocation succeeded? by jagrsw on Saturday November 06, 2021 @05:28AM (#29128757)
I think setenv() will use malloc() (it might use mutexes), same for error printing if it uses printf() or perror().

Also, this is all mess if fork() is called from a multi-threaded context. Esp. via clone(), which up to certain glibc() version cached getpid() result, and returned parent'd PID in the child (for performance reasons of course ;).

Re:In C, how do you know if the dynamic allocation succeeded? by alexgartrell on Friday November 05, 2021 @04:19PM (#29124302)
Allocated-but-unavailable is a totally reasonable part of the memory hierarchy.

Main Memory => zswap (compressed memory) => swap

In this case, the pages may be logically allocated or not -- the assurance is that the data will be the value you expect it to be when it becomes resident.

Should those pages be uninitialized, the "Swapped" state is really just "Remember that this thing was all zeros."

We could do computing your way, but it'd be phenomenally more expensive. I know this because every thing we introduce to the hierarchy in practice makes computing phenomenally less expensive.

Re:In C, how do you know if the dynamic allocation succeeded? by cturner on Friday November 05, 2021 @04:24PM (#29124357)
"We could do computing your way, but it'd be phenomenally more expensive"

It must be viable - Windows prevents overcommit. But it has slow child-process-creation (edit: previously said "forking"), and this steers development towards native threads which is its own set of problems.

I had never previously joined the dots on the point pcwalton makes at the top of this thread. It is a dramatic trade-off.

Re:In C, how do you know if the dynamic allocation succeeded? by Rusky on Saturday November 06, 2021 @12:53AM (#29127629)
I'm not sure there's any causality between Windows preventing overcommit and Windows having slow process creation- there's nothing inherently slower about CreateProcess()/posix_spawn() than fork() + exec() as an API.

It seems more that overcommit is a workaround for the way fork() can potentially lead to copying the entire address space into new pages, but usually doesn't. Because CreateProcess() knows more precisely how much to allocate before it returns to the child process, it can just reserve that amount and signal an error immediately if there's not enough memory+swap to back it.

(And on the other hand, Windows has a lot of legacy and backwards compatibility behavior around processes that could easily explain the slower process creation independent of the API.)

Re:In C, how do you know if the dynamic allocation succeeded? by pjc50 on Friday November 05, 2021 @04:35PM (#29124449)
Windows does not have fork(), apart from the hidden ZwCreateProcess; https://news.ycombinator.com/item?id=9653975

For ages cygwin had to emulate it manually by hand-copying the process.

Re:In C, how do you know if the dynamic allocation succeeded? by throwawaylinux on Friday November 05, 2021 @08:23PM (#29126189)
Linux can prevent overcommit. And that's not the reason for much faster process creation than Windows as far as I know.
Re:In C, how do you know if the dynamic allocation succeeded? by OldHand2018 on Friday November 05, 2021 @04:58PM (#29124676)
Let's say you malloc some memory and the computer actually has everything available.

Everything is great, up until some other process on your system does a fork bomb of an infinitely recursive program that allocates nothing on the heap. You've just got a whole lot of quickly growing stacks hoovering up your physical memory pages.

Re:In C, how do you know if the dynamic allocation succeeded? by tsimionescu on Saturday November 06, 2021 @01:29AM (#29127770)
Stack vs Heap is a userspace level concept. As far as the kernel memory manager knows, it's all just allocated memory - some is allocated by the process- or thread-spawning routines, some is allocated by malloc(), but they're using the exact same pool.

If overcommit is disabled and someone has allocated most system memory, fork() and exec() and pthread_create() etc. will theoretically fail with ENOMEM.

A bigger problem on Linux at least is that the kernel will swap out all possible memory before returning an allocation error. Even if you have not allocated any swap space, it will swap out any memory mapped files.

And even if none of your programs have explicitly mmap()ed anything, the actual application code is mapped, so it will start swapping out all code pages to disk before it refuses an allocation. And now this means that your system has become entirely unusable and will have to be hard rebooted, because it is now swapping data to and from disk on every instruction execution after every context switch. At least your CPU will get to stay nice and cool for a while.

Re:In C, how do you know if the dynamic allocation succeeded? by Dylan16807 on Saturday November 06, 2021 @02:21AM (#29127959)
It's really unfortunate that you can't set minimum and maximum disk cache on Linux.