News for hackers. Stuff that matters.
Re:In C, how do you know if the dynamic allocation succeeded? by giomasce on Friday November 05, 2021 @05:33PM (#29124951)
I suppose vfork might help with that (though I don't understand why they don't directly add fork_and_execve, it would seem much easier).

Also, the problem with Linux is not having overcommit, but notv being able to choose when to overcommit and when not. Windows makes that easier, AFAIU.

Re:In C, how do you know if the dynamic allocation succeeded? by gpderetta on Friday November 05, 2021 @05:42PM (#29125025)
They do. Man posix_spawn. It comes with its own dsl to be able to support a small subset of all operations that are often performed between fork and execv.

Vfork is usually a better solution.

Edit: and yes, I would love to be able to disable overcommit per process.

Re:In C, how do you know if the dynamic allocation succeeded? by scottlamb on Friday November 05, 2021 @08:00PM (#29126023)
> Vfork is usually a better solution.

I think the biggest downside to vfork is (from the Linux manpage) "the behavior is undefined if the process created by vfork() [...] calls any other function before successfully calling _exit(2) or one of the exec(3) family of functions."

So strictly speaking I think doing any of those operations documented by posix_spawn yourself between vfork and execv is undefined behavior. In practice, I believe it's fine, and on glibc posix_spawn is apparently written in terms of vfork, but libc is allowed to make assumptions that portable programs shouldn't.

Story time: I once worked on a Unix-based system with no MMU. The fork implementation did something insane: it looked for things that were possibly pointers (four-byte aligned memory locations which can be interpreted as valid physical memory locations on this system allocated to that program) and adjusted them. It was kind of like a conservative GC in that it treated anything that looked like a pointer as if it were a pointer. But no reasonable person would call modifying memory that may or may not be pointers to be "conservative". Amazingly, it worked most of the time, but sometimes strings got corrupted, and as a workaround there were a bunch of places where string buffers were fully zeroed where just a NUL byte would otherwise do.

This was early in my career, and I didn't design the system anyway. If I were working on it today I'd remove the fork implementation and make everything use vfork and/or posix_spawn instead. Apparently that's what posix_spawn was made for.

Another weird problem with the lack of MMU: the compiler also didn't use register-based addressing (what's the term? like position-independent code but for the data segment?), so global variables were truly global, not just global to the process. A bunch of code needed to be "deglobalized" to deal with this. I wanted to improve the compiler, but long story short I got offered another job first.

Re:In C, how do you know if the dynamic allocation succeeded? by pcwalton on Friday November 05, 2021 @08:14PM (#29126118)
No-MMU Linux usually just disables fork and uses vfork instead. The really annoying part is that you can't use shared libraries without doubling the size of function pointers (FD-PIC ABI).
Re:In C, how do you know if the dynamic allocation succeeded? by scottlamb on Friday November 05, 2021 @09:10PM (#29126529)
> The really annoying part is that you can't use shared libraries without doubling the size of function pointers (FD-PIC ABI).

Interesting, thanks. That didn't come up on our system—not only did we not use shared libraries but we also linked the whole system into one binary, kernel and all. Link times were atrocious in combination with identical code folding and big VLIW sentences, but it worked.

Re:In C, how do you know if the dynamic allocation succeeded? by OldHand2018 on Friday November 05, 2021 @05:53PM (#29125108)
This paper [1] argues that fork needs to be deprecated in favor of posix_spawn() or other more modern solutions. It claims that, among many other reasons, that fork encourages overcommit and that programs such as Redis are extraordinarily constrained if overcommit is disabled - because of fork.

[1] https://dl.acm.org/doi/10.1145/3317550.3321435

Re:In C, how do you know if the dynamic allocation succeeded? by jart on Saturday November 06, 2021 @01:45AM (#29127835)
> They do. Man posix_spawn. It comes with its own dsl to be able to support a small subset of all operations that are often performed between fork and execv.

In other words, you write your C code inside a C string instead of in C.

Re:In C, how do you know if the dynamic allocation succeeded? by joosters on Friday November 05, 2021 @05:49PM (#29125077)
It’s a weakness of the fork()+exec() model, for sure. However, creating a fork_and_execve() API is extremely tricky. Just think of all the innumerable setup options you would need to give it, e.g. what file handles should be closed or left open? What directory should it start in? What environment variables should be set - or cleared? And on and on and on…

the flexibility of a separate fork() then exec() means you can set up the initial state of a new process exactly as you want, by doing whatever work is needed between the two calls. If you merge them into one, then you will never be able to encapsulate all of that.

Re:In C, how do you know if the dynamic allocation succeeded? by wmf on Friday November 05, 2021 @08:20PM (#29126165)
A third option would be the ability to create a blank process, configure it, then exec within it. I think Mach allows this.
Re:In C, how do you know if the dynamic allocation succeeded? by saagarjha on Friday November 05, 2021 @07:14PM (#29125744)
I mean, posix_spawn exists. It's a messy function, but its job is messy for exactly the reasons you describe. (FWIW, there are very few things you can legally perform between fork and exec.)
Re:In C, how do you know if the dynamic allocation succeeded? by marcan_42 on Friday November 05, 2021 @09:47PM (#29126793)
Are there any things that are illegal between fork and exec? It is perfectly legal to exec whenever you want, and it it perfectly legal to fork whenever you want. I'm not aware of any requirements around the fork/exec sequence.
Re:In C, how do you know if the dynamic allocation succeeded? by tsimionescu on Saturday November 06, 2021 @01:12AM (#29127702)
For a single-threaded process, sure. But in a multi threaded context, almost any operation done in the child after fork() can royally mess up the system. There are many versions of libcs where malloc() after fork() is likely to deadlock between the parent and child processes, as they share the internal malloc() locks.
Re:In C, how do you know if the dynamic allocation succeeded? by marcan_42 on Saturday November 06, 2021 @10:31PM (#29136001)
Ah, yes, multithreading. That always interacts poorly with old UNIX APIs...
Re:In C, how do you know if the dynamic allocation succeeded? by spacechild1 on Saturday November 06, 2021 @05:44AM (#29128828)
From the fork() man page:

“After a fork() in a multithreaded program, the child can safely call only async-signal-safe functions (see signal-safety(7)) until such time as it calls execve(2)“

For example, you can't do any dynamic memory allocation or call a function that may allocate.

Re:In C, how do you know if the dynamic allocation succeeded? by jart on Saturday November 06, 2021 @01:58AM (#29127874)
There's no restrictions in that sense. However the computer might not do what you intended if you were to do something like fputc('x',stdout);fork();fflush(0);
Re:In C, how do you know if the dynamic allocation succeeded? by nextaccountic on Friday November 05, 2021 @11:25PM (#29127238)
I think you mean between vfork and exec. The Linux manpage says:

Standard description

(From POSIX.1) The vfork() function has the same effect as fork(2), except that the behavior is undefined if the process created by vfork() either modifies any data other than a variable of type pid_t used to store the return value from vfork(), or returns from the function in which vfork() was called, or calls any other function before successfully calling _exit(2) or one of the exec(3) family of functions.

Re:In C, how do you know if the dynamic allocation succeeded? by jart on Saturday November 06, 2021 @02:00AM (#29127883)
Note that modern POSIX recommends just calling fork() because "[vfork] was previously under-specified". The glibc man pages shouldn't say undefined since that's too strong a word. There is so much you can actually do in practice during vfork(). Pretty much all system calls are just as safe. It's mostly a question of whether or not your userspace libraries might clobber some global, due to how vfork puts the process in a state where implementation details that are normally private become part of the public surface area of these apis, and not many library authors are disposed to offer those kinds of assurances.