News for hackers. Stuff that matters.
Re:Fork() without exec() is dangerous in large programs (2016) by garethrowlands on Tuesday June 14, 2022 @12:38PM (#31741995)
"The received wisdom suggests that Unix’s unusual combination of fork() and exec() for process creation was an inspired design. In this paper, we argue that fork was a clever hack for machines and programs of the 1970s that has long outlived its usefulness and is now a liability. We catalog the ways in which fork is a terrible abstraction for the modern programmer to use, describe how it compromises OS implementations, and propose alternatives."

from A Fork in the Road, <https://www.microsoft.com/en-us/research/uploads/prod/2019/0...>

Re:Fork() without exec() is dangerous in large programs (2016) by wongarsu on Tuesday June 14, 2022 @01:29PM (#31742900)
fork() and exec() is a reflection of the unix philosophy: small tools for separate tasks, and as a result you have great composability. You can set up right etc for your new process by calling any number of system calls between fork() and exec().

Windows is a great example of the alternative (following "Windows philosophy"): There are about three different API calls for creating a new process, each with a heap of complicated optional arguments. The API becomes more complex, less composable, less extensible and less powerful. But also easier to reason about and easier for the kernel to provide, and arguably with fewer footguns.

Re:Fork() without exec() is dangerous in large programs (2016) by fpoling on Tuesday June 14, 2022 @02:16PM (#31743592)
Windows approach is not the only alternative. Simply provide API to create a process in a suspended state, then adjust its properties based on pid/handle and then start the process execution.
Re:Fork() without exec() is dangerous in large programs (2016) by ncmncm on Tuesday June 14, 2022 @02:55PM (#31744248)
A huge number of tricky thread problems go away if the child thread is blocked at startup, and allowed to run only after the parent allows it. To retrofit this, it is easiest to lock a mutex before spawning and have the child block on that. Then the parent unlocks it to let the child run.
Re:Fork() without exec() is dangerous in large programs (2016) by lanstin on Tuesday June 14, 2022 @05:21PM (#31746253)
I think shared libraries can spawn threads on their load/init phase that you don't know about. Then you are hosed but you only know about it due to sporadic weird problems, that if you restart on failure, e.g. a pre-forked worker scenario, you might never even really care about.
Re:Fork() without exec() is dangerous in large programs (2016) by Blikkentrekker on Wednesday June 15, 2022 @05:16AM (#31750722)
I always felt that the way to create new processes and new threads should initially start the same. A new process is simply a thread that after creation does a syscall to isolate itself from the other thread and receive a unique copy of all resources.

This could also solve the issue with forking from multithreaded programs since we can ensure we own all shared resources when we isolate our thread, to effectively thus become a new process.

So instead of fork we have clone/isolate.

A new thread can also of course immediately suspend itself, allow other threads to work on it's data in some way, who then give it a signal to resume itself and then execute if need be.

Re:Fork() without exec() is dangerous in large programs (2016) by klodolph on Tuesday June 14, 2022 @01:42PM (#31743091)
fork() is not a small tool.

It is a MASSIVE, unwieldy tool that is difficult to use correctly. It happens to have a small interface.

Re:Fork() without exec() is dangerous in large programs (2016) by adastra22 on Tuesday June 14, 2022 @01:46PM (#31743154)
...is it?

In it is original implementation, fork() was pretty trivial. All it did was create a new process entry in the kernel table, with all the pages and capabilities and such copied from the original process. Then mark all pages as copy-on-write, and return to the caller. Maybe not trivial, but much less complicated than loading an executable file from disk.

My understanding of Linux internals is maybe 20 years out of date, so I am legitimately curious what makes fork() so complicated these days.

Re:Fork() without exec() is dangerous in large programs (2016) by cryptonector on Tuesday June 14, 2022 @01:52PM (#31743244)
fork() is not trivial now. Processes are huge now -- they have huge heaps among other things. Copying all that is expensive. In the 80s we tried COW, but that turns out to be very slow as well. What operating systems do now is immediately copy the resident set, then do COW for the rest of writable memory, but in large, multi-threaded processes, this is still too slow.

Use vfork() or posix_spawn().

Re:Fork() without exec() is dangerous in large programs (2016) by adastra22 on Tuesday June 14, 2022 @02:09PM (#31743481)
Hrm. Googling "fork linux copy-on-write" seems to find a lot of stack overflow answers from 2014-2015 claiming Linux marks pages as copy-on-write when fork() is called. I didn't see anything more recent in the first page of results.

I could see it being worthwhile to immediately copy a few pages, like the top of the stack, but copying the whole resident set seems excessive. Especially since some of that data might not even be written to.

Re:Fork() without exec() is dangerous in large programs (2016) by CHY872 on Tuesday June 14, 2022 @05:46PM (#31746471)
So the problem is what happens to the old and new processes after the fork. To CoW, you need to mark all the pages read only in _both_ old and new which means that every memory write in the caller will now pagefault since the OS now has to lazily copy on both sides. So with true copy on write the fixed costs may be low but the marginal cost per memory write may be high in both parent and child. In this case you can see why the resident set is copied, yes? It’s the smallest amount of memory that guarantees predictable performance subsequent to the call returning.
Re:Fork() without exec() is dangerous in large programs (2016) by cryptonector on Tuesday June 14, 2022 @06:28PM (#31746836)
If the parent is threaded and the host has more than one CPU, then fork() == TLB shootdowns, which are slow.

As well there's the cost of all those page faults that the two processes are likely to take to do the copying.

And lastly there's all sorts of complexity involving multiple parent threads calling fork(), or the child calling fork() again (or vfork()) before calling exec.

It's just much easier to copy the resident set and mark the address space as being CoW, because now you only have to worry about page faults for pages that are not in core anyways and so were going to fault anyways, and that means you don't have to worry about TLB shootdowns either (if a page is not in core, it's not referenced by any TLB either). You still have the multi-fork issues, but now you can use an atomic reference count on the address space.

Re:Fork() without exec() is dangerous in large programs (2016) by btilly on Tuesday June 14, 2022 @09:44PM (#31748198)
The classic pattern was that you'd spin up something like Apache, load it full of read-only data, then start forking its children. Hoping that you'd share memory between children.

With what you describe, you'd share nothing because at the point you've loaded it up, all the data you just loaded is resident. :-(

Re:Fork() without exec() is dangerous in large programs (2016) by cryptonector on Tuesday June 14, 2022 @10:27PM (#31748449)
If loading == mmap(2)ing read-only, then it's not a problem. Read-only pages don't get CoW'ed.
Re:Fork() without exec() is dangerous in large programs (2016) by btilly on Tuesday June 14, 2022 @11:03PM (#31748660)
Back when I did this, loading usually meant querying a bunch of constant data in mod_perl.
Re:Fork() without exec() is dangerous in large programs (2016) by cryptonector on Wednesday June 15, 2022 @01:23AM (#31749409)
So write the query results to files and then mmap() them!
Re:Fork() without exec() is dangerous in large programs (2016) by btilly on Wednesday June 15, 2022 @08:39PM (#31761245)
You'd be amazed at how difficult it is to make useful use of mmap() from within a variety of common scripting languages.
Re:Fork() without exec() is dangerous in large programs (2016) by spc476 on Tuesday June 14, 2022 @03:04PM (#31744398)
Hmmm, from <https://www.man7.org/linux/man-pages/man3/posix_spawn.3.html>:

    The posix_spawn() and posix_spawnp() functions provide the
    functionality of a combined fork(2) and exec(3), with some
    optional housekeeping steps in the child process before the
    exec(3).  These functions are not meant to replace the fork(2)
    and execve(2) system calls.  In fact, they provide only a subset
    of the functionality that can be achieved by using the system
    calls.
Also, there's no way to set resource limits in the child process, nor switch user or group ID, using posix_spawn().
Re:Fork() without exec() is dangerous in large programs (2016) by GoblinSlayer on Wednesday June 15, 2022 @07:09AM (#31751344)
For that you may need posix_spawn and exec, but still can evade fork completely.
Re:Fork() without exec() is dangerous in large programs (2016) by duskwuff on Tuesday June 14, 2022 @03:20PM (#31744627)
Using fork() also means you end up with shared ownership of resources like file descriptors, which can have some pretty weird consequences.
Re:Fork() without exec() is dangerous in large programs (2016) by cryptonector on Tuesday June 14, 2022 @03:27PM (#31744767)
This is true with all process creation APIs.

Windows defaults to CLOEXEC semantics and you have to opt-in to child process inheriting open file handles, and that has caused problems.

Unix defaults to not-CLOEXEC sematincs, and that too has caused problems.

Re:Fork() without exec() is dangerous in large programs (2016) by temac on Tuesday June 14, 2022 @05:41PM (#31746427)
The Windows default can cause problems because of simple logic bugs.

The Unix default can cause unsolvable problems because of races between threads.

You should use CLOEXEC everywhere. Except you can't because you are using libraries.

Re:Fork() without exec() is dangerous in large programs (2016) by cryptonector on Tuesday June 14, 2022 @05:44PM (#31746453)
True. The main problem with the Unix default is that there wasn't a way to set O_CLOEXEC on all new FDs race-free until recently. That's a real problem. FD leaks to children can be bad, but most of the time they are not the end of the world, and often one can steal a closefrom() implementation from a BSD or Illumos as a workaround when you know exactly what you want to allow the child to inherit.
Re:Fork() without exec() is dangerous in large programs (2016) by asveikau on Tuesday June 14, 2022 @05:46PM (#31746475)
closefrom() comes in handy for this. It's missing on some platforms (notably glibc and mac iirc) but actually not too hard to implement a work-alike.
Re:Fork() without exec() is dangerous in large programs (2016) by cryptonector on Tuesday June 14, 2022 @06:30PM (#31746845)
I've copied a closefrom() many a time.

A common hack I've had to add is an argument FD that is not to be closed because. e.g., an flock is held on it.

Re:Fork() without exec() is dangerous in large programs (2016) by bregma on Tuesday June 14, 2022 @04:12PM (#31745406)
Or more importantly, IPC mechanisms like mutexes. If they're in shared memory, you now have two problems. The runtime of a very very popular scripting languages does this.
Re:Fork() without exec() is dangerous in large programs (2016) by turminal on Tuesday June 14, 2022 @03:09PM (#31744469)
> Then mark all pages as copy-on-write, and return to the caller.

Unix actually copied the memory over initially.

Re:Fork() without exec() is dangerous in large programs (2016) by moomin on Tuesday June 14, 2022 @02:50PM (#31744139)
It was simple when processes were simple, but requirements got serious.
Re:Fork() without exec() is dangerous in large programs (2016) by sitkack on Tuesday June 14, 2022 @04:54PM (#31745942)
Fork is not a reflection of unix philosophy! It is a beautiful hack tho.
Re:Fork() without exec() is dangerous in large programs (2016) by phendrenad2 on Tuesday June 14, 2022 @03:17PM (#31744593)
Has Linux gained syscalls equivalent to those Windows API calls yet? Or is linux too different from the windows kernel to make that happen? (In that case, what does WINE do?)
Re:Fork() without exec() is dangerous in large programs (2016) by lanstin on Tuesday June 14, 2022 @05:28PM (#31746315)
On linux, fork libc function(s) is(are) a wrapper to clone system call which is more flexible (they added some stuff to make Wine work better):

https://lwn.net/Articles/826313/

Clone is also used to start threads, IIRC:

https://stackoverflow.com/questions/4856255/the-difference-b...

Re:Fork() without exec() is dangerous in large programs (2016) by indrora on Tuesday June 14, 2022 @06:51PM (#31746993)
To present some contrast:

Windows doesn't have fork(). It has a real, fully mature thread and process model. In Windows NT, every process consists of a handle that is a "Process", which in turn points to a structure containing a list of "Threads". A process is done when its main thread exits or all threads exit, whichever is defined by the main process. Fork/Exec is replaced with CreateProcess (or ShellExecute, your choice).

For a very zen-like example of the fork/exec and pipe management that you'd do on a POSIX system done in Windows, the [MSDN Docs](https://docs.microsoft.com/en-us/windows/win32/procthread/cr...) are quite informative.

Re:Fork() without exec() is dangerous in large programs (2016) by skissane on Wednesday June 15, 2022 @06:43AM (#31751186)
> Windows doesn't have fork()

It does (or maybe did, not sure if it still works) have the literal equivalent of fork() - NtCreateProcess() with NULL SectionHandle argument creates a new process which is a clone of the caller. However, it never really worked. The Win32 API did not support forking, and so any process which forked itself and then tried to invoke any Win32 API calls would soon crash or fail mysteriously. In theory forking did work for pure NT API applications, but those are rather limited in their abilities.

Re:Fork() without exec() is dangerous in large programs (2016) by dblohm7 on Thursday June 16, 2022 @11:11AM (#31766966)
The original POSIX subsystem would have used it.
Re:Fork() without exec() is dangerous in large programs (2016) by btilly on Tuesday June 14, 2022 @09:46PM (#31748216)
When I see what most people try to do with "a real, fully mature thread and process model", I wind up shaking my head about bad engineering processes.
Re:Fork() without exec() is dangerous in large programs (2016) by jiggawatts on Wednesday June 15, 2022 @05:38AM (#31750865)
Bad... how?

From my perspective watching the various techniques used by a multitude of operating systems over many decades, the Windows-style process+thread model seems to be winning out over the UNIX fork() model.

For example, PostgreSQL seems to suffer greatly from the "forked process per connection" model, necessitating front-ends that do connection pooling. Database engine after database engine seems to go through this phase and then "upgrade" to either a single-process thread pool model, or start using async IO in some way. (Web servers also.)

For reference, Microsoft SQL Server back in the 1990s could on Windows NT 4 could handle more connections than PostgreSQL in 2020...

Re:Fork() without exec() is dangerous in large programs (2016) by cryptonector on Tuesday June 14, 2022 @01:55PM (#31743280)