Also I've seen a project which persists large amounts of data to disk by forking and using a consistent snapshot of the data structures (which the new process will not mutate, so there's no need for locking).
I'm pretty skeptical of that approach. If there's heavy churn, the copy-on-write behavior means that the mark phase _increases_ memory use, when presumably you're doing GC because you need to _decrease_ memory use. And I assume there's some cost to these VM manipulations in general. Is this implementation competitive with other D garbage collectors in terms of memory usage, CPU time, latency, etc? are D's garbage collectors in general competitive with those of Go and Java?
It isn't an uncommon strategy. For example, the Factorio dedicated server can (optionally) do that if you're running it on non-Windows. The performance is great, but it's marked as experimental, probably because of issues like this article describes.
https://redis.io/topics/faq#background-saving-fails-with-a-f...
So we had a process that would start up, initialize the framework, then look for jobs to run. For each job, it would fork, giving the job an initialized framework but also giving jobs some isolation.
Android does something similar:
"Each app process is forked from an existing process called Zygote. The Zygote process starts when the system boots and loads common framework code and resources (such as activity themes). To start a new app process, the system forks the Zygote process then loads and runs the app's code in the new process. This approach allows most of the RAM pages allocated for framework code and resources to be shared across all app processes."
( https://developer.android.com/topic/performance/memory-overv... )
I believe the Apache web server's Prefork MPM module does something similar—fork() allows the server to spin up new child processes in a pre-initialized state, ready to respond to requests. Servers in general tend to be a good fit for this style of execution.
Emacs works this way, because it has a lot of Lisp code to load in and compile and doesn't have a central daemon (and even if it did, it would want to avoid recompiling the code when the computer is rebooted.) The compiled C binary "temacs" is instructed to load and evaluate a bunch of Lisp and then more-or-less coredump itself in an orderly way to a binary called "emacs"; when that binary is run, everything is in memory as if you had forked temacs after initialization. https://www.gnu.org/software/emacs/manual/html_node/elisp/Bu...
For your average server process reading config files, it probably suffices to have a "compiled" form of the config files in Cap'n Proto or some similar format that is directly usable as in-memory structures, and mmap that in. (Or perhaps just actual native data structures, if the format doesn't need to have a stable representation across versions of your software.)
We have a single process on each server that starts up and loads every single game resource. After this is waits for requests from the instance manager to create new instances. As new instances are required, this "spawner" process fork()s to create one.
Each process uses less memory because all the loaded resources the game needs to run are shared between all the copies.
You also get slightly faster performance because commonly used data still in the CPU cache when switching between processes.
If we used a thread per instance, then we wouldn't have crash isolation between the different game instances, which is handy.
It's also possible to explicitly mmap a shared memory segment to load resources in that without using fork()s copy on write, but it's just more tricky, and the fork() method is totally robust no matter what code other programmers might write.
Apart from that, the only thing I have used fork for (apart from fork+exec) is doing concurrent blocking I/O while prototyping, before rewriting the code to use non-blocking I/O with no forking.
I don't classify these use cases are concurrency-after-development cases. Lots of systems use this to do interesting things.
Ages ago I made a DNS server, and near the top of the program was a simple:
for(i=0;i<ncpus;++i)if(!fork())break;
> Most use cases seem covered by either threads or fork+exec process spawning.Most of the use cases of threads are covered by fork+explicit mmap/vm_remap.
A "shared-nothing" will also contain less code, and having no need for lock/unlock keeps a lot of spam invalidations away from the memory controller making it for many tasks faster.
> Even if you're safe from other threads, there is a lot of other unknown state like fd's still inherited.
The problems you get with threads are bigger.
> unlike threading it also gives you isolation.
If you want isolation why not launch processes? With forks, that copy-on-write thing can increase total RAM usage by a huge factor, if the parent process writes to RAM a lot.
> There are plenty of IPC mechanisms available to communicate things back and forth when you need to.
For communicating gigabytes of data back and forth you want shared memory anyway.
And even for small messages, with threads IPC is faster (1) they are in the same address space, TLB stays current, context switches are faster (2) In some cases, shared memory enables more efficient user-mode IPC methods like busy waiting and lock-free stuff.
They aren't unrelated, they are parent and child. They know PIDs of each other. Parent can read/write stuff to child's streams, input output error. Child inherits opened files. Parent can set command-line arguments & environment variables for the child.
Not really. You are better off communicating what to do with the data, rather than sharing the data.
Not if you actually want to distribute the work over several cores.
Example: Your data is partitioned into 4 segments A, B, C, and D. In step 1, node X processes A&B and node Y processes C&D; in step 2, node X processes A&D and node Y processes C&B. If A, B, C, and D are large, this is a prime example where shared memory trumps message passing in terms of performance. If you think that's a contrived case: dense matrix multiplication behaves like this.
Another way to look at it is that the hardware has a very efficient word-level message-passing interface built-in, via shared memory. Why not take advantage of it?
Because shared mutable state is the root of all evil, or at least very difficult to reason about. Better to have clear owners for all state; if you need to conduct multiple operations on the same data you should move the code to where the data is rather than sharing the data between different processing units (this has long been a truism of distributed system design; modern multicore processors behave much the same as distributed systems if you zoom in enough, and the same techniques are appropriate).
The example given by the author:
seems risky to me: Even if you're safe from other threads, there is a lot of other unknown state like fd's still inherited. And concurrency can be introduced after development, e.g. by using message passing. So going all the way threaded or re-execing yourself seem both safer to me with no real downsides.