This is a super common occurrence in training loops for ML models because PyTorch uses multiprocessing for its dataloader workers. If you want to read more, see the discussion in this issue: https://github.com/pytorch/pytorch/issues/13246#issuecomment...

As you’ve pointed out fork() isn’t ideal for a number of reasons and in general it’s preferred to use torch tensors directly instead of numpy arrays so that you are not forced into using fork()

There’s also this write up which I found to be quite useful for details: https://ppwwyyxx.com/blog/2022/Demystify-RAM-Usage-in-Multip...

“I believe it's Copy-on-Write semantics, right? What happens when the parent process changes the array?”

Yes and you just answered your question.

For a programming novice: Is CoW where a 'child' process operates on a copy of data stored elsewhere in memory, but you have some mechanism to always update the copy any time the parent process modifies/writes to the original data?

No, if I have stuff, and I fork(), you, my child process is expected to have a copy of that stuff in it's current state. However, the kernel lies to you. It delays copying the stuff until I write to it i.e until I need to see a new state. Hence, "copy-on-write".

In this case, I am the parent python process, you are the child processes started with the `multiprocessing` module, and the data is the numpy array.

The implication is that the changes made by the parent process to the numpy array post fork() won't be visible to the child processes.