Re:What's up, Python? The GIL removed, a new compiler, optpa...
by apelapan on Monday July 31, 2023 @04:11AM
(#36939942)
Sure, with multiprocessing you can get 96 python processes running at 100% CPU while sharing a large dataset.
Only problem is that 99% of that CPU usage is for serializing/deserializing IPC messages and total throughput would have been higher using a single process.
There are use-cases for multiprocessing. As long as data sharing between processes is insignificant, it can be quite performant. Just like using a bash-wrapper script that orchestrates a bunch of python (or other) processes.
Only problem is that 99% of that CPU usage is for serializing/deserializing IPC messages and total throughput would have been higher using a single process.
There are use-cases for multiprocessing. As long as data sharing between processes is insignificant, it can be quite performant. Just like using a bash-wrapper script that orchestrates a bunch of python (or other) processes.