Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on Mpire: A Python package for easier and faster multiprocessing. Parent comment
In standard multiprocessing, all arguments are pickled and pushed to a queue for processes in the pool to use.
To pass heavy arguments, the trick to using CoW was to place them as global variables before the map.
My understanding from Mpire is that they do the same thing, but expose a `shared_objects` parameter to make it less hacky than global variables.
I guess their benchmarks compare against pickling arguments, not against using global variables/CoW, which is why they boast performance increase.
There are no comments on this comment yet.
In standard multiprocessing, all arguments are pickled and pushed to a queue for processes in the pool to use.
To pass heavy arguments, the trick to using CoW was to place them as global variables before the map.
My understanding from Mpire is that they do the same thing, but expose a `shared_objects` parameter to make it less hacky than global variables.
I guess their benchmarks compare against pickling arguments, not against using global variables/CoW, which is why they boast performance increase.