Re:Free-threaded CPython is ready to experiment with
by viraptor on Saturday July 13, 2024 @12:50AM
(#40951848)
> I use numpy, pandas, xarray, pytorch, etc.
That means your code is using python as glue and you do most of your work completely outside of cPython. That's why you don't see the impact - those libraries drop GIL when you use them, so there's much less overhead.
Re:Free-threaded CPython is ready to experiment with
by quietbritishjim on Saturday July 13, 2024 @03:53PM
(#40956421)
The parent commenter said they're using the multiprocessing module, so it's irrelevant to them whether those modules drop the GIL (except for the fact that they are missing an opportunity to using threading instead). The overhead being referred to, whether significant or not, is that of spawning processes and doing IPC.
I use coroutines and multiprocessing all the time, and saturate every core and all the IO, as needed. I use numpy, pandas, xarray, pytorch, etc.
How did this terrible GIL overhead completely went unnoticed?