What's up, Python? The GIL removed, a new compiler, optparse deprecated

Re:What's up, Python? The GIL removed, a new compiler, optpa... by coldtea on Sunday July 30, 2023 @08:15PM (#36937474)

>Use the python multiprocessing module. If you've already written it with the multithreading module, it is a drop in replacement. Your data structure will live in shared memory

Only if it can be immutable. So it can't be shared and changed by multiple processes as needed (with synchronization).

And you can not even merely update it (e.g. after some time read a newer large file from disk to load into your data structure) without restarting the whole server and processes.

So, it could work for this case, but it's hardly a general solution for the problem.

[ Parent | on HN ]

Re:What's up, Python? The GIL removed, a new compiler, optpa... by felipetrz on Monday July 31, 2023 @02:13AM (#36939340)

For this use case it would be better to put the data in a shared SQLite database than relying on multiprocessing CoW.

Even accessing objects from the shared memory would cause the reference counter to increment and the data would be copied, causing a memory usage explosion.

[ Parent | on HN ]

Re:What's up, Python? The GIL removed, a new compiler, optpa... by coldtea on Monday July 31, 2023 @04:30AM (#36940061)

>For this use case it would be better to put the data in a shared SQLite database than relying on multiprocessing CoW

In Python yes. In Java you could take advantage of shared memory and get spared the overhead of SQLite.

[ Parent | on HN ]