I want Thread to run in parallel.

I've tried multiprocessing but I found it buggy for my use case.

I tried to create a worker queue and submit items to worker threads with JoinableQueue but eventually the system deadlocks and no progress is made. I'm not sure what the problem is. It never finishes.

It's a sentence correlator using multiprocessing - it generates correlations of words in your sentences.

https://github.com/samsquire/notebook/blob/master/sentence-c... https://github.com/samsquire/notebook/blob/master/workers.py

Maybe somebody can catch what I'm doing wrong.

edit: it may be because i've forgotten to run task_done on joinable queue. edit2: yes it was that! it works!

Out of curiosity, is using multiprocessing.Pool’s map/etc.() methods not an option? It’s a higher-level API which would remove the need to manually manage workers/a queue.

Pretty sure they're referring to the GIL.[1]

1. https://realpython.com/python-gil/

yes, for many of us, we really do want a CPython that runs multiple threads of the same interpreter, with memory sharing. That said after decades of CPython not supporting it, I've rewritten most of my code to use multiprocessing with granular parallelism.