Re:Nuitka: a Python compiler written in Python
by samsquire on Thursday June 17, 2021 @09:53AM
(#27539574)
I want Thread to run in parallel.
I've tried multiprocessing but I found it buggy for my use case.
I tried to create a worker queue and submit items to worker threads with JoinableQueue but eventually the system deadlocks and no progress is made. I'm not sure what the problem is. It never finishes.
It's a sentence correlator using multiprocessing - it generates correlations of words in your sentences.
Re:Nuitka: a Python compiler written in Python
by aw1621107 on Thursday June 17, 2021 @10:58AM
(#27540268)
Out of curiosity, is using multiprocessing.Pool’s map/etc.() methods not an option? It’s a higher-level API which would remove the need to manually manage workers/a queue.
I've tried multiprocessing but I found it buggy for my use case.
I tried to create a worker queue and submit items to worker threads with JoinableQueue but eventually the system deadlocks and no progress is made. I'm not sure what the problem is. It never finishes.
It's a sentence correlator using multiprocessing - it generates correlations of words in your sentences.
https://github.com/samsquire/notebook/blob/master/sentence-c... https://github.com/samsquire/notebook/blob/master/workers.py
Maybe somebody can catch what I'm doing wrong.
edit: it may be because i've forgotten to run task_done on joinable queue. edit2: yes it was that! it works!