Re:We have to talk about this Python, Gunicorn, Gevent thing
by ryanobjc on Saturday March 07, 2020 @08:59PM
(#22515784)
Is there a n:m green threading mode for python? What does gunicorn/gevent do? Sounds like “not that”.
Yes best practices and all that, but that really doesn’t sound like what’s happening.
Re:We have to talk about this Python, Gunicorn, Gevent thing
by pas on Sunday March 08, 2020 @07:45AM
(#22517563)
Python was always more about breadth than depth. (CPython is full of known inefficiencies, but it's with us since 1989, and basically the core dev who worked most on performance - Victor Stinner - thinks the best way is to introduce subinterpreters - https://github.com/vstinner/talks/blob/master/2019-EuroPytho... )
Oh, that PDF is interesting, Python 3.8 has shared memory for multiprocessing, no more pipe objects between processes.
Furthermore extension and internal stuff always had the ability to release the GIL and do its own thing (for example, on a threadpool, or using async/nonblocking I/O). But I have no idea about Gevent. I never liked it. (Just as Twisted/Tornado it was too much magic for too little benefit.)
Re:We have to talk about this Python, Gunicorn, Gevent thing
by Matthias247 on Saturday March 07, 2020 @10:32PM
(#22516169)
I don’t know what gevent is doing. But if you have a global interpreter lock then M:N might not be worth it, since only thread will make progress (outside of the syscalls, which are non blocking).
Also, isn't the problem that JSON decoding (or whatever computation) simply block the thread and the other green threads cannot proceed at all, because there are simply no safepoints (yield points) inside these low level functions?
And in all these cases shouldn't the application estimate work (eg. in case of JSON if the string is longer than 100K), and if it's too big just put it on a dedicated heavy compute N:N thread pool?
For Python it's best practice anyway because the GIL, no?