Also, isn't the problem that JSON decoding (or whatever computation) simply block the thread and the other green threads cannot proceed at all, because there are simply no safepoints (yield points) inside these low level functions?
And in all these cases shouldn't the application estimate work (eg. in case of JSON if the string is longer than 100K), and if it's too big just put it on a dedicated heavy compute N:N thread pool?
For Python it's best practice anyway because the GIL, no?
Yes best practices and all that, but that really doesn’t sound like what’s happening.
Oh, that PDF is interesting, Python 3.8 has shared memory for multiprocessing, no more pipe objects between processes.
Furthermore extension and internal stuff always had the ability to release the GIL and do its own thing (for example, on a threadpool, or using async/nonblocking I/O). But I have no idea about Gevent. I never liked it. (Just as Twisted/Tornado it was too much magic for too little benefit.)
2. Green threads have lower overhead, but it's a false economy if it causes you to do needlessly redo work because of timeouts that could have been avoided.
Which it seems it must because the kernel doesn't have the insight to know whether a green thread is doing that epoll because it's ACTUALLY idle or because it's not but it's willing to try to juggle a second (or third) thing while it has something on the back burner. So the kernel indiscriminately assigns work to threads without regard for whether they are juggling a lot or nothing.
Whereas with native threads, they never ask the kernel for more work while they're blocked on something else because they are literally blocked and thus won't be making that epoll system call.
(The article also mentions something about LIFO policy, which exacerbates the problem because it favors assigning work to the process which is likely to already have most of it.)