Re:Memory efficiency of parallel IO operations in Python
by marcosdumay on Tuesday March 13, 2018 @08:11AM
(#16575467)
A thread requires at minimum a new stack segment. That is 1kb or something like that, but if you plan on accepting a lot of connections and keep them on hold, it adds up fast and quickly becomes the bottleneck of your server.
A new process requires a new stack and a new heap segment, that later one is usually on the MB range.
Besides, starting and stopping threads take some time. If you are serving small files, forking at their start means that your process will spend most of its time forking. And if you don't use an easy architecture that forks on every connection, it isn't much more difficult to go all the way and make your server fully asynchronous.
A new process requires a new stack and a new heap segment, that later one is usually on the MB range.
Besides, starting and stopping threads take some time. If you are serving small files, forking at their start means that your process will spend most of its time forking. And if you don't use an easy architecture that forks on every connection, it isn't much more difficult to go all the way and make your server fully asynchronous.