Re:Memory efficiency of parallel IO operations in Python
by ben0x539 on Tuesday March 13, 2018 @04:37PM
(#16579554)
I really enjoy phrasing the problems in terms of where the state lives. The implicit state machine of where in the code you are and all the crud in your stack is so ingrained in how I reason about code, it's like it isn't even there at all! But if I have to think hard about state and state transitions on every yield point or whatever, it's a bit of a headache.
For each of those network connections, your program needs to retain some state for what it is doing.
There are various places to put the state. You can put the state in a callstack and use blocking IO, but then you have to have one process per connection, which has quite a high minimum memory overhead.
So people have developed a lot of frameworks which keep (connection,program) state in various ways and use the operating system's asynchronous frameworks, so you can have two worker processes handling thousands of connections.