Re:Asynchronous Python and Databases
by thrownaway2424 on Sunday February 15, 2015 @06:16PM
(#9054596)
I'd want data. The system I work on does in fact spawn a thread to handle each and every connection and in fact each connection thread spawns numerous child threads to exploit available parallelism within the request. The code is fully blocking and linear and anyone can read it and see what it is doing. The mentioned system is one of the largest public networks services on earth.
I am very skeptical of the idea that you must not handle thousands of connections with a thread per connection. High tens of thousands of threads per core is the minimum level where I would start to worry.
Show me your application and I'll tell you where it's either calling select() or is limiting connections to keep itself from locking up the OS.
Re:Asynchronous Python and Databases
by thrownaway2424 on Sunday February 15, 2015 @07:40PM
(#9054788)
You're right on the second point. I have it limited to 4 million sockets per machine.
For anything where you're dealing with thousands of connections from random Internet hosts, "just spawn a thread for it" does not cut it. If you take that approach, you're setting yourself up to be accidentally DoS'd at some point in the near future. Async, on the other hand, has more than proven itself to be apt for this kind of scenario.