Adding threads only helps if you have more cores, or if you're forced to use something that may block.
But even then, it's better to have a set number of threads doing different tasks, rather than one per user. eg have a network thread, a db thread(s) etc.
First: this is a really interesting thread and I have a lot of respect for your experience.
But is it really better to statically allocate resources to threads? You may have 8 cores on a box and 1 of them burning and 7 of them cruising. By utilizing a small thread pool and letting the scheduler spin things off dynamically you can turn that into 8 cruising instead.
Just curious.
The networking thread shouldn't be using hardly any CPU. If it is, then somethings badly wrong. It's better to have a single networking thread, trawling through the connections, moving data around, and have it talking to other threads that handle long jobs, cpu intensive tasks, or anything that might block.
I'm not trying to be argumentative, I have never done a very large scale website, and only now did I check your profile to see that you did.
But I still don't understand how come it's better to do a single thread. Also, all my websites have been CPU bound.
And even if it's IO bound, if you have more than one disk, adding a thread can only help, no?