Its not as clear cut or obvious as it first seems:

http://mailinator.blogspot.com/2008/02/kill-myth-please-nio-...

Edit: The original presentation is more interesting: http://www.mailinator.com/tymaPaulMultithreaded.pdf

It's not just speed (The speed obviously changes as you change number of connections - 100k threads isn't going to beat 100k NIO connections). It's also that it's more scalable, better code, more maintainable, less prone to bugs, and you don't have to worry as much about concurrency issues. I'd argue that the lines of code is likely to be less also. And memory usage is likely to be less.

Or do you know of specific NIO performance improvements between JDK6 in 2008 and JDK6u26, or in JDK7?

100k threads isn't going to beat 100k NIO connections

That may be true but I wouldn't assume it with certainty without evidence. Threading has been improving, too. Tyma's 2008 presentation mentioned a JVM limit at the time of 16000 threads, but that may have changed as well.

A single-threaded NIO implementation, using only one core, could very plausibly lose to a 100K-threaded implementation on a multicore machine. And once you split your NIO over a few threads to use cores effectively, you have most of the same concurrency worries as with connection-per-thread approaches.

Depends on the application. If the connection is handling real-time financial data(for example), there might be enough heavy lifting on the data stream to warrant one thread per connection. A good programmer wouldn't have too many of these threads running on a given machine, however, because one thread can easily consume a full CPU core, and still need more.

For connections that aren't doing much work, it's definitely better to aggregate connections onto a much smaller number of threads to avoid context switches.

>In my experience once you get up to about 1k threads in Java you'll be wasting most of the CPU up switching between them.

I agree. Thread context switching is extremely expensive. I've seen 10 thread applications consume full CPU time with context switches alone. There are a lot of tricks that can be done to mitigate it, like binding threads to certain CPU cores, making sure that threads that are communicating with each other are on the same CPU to avoid L2/L3 cache thrashing, etc. It's a lot of work, and requires a lot of knowledge about the code and system, but the results are staggeringly good. We've seen performance boosts on the order of 25% with single threaded applications, and boosts around 30-40% for multithreaded apps. Almost all of this is done outside the application code, however.

I can't imagine any reason why you'd have 1k threads running on the same box in one process, to be honest. 1k+ threads seems to indicate a lot of threads doing very little work. If they're doing basically nothing, consuming less than 1% of the CPU each, it's better to try to consolidate the work they're doing into a much smaller number of threads. It's a lot of work, but will get pretty considerable performance gains.

I would agree with the suggestion that if you are going to use a dispatcher pattern with NIO, use a library like netty which does this for you. (In any case you don't write it youself)