Ah, absolutely, I forgot that the state of disk IO on Linux is terrible - although this still isn't quite the case, since there's a network socket involved in copying from disk to socket, so if the socket's buffer becomes full the scheduler will run.

It seems that nginx can use thread pools to offload disk IO, although doesn't unless configured to - by default disk IO will block the worker process. And FreeBSD seems to have a slightly better AIO system it can use, too.