You're right. Effectively the kernel is having to setup the event listeners for each FD on every select() call, whereas with epoll it can be done once and the state is preserved, leading to less overall work. Also, the epoll syscalls are surprisingly fast and lightweight.