(This is a pretty common pattern in a lot of big servers.)

Now you want to make sure you utilize multiple cores effectively. The naive approaches are:

1. Create a thread per job, each waiting on its own epoll specific to the job. This may be expensive if there are many jobs, and could allow too much concurrency.

2. Have a single epoll and a pool of threads waiting on it. Each thread must lock a mutex for the job that owns the task it's going to run. But a thread could receive an event for a task belonging to a job that's already running on another thread, in which case it has to synchronize with that other thread somehow, which is a pain. Be careful not to create a situation where all threads are blocked on the mutex for one job while other jobs are starved.

Epoll nesting presents a clean solution:

3. Create an epoll per job, plus an outer epoll that waits on other epolls. A pool of threads waits on the outer epoll, which signals when a per-job epoll becomes ready. The thread receiving that event then takes ownership of the per-job epoll until the event queue is empty.

What do you mean by that? Take that per-job epoll out of the outer epoll?

But I imagine you'd add the per-job epoll to the global epoll with EPOLLONESHOT, so that once an event is reported, it is unregistered with the global epoll. Whatever thread received that event then owns the job. When that thread decides there's nothing more to do, it adds the job epoll back to the global epoll with EPOLLONESHOT again.