One of the patterns I've used before is to use thread pools rather than starting and stopping threads all the time. Each thread in the pool can be passed a list of the tasks it needs to do, or grab tasks from a central list as it finishes a previous one.

Then after a few runs you can tune the pool size so that you get the most out of the machine. Obviously there are still gains to be made by switching to async io, avoiding any serial choke points etc, and keeping the pool size small, but it seems to be a decent approach.