std::async is included in the C++11 standard. The number of threads that async will use is implementation dependent, I bet on Windows with VS2012 you will see a different number of threads. You could force async to run asynchronously or in a lazy evaluation mode using launch policies.

> Unless you use a language with cheap and fast green threads that multiplex on OS threads (like Haskell), you should use a number of threads that is roughly in the same order of magnitude as the number of CPU cores + hyper threads.

The optimal number of threads per core is workload and machine dependent, but rarely is that number 1 per hardware thread[1]. For maximum CPU utilization and computational efficiency, you need multiple software threads per hardware thread so that the OS always has a thread ready to go when another thread blocks or the CPU has an unused component. How much the threads block and what parts of the CPU the exercise is the workload-dependent part. Ideally, the number of threads would be a tunable performance parameter.

As an example of this, I recently wrote some math-heavy code to transform the cells of a matrix by a particular transfer function. Each cell calculation was independent, so I parallelized them. The most efficient parallelization for my 6-core AMD processor turned out to be 6 threads per core.

[1] The main exception I can think of is when running multiple applications in parallel that in turn parallelize their work.

This is dangerous misinformation.

I'm sorry, but if your "math-heavy" code performs better when oversubscribed, then it is a massively suboptimal implementation. As a performance-oriented mathematical library developer, with colleagues at many of the major supercomputing centers and vendors, oversubscription is a hazard that we guard against on CPU architectures because it is always slower. Many supercomputing environments use a kernel that does not even support over-subscription.

Note that GPUs are different in the sense that over-decomposition is necessary to cover latency, but even there, you generally get better performance at lower occupancy due to better register reuse. SMT CPU systems, most notably, Blue Gene/Q, often require use of multiple hardware threads per core to cover instruction latency and keep the prefetcher busy (due to allocation of outstanding requests). This is not over-subscription because those are bonified hardware threads.

Can I pick your brains???

What happens during a cache miss? Does the thread block?

Other instructions in the same thread are executed (when they can be, e.g. all data's available). Or in another hyperthread on the same core.

> The optimal number of threads per core is workload and machine dependent, but rarely is that number 1 per hardware thread.

This is exactly why I said it should be in the same order of magnitude, not exactly the same. Definitely not 250+ threads on a dual core. It might make sense to add a few more threads to keep the CPU busy if one or more of the threads will have to wait.

You're right, I misinterpreted "order of magnitude".

Having recently worked with Nginx, I am now a proponent of writing non-blocking code. Then, it makes sense to use the same number of threads as CPUs.