This can be vectorised for CPU execution (which is nice!), but it will not solve the problem for GPUs. Moving just that scalar-vector multiplication to a GPU would be detrimental to performance, because the run-time would be dominated by the cost of moving the 'b' array to the GPU, followed by moving the 'a' array back. You get into all kinds of issues with where to leave data that has no obvious right answer.

It also does not help with problems that are not as embarrassingly parallel, but could still benefit from vectorisation, such as summations (or reductions in general), or things that contain their own inner control flow (think a Mandelbrot set, where you have a larger outer parallel loop with an inner serial while-loop).

That actually depends upon the architecture. If you have shared memory between GPU and CPU the copy latency is reduced by orders of magnitude - you're only waiting on the values to be populated in GPU's caches.

It'll be interesting to see what sort of context-switch performance comes out of the Zen APU devices when they are released (next year I believe).

I'd be somewhat surprised if Intel and NVidia aren't working on similar things.

> If you have shared memory between GPU and CPU the copy latency is reduced by orders of magnitude - you're only waiting on the values to be populated in GPU's caches.

Shared memory by itself will not have a significant performance impact here (although it simplifies the data movement substantially). For a scalar-vector multiplication, you are strongly memory bound, and so you still have to wait for all the memory to be retrieved over the bus. You are correct if both CPU and GPU are behind the same bus (such as if the GPU is on the same die), but modern GPUs support shared virtual memory even for non-integrated GPUs.

> Another aspect of D that enables parallization is the use of ranges + algorithms.

Is that similar to the C++ std::algorithm execution policy features introduced in C++17?

No. What Walter is talking about in D is if you have

Foo[] foo = ...;

foreach(f; foo) { f.bar.baz.quux; }

and the computation is parallelisable then you can write

import std.parallelism; foreach(f; parallel(foo)) { f.bar.baz.quux; }

and it will parallelised across threads.

I guess std.parallelism's parallel might be more analogous to an "instance" of an executor.

So that's similar to OpenMP parallel for looping?

I'm not familiar with that aspect of C++17.

Its the tag based dispatching for the `par` in

std::sort(par,a.begin(),a.end());

to call a parallel implementation of sort on `a`.