Perhaps the best thing to do is to change zip, or add a new parameter to zip, to allow it to "chunk" its inputs and thereby enable vectorization. Or we could have a special type that encapsulates two or more vectors and checks up front that they're the same length, so that zip can then assume that they're the same length. It also might be possible to add an optimization to LLVM to have it figure out that the vectors are the same length.
zip :: [a] -> [b] -> [(a, b)]
Not: zip :: [a] -> [b] -> [(Maybe a, Maybe b)]
If you want to allow processing a sequence of tails, you could have: zipWithTail :: [a] -> [b] -> ([(a, b)], Either [a] [b])
Which would return a list of pairs and a list of either the tail of the first or the second input, whichever was longer.Code at http://ix.io/cUd
I've looked at the output of both inner loops, and it seems I put too much faith in std::valarray's expression templates. Changing the distance function to
int distance(std::valarray<int> const& x, std::valarray<int> const& y) {
return inner_product(begin(x), end(x), begin(y), 0,
std::plus<int>(), [](int x, int y){ return (x-y)*(x-y); });
}
results in the exact same inner loop as Rust. The remaining speed difference comes from the file loading code, which admittedly is crap (way too many memory allocations). So for all intents and purposes, I admit defeat.Possibly. There's been some talk in that direction, but also some "can't we just fix llvm?" objections. I guess someone needs to write up a pull request (and see what other iterator traits to implement, maybe).
But note that the C++ version has no bounds checks, so it's unsafe. You could always write C in Rust in the unsafe sublanguage, so saying it's "fundamentally inferior" is not true in any case.
There is an issue to remove the notion of a "default" integer type.
- `int` is the fastest available integer type in C(++), whereas in Rust it is defined to be pointer-sized. This allows the C++ version to vectorize better than Rust by using 32-bit integers (also there aren't native SIMD 64-bit multiplications in current x86).
- Even when changing the native type from `int` to `i32`, to match the C++ code, rustc does not vectorize the distance function. I do not know the exact reason for this, but I would guess the iterator code is unable to elide null pointer checks. It's unclear to me whether this is a language limitation or simply a current compiler limitation.