News for hackers. Stuff that matters.
Re:Command-line tools can be faster than a Hadoop cluster (2... by aub3bhat on Sunday September 11, 2016 @02:46PM (#12474923)
"The paper says these exact things. You have to keep reading, and it's hard I know, but for example the last paragraph of section 5 says pretty much exactly this."

Thanks, that addresses my concern. I take back my comment.

But why stop at Rust implementation, there are vendors optimizing it down to FPGA. This sort of comparison is hardly meaningful.

Re:Command-line tools can be faster than a Hadoop cluster (2... by frankmcsherry on Sunday September 11, 2016 @03:17PM (#12475086)
The only point of the paper is that the previous publications sold their systems primarily on performance, but their performance arguments had gaping holes.

The C# and Rust implementations have the property that they are easy and you don't need to have any specific skills to write a for-loop the way we did (the only "tricks" we used were large pages and unbuffered io in C#, and mmap in Rust).

The point is absolutely not that these are the final (or any) word in these sorts of computations; if you really care about performance, use FPGAs, ASICs, whatever. There will always be someone else doing it better than you, but we thought it would be nice if that person wasn't a CS 101 undergraduate typing in what would literally be the very first thing they thought of.