Re:BlazingDB uses GPUs to manipulate huge databases in no time
by hendzen on Monday September 12, 2016 @11:33PM
(#12485463)
That sounds like an interesting technique - but it unfortunately does not apply on a majority of real-world data.
In particular I mean sorting non-English text which any serious database needs to be able to do. Subtracting characters doesn't work when you're dealing language specific Unicode collation [0].
Re:BlazingDB uses GPUs to manipulate huge databases in no time
by zlynx on Wednesday September 14, 2016 @03:32PM
(#12500457)
Yeah, well, you just can't do it that way and be fast at the same time. I mean seriously fast.
What you can do is decide how your comparison should be collated and preprocess your strings into a sort key form that is strictly big-endian binary (first byte has highest weight). Or little-endian I suppose, whichever works best for your hardware.
Your sort key doesn't have to be the actual string.
There have been some very sweet algorithms for doing B+ tree searches for Itanium and for AVX and those work just as well, if not better on GPU.
With some thinking, branch operations can be converted into math and applied to masses of data without checking for branch conditions. This wastes some work but is still faster than branching.