http://dl.acm.org/citation.cfm?id=1687730
This one appears to be a more recent survey paper by some of those authors:
http://www.morganclaypool.com/doi/abs/10.2200/S00514ED1V01Y2...
As someone else pointed out, this was also one of Netezza's differentiation.
You get to cheat at that, because the naive implementation loads too much data, while most of it is going to drop off at the filter level.
The ideal TPC-DS schema stores data partitioned by date and maintains a lookup index by item, store and clustered by store's state.
In a database organized like that, neither part of the Query-3
item.i_manufact_id = 436 or dt.d_moy=12
should actually look at every row in the data-set in brute-force fashion - you get to skip massive parts of the data by just inspecting the index or going to the specific date partition and do it much faster than a naive C++ full-scan.
However, if you were limited to a full-load + scan, this would really be killer to have a columnar format which can be handled by an FPGA. The really interesting quote in the blog (for me) is
> the data for the queries is pushed to the accelerator card in columnar format (which is blazingly fast for queries)
I sure could use FPGA filters for columnar data ... even after I've done all my indexing, since I can't index down AND clauses or OR clauses - unlike a traditional CPU, the FPGA will be able to evaluate my entire condition in one sequential flow instead of check + branching.
Hard-wiring operators is a minor improvement but really not much because compared to memory access it's not the bottleneck.
You do have to pass your data through the accelerator to get the processing... which potentially means huge volumes of data moving into this physical processing layer (probably can be done in parallel over a network at high speed) - I would assume this is why shared memory bandwidth was a problem.
This would provide some really interesting options though - imagine feeding data from two disparate databases (say, Oracle and SQL server) in a data flow into this thing - now you have accelerated cross database joins (as long as you can handle the bandwidth and processing on the way in).
There was this post before on HN previously lamenting the state of tools for working with FPGAs, and my related comment wondering if what Baidu has done here was possible:
- https://news.ycombinator.com/item?id=9408881
- https://news.ycombinator.com/item?id=9410160