Re:TiSpark sits Spark SQL on top of a storage engine to answ...
by jamesblonde on Friday August 04, 2017 @05:46PM
(#14932595)
Do you mean Hive on Hops, I guess? HopsFS is a filesystem (drop-in replacement for HDFS with distributed metadata).
Partition-pruned index scans are key for high performance in distributed databases. Index scans and full-table scans hit all shards and kill performance. Hive supports partition-pruned index scans and push-down filtering for projections.
I'm curious if HopsFS has any support for partition-pruned secondary index scans? (If not, is it on the roadmap?) I feel like the lack of it in many current big-data systems is a major bottleneck to adoption for many kinds of workloads.