News for hackers. Stuff that matters.
Re:TiSpark sits Spark SQL on top of a storage engine to answ... by elvinyung on Friday August 04, 2017 @05:02PM (#14932234)
Very nice! I actually read the FAST '17 paper a while back.

I'm curious if HopsFS has any support for partition-pruned secondary index scans? (If not, is it on the roadmap?) I feel like the lack of it in many current big-data systems is a major bottleneck to adoption for many kinds of workloads.

Re:TiSpark sits Spark SQL on top of a storage engine to answ... by jamesblonde on Friday August 04, 2017 @05:46PM (#14932595)
Do you mean Hive on Hops, I guess? HopsFS is a filesystem (drop-in replacement for HDFS with distributed metadata).

Partition-pruned index scans are key for high performance in distributed databases. Index scans and full-table scans hit all shards and kill performance. Hive supports partition-pruned index scans and push-down filtering for projections.