News for hackers. Stuff that matters.
Re:Presto: Interacting with petabytes of data at Facebook by electrum on Wednesday November 06, 2013 @07:27PM (#6686640)
Presto does read Hive data directly out of HDFS, bypassing Hive/MapReduce. However, neither HDFS nor the file formats like RCFile are efficient. The entire design of HDFS makes it difficult to build a real column store. Having a native store for Presto allows us to have tight control over the exact data format and placement, enabling 10-100x performance increases over the 10x that Presto already has over Hive.

Additionally, in the long term, we want to enable Presto to be a completely standalone system that is not dependent on HDFS or the Hive metastore, while enabling next-generation features such as full transaction support, writable snapshots, tiered storage, etc.

Re:Presto: Interacting with petabytes of data at Facebook by flyovercountry on Wednesday November 06, 2013 @10:01PM (#6687187)
HDFS now supports placement groups with one use case being columnar storage.