News for hackers. Stuff that matters.

A comment on Building CockroachDB on top of RocksDB. Parent comment

Re:Building CockroachDB on top of RocksDB by ryanworl on Friday January 18, 2019 @09:37AM (#18939541)
No, I am not aware of any storage engine that provides that out of the box. The techniques are very tied into what your query processing engine can do and expects the data to look like.

For example, do you materialize tuples immediately, or do you fully run it through your processing pipeline and not materialize until the end?

Your storage engine and format needs to be at least somewhat involved in answer that question, because you need to know what data to read and when.

[ Parent | on HN ]
Re:Building CockroachDB on top of RocksDB by arjunnarayan on Friday January 18, 2019 @09:42AM (#18939577)
Unfortunately most of the systems that build what you're describing are closed source (e.g. Snowflake, Microsoft SQL Server, Vertica, Teradata). There isn't an open-source project that does all of those things.
[ Parent | on HN ]
Re:Building CockroachDB on top of RocksDB by georgewfraser on Friday January 18, 2019 @11:21AM (#18940342)
What about Presto?
[ Parent | on HN ]
Re:Building CockroachDB on top of RocksDB by FridgeSeal on Friday January 18, 2019 @06:06PM (#18943985)
Presto is more of a distributed SQL solution: you run it on a cluster of nodes, point them at your storage later, it’s more optimised at querying very large datasets and it’s not built or tuned for high performance (in terms of latency or execution time).
[ Parent | on HN ]