News for hackers. Stuff that matters.
Re:Google’s Dremel Makes Big Data Look Small by majorturd on Friday August 17, 2012 @08:08AM (#4395948)
From TFA "We discuss the core ideas in the context of a read-only system, for simplicity. Many Dremel queries are one-pass aggregations; there-fore, we focus on explaining those and use them for experiments in the next section. We defer the discussion of joins, indexing, up-dates, etc. to future work." Really, it takes Dremel multiple SECONDS to complete trivial massively parallelized read queries? It must take hours for an UPDATE or JOIN then. Wake me up when you move past the trivial, until then, enjoy your hair.
Re:Google’s Dremel Makes Big Data Look Small by jrockway on Friday August 17, 2012 @09:01AM (#4396116)
Dremel is a query tool, not a database.
Re:Google’s Dremel Makes Big Data Look Small by virmundi on Friday August 17, 2012 @09:36AM (#4396245)
That's true, BUT to query you need to have the ability to perform joins. That is what makes raw MapReduce such a pain and even higher level abstractions slow. I like the idea that Dremel is showing, I even downloaded the Google paper to read tonight, but the Apache implementation needs to have joins otherwise it's not a "query tool".
Re:Google’s Dremel Makes Big Data Look Small by ag3mo on Friday August 17, 2012 @10:43AM (#4396568)
You can join on top of BigQuery with small join tables. https://developers.google.com/bigquery/docs/query-reference#...
Re:Google’s Dremel Makes Big Data Look Small by majorturd on Friday August 17, 2012 @11:17AM (#4396772)
cough Here small means less than 8MB of compressed data cough
Re:Google’s Dremel Makes Big Data Look Small by akldfgj on Friday August 17, 2012 @11:58AM (#4397063)
What is the performance of your JOIN system?