Don't use Hadoop when your data isn't that big (2013)

Re:Don't use Hadoop when your data isn't that big (2013) by dswalter on Tuesday May 23, 2017 @12:27PM (#14402597)

I've been using PrestoDB for a few months now, and I'm deeply in love. It's such a well-designed piece of technology. The query execution engine is a tremendous boon to anyone with inconvenient-sized data. And it does most (all?) of ANSI-SQL!

[ Parent | on HN ]

Re:Don't use Hadoop when your data isn't that big (2013) by threeseed on Tuesday May 23, 2017 @02:46PM (#14404012)

Spark SQL is actually ANSI SQL.

Plus you can mix and match R, Scala and SQL all together.

[ Parent | on HN ]

Re:Don't use Hadoop when your data isn't that big (2013) by dswalter on Tuesday May 23, 2017 @07:19PM (#14406309)

I used to use Spark SQL for this purpose, but I've switched. I now use Spark for when I want to transform data. But when I'm writing ad-hoc data exploration/investigation queries, PrestoDB is my jam; it's what it's designed for. Parquet as the data storage method makes both of these quite workable.

[ Parent | on HN ]