News for hackers. Stuff that matters.
Re:Data Warehouse Benchmark: Redshift, Snowflake, Azure, Pre... by georgewfraser on Monday September 10, 2018 @11:37AM (#17952545)
I’m one of the authors, happy to answer questions!
Re:Data Warehouse Benchmark: Redshift, Snowflake, Azure, Pre... by massaman_yams on Monday September 10, 2018 @02:27PM (#17953875)
This is fantastic, thanks so much for putting this together. In the past I've relied a bit too much on benchmarks here [0] for assessment of warehousing solutions, but it's pretty clearly skewed toward certain query patterns due to operating on a single table (no joins) and limited aggregation.

Using TPC with a wider variety of queries and evaluating on dimensions other than performance provides much better perspective.

[0] http://tech.marksblogg.com/benchmarks.html

Re:Data Warehouse Benchmark: Redshift, Snowflake, Azure, Pre... by vgt on Monday September 10, 2018 @12:34PM (#17952979)
Great post!

A couple of thoughts:

1. It'd be interesting to see these benchmarks "under real-world adversity". For example, as data is ingested into the DW, as data shapes change, as concurrency of workloads varies.

2. It'd be great to measure "effort" to reach and maintain this performance.

DW vendors are trained on optimizing for static TPC benchmarks with concurrency of 1, which are useful, but can miss the boat on what users actually experience.

(BQ PM)

Re:Data Warehouse Benchmark: Redshift, Snowflake, Azure, Pre... by georgewfraser on Monday September 10, 2018 @12:42PM (#17953049)
We intend to keep iterating on this! If you have a concrete idea, it would be fantastic if you opened an issue or even a PR in our github repo: https://github.com/fivetran/benchmark
Re:Data Warehouse Benchmark: Redshift, Snowflake, Azure, Pre... by massaman_yams on Monday September 10, 2018 @02:37PM (#17953962)
Thanks for the insight! Maybe it was just a brain fart, but I had to look up BQ (BigQuery) just now - the acronym might not be as well-known outside of Google.
Re:Data Warehouse Benchmark: Redshift, Snowflake, Azure, Pre... by vgt on Monday September 10, 2018 @02:43PM (#17954029)
noted :)
Re:Data Warehouse Benchmark: Redshift, Snowflake, Azure, Pre... by djhworld on Monday September 10, 2018 @12:53PM (#17953145)
I'd like to understand more details about your Presto testing.

The post mentioned ORC files, but doesn't detail how big they are, how many, is the dataset partitioned? Is the data sorted in any particular way?

Re:Data Warehouse Benchmark: Redshift, Snowflake, Azure, Pre... by kwillets on Monday September 10, 2018 @12:15PM (#17952818)
No Vertica?
Re:Data Warehouse Benchmark: Redshift, Snowflake, Azure, Pre... by tomnipotent on Monday September 10, 2018 @12:36PM (#17952987)
Does Vertica have a SaaS offering?
Re:Data Warehouse Benchmark: Redshift, Snowflake, Azure, Pre... by kwillets on Monday September 10, 2018 @02:11PM (#17953727)
Re:Data Warehouse Benchmark: Redshift, Snowflake, Azure, Pre... by secondrow on Monday September 10, 2018 @04:00PM (#17954710)
Presto Starburst distro (which has been upgraded to 203e since the benchmark) - also with hourly pricing https://aws.amazon.com/marketplace/pp/B07DKV5659?qid=1536609...