News for hackers. Stuff that matters.

A comment on Redshift Spectrum – Exabyte-Scale In-Place Queries of S3 Data. Parent comment

Re:Redshift Spectrum – Exabyte-Scale In-Place Queries of S3 ... by ah- on Wednesday April 19, 2017 @07:05PM (#14152571)
I wonder where the bottleneck was. Right below that result they have the performance plot that shows how it used up almost no CPU or network bandwidth. I'd expect S3 to be faster than this?

FWIW, I've been playing with parquet and Kudu lately and on a reasonably sized single machine you can run similar queries on similar scale data in <5 seconds.

[ Parent | on HN ]
Re:Redshift Spectrum – Exabyte-Scale In-Place Queries of S3 ... by awgupta on Thursday April 20, 2017 @11:28AM (#14157528)
The constraint for Redshift Spectrum queries tends to be between S3 and Spectrum. The graphs Jeff shows are in the Redshift cluster itself which, for the sample query he is running, is lightly loaded. We are working on upgrading our performance dashboards...
[ Parent | on HN ]