Re:Data-Processing Frameworks Benchmark: Redshift, Hive, Sha...
by keithgabryelski on Monday September 30, 2013 @12:32AM
(#6467805)
I also don't see a schema that is tuned for redshift.
There is no description of a sortkey or a distribution key (let alone which compression encoding was used)
And at least for the first query you could use UNLOAD instead of select (depending on how you are managing the data coming out of redshift it might be a reasonable solution and doesn't force all your data through the leader node constrained by whatever client driver you are using to read the results).
Instead of trying to select out of redshift this much data -- select the data into another table or (again) use unload.
There is no description of a sortkey or a distribution key (let alone which compression encoding was used)
And at least for the first query you could use UNLOAD instead of select (depending on how you are managing the data coming out of redshift it might be a reasonable solution and doesn't force all your data through the leader node constrained by whatever client driver you are using to read the results).
Instead of trying to select out of redshift this much data -- select the data into another table or (again) use unload.