A Billion NYC Taxi and Uber Rides in AWS Redshift

Re:A Billion NYC Taxi and Uber Rides in AWS Redshift by manigandham on Sunday February 14, 2016 @06:01AM (#11097825)

> I'm not sure I would want to try analyzing a 267 GB database on most modern machine.

Why? Modern servers come with more RAM than that. Even multi-terabyte datasets are not a problem. Today's hardware is way better than most think.

[ Parent | on HN ]

Re:A Billion NYC Taxi and Uber Rides in AWS Redshift by lwhalen on Sunday February 14, 2016 @08:17AM (#11098145)

Could not agree more. Production data needs to be bigger than a petabyte or three before my default response is anything other than 'let's load it into RAM.' :-)

[ Parent | on HN ]

Re:A Billion NYC Taxi and Uber Rides in AWS Redshift by sokoloff on Sunday February 14, 2016 @08:36AM (#11098208)

You mean terabyte, right?

[ Parent | on HN ]

Re:A Billion NYC Taxi and Uber Rides in AWS Redshift by lwhalen on Sunday February 14, 2016 @03:05PM (#11099739)

Well shucks, this is why I try not to HN at 3am on the regular... Yes, terabyte, not petabyte. And I'm well past the edit interval at this point; how embarrassing, mea culpa :-)

[ Parent | on HN ]

Re:A Billion NYC Taxi and Uber Rides in AWS Redshift by JimmyAustin on Sunday February 14, 2016 @02:51PM (#11099692)

I meant in the context of people playing at home who might be working off a laptop or a desktop with only 16-32 GB of ram, not someone with half a terabyte of RAM on a multi-socket server.

[ Parent | on HN ]

Re:A Billion NYC Taxi and Uber Rides in AWS Redshift by manigandham on Sunday February 14, 2016 @04:23PM (#11100075)

That's still fine, it will take longer but will still run without problems. A good SSD is recommended though.

[ Parent | on HN ]