News for hackers. Stuff that matters.
Re:Uber’s Big Data Platform: 100+ Petabytes with Minute Latency by Iwan-Zotow on Wednesday October 17, 2018 @07:14PM (#18244644)
What exactly is "Big" here? It is about 1000 hard drives, several racks...
Re:Uber’s Big Data Platform: 100+ Petabytes with Minute Latency by nemothekid on Wednesday October 17, 2018 @08:09PM (#18244935)
Usually the “big” qualifier is a fuction of RAM, not hard disk space. Getting hundreds of petabytes of data onto persistent storage in a “large” room has been possible for many years now.

“Big” data was never about how to store the data.

Re:Uber’s Big Data Platform: 100+ Petabytes with Minute Latency by Iwan-Zotow on Thursday October 18, 2018 @10:57AM (#18248817)
What I'm trying to say, as soon as you could fit data and processing unit(s) into one well cooled room in datacenter, managed by two guys per shift, it is not "Big" problem anymore. Making it all local will probably speed-up their queries/analytics enormously as well
Re:Uber’s Big Data Platform: 100+ Petabytes with Minute Latency by teraflop on Wednesday October 17, 2018 @08:30PM (#18245013)
In my book, "bigger than 99.9% of organizations will ever encounter" is big enough to qualify as "big".
Re:Uber’s Big Data Platform: 100+ Petabytes with Minute Latency by randyrand on Wednesday October 17, 2018 @07:35PM (#18244750)
Ya, big data only starts at 100,000,000 PB. Everyone knows that.

There is no size requirement. It's more about what you collect, the frequency, the coverage, and how you use it.

Re:Uber’s Big Data Platform: 100+ Petabytes with Minute Latency by Iwan-Zotow on Wednesday October 17, 2018 @07:48PM (#18244814)
Of course there is

And anything you could fit into, basically, one well cooled room is not "Big" anymore, sorry to tell you that

Re:Uber’s Big Data Platform: 100+ Petabytes with Minute Latency by PhasmaFelis on Thursday October 18, 2018 @07:23PM (#18253192)
"Big data" is anything that's too big to cram into a standard database and still access in reasonable times. There's a practical definition, it doesn't just mean "YUUUGE".
Re:Uber’s Big Data Platform: 100+ Petabytes with Minute Latency by TomVDB on Wednesday October 17, 2018 @07:31PM (#18244734)
I think you're off by a factor of 100?
Re:Uber’s Big Data Platform: 100+ Petabytes with Minute Latency by Iwan-Zotow on Wednesday October 17, 2018 @07:47PM (#18244806)
Re:Uber’s Big Data Platform: 100+ Petabytes with Minute Latency by saagarjha on Wednesday October 17, 2018 @08:16PM (#18244967)
That website doesn't list a price, but I doubt that running a rack of these would be cost-effective. I wouldn't be surprised if those drives have a cost per terabyte at least a couple of times that of commodity drives.
Re:Uber’s Big Data Platform: 100+ Petabytes with Minute Latency by samontar on Thursday October 18, 2018 @03:34AM (#18246604)
Are you using this in prod? I’d be very surprised. I wouldn’t use this for prod data.
Re:Uber’s Big Data Platform: 100+ Petabytes with Minute Latency by Iwan-Zotow on Thursday October 18, 2018 @10:45AM (#18248712)
Not yet, but people are looking into them

My point is, you could fit all those Uber data (I know, I know, replication, sync etc) into racks in the SINGLE well cooled room in datacenter, managed by 2 guys per shift.

And this is not "Big" as far as I can see.

Probably will speed up their queris/analytics as well all things being local