News for hackers. Stuff that matters.
Re:Feeding data to 1000 CPUs – comparison of S3, Google, Azu... by zbjornson on Tuesday January 05, 2016 @08:32PM (#10847754)
I'm still trying to parse the docs and Manta source code to see what it actually does, but it seems unique if the data storage nodes are also the data processing nodes and no data transfer happens from some storage service before the job begins. The other key factor is having neither startup time nor the cost of a perpetually running cluster. Per my comment below [1], we have used Lambda with S3 to get something like this, as well as our own architecture built on plain EC2/GCE nodes.

[1] https://news.ycombinator.com/item?id=10846514

Re:Feeding data to 1000 CPUs – comparison of S3, Google, Azu... by qaq on Wednesday January 06, 2016 @12:24AM (#10848789)
Not only that but the thing is built by guys who really know what they are doing like Bryan Cantrill and other former SUN top people.
Re:Feeding data to 1000 CPUs – comparison of S3, Google, Azu... by vgt on Tuesday January 05, 2016 @08:41PM (#10847809)
got it. thanks!