News for hackers. Stuff that matters.
Re:Spark Breaks Previous Large-Scale Sort Record by discardorama on Friday October 10, 2014 @02:43PM (#8439382)
Not just 800GB of SSD; 8x 800GB of SSD!
Re:Spark Breaks Previous Large-Scale Sort Record by trhway on Friday October 10, 2014 @03:16PM (#8439528)
yes, that's the key. The local IO on previous reported systems was HDDs, thus 300-600Mb/s at best with 6 drives per machine, while this guys are getting 3.3Gb/s. Getting full performance - 1.1Gb/s - of 10Gb network - were they lucky or AWS now is that good ? (couple years ago i was able to get only 400Mb/s node-to-node on cluster compute nodes there)