News for hackers. Stuff that matters.
Re:GNU Parallel – The command line power tool by shrike on Wednesday August 14, 2013 @12:04AM (#6209882)
I use GNU Parallel with s3cmd to move big data sets in and out of S3. I can easily saturate any network connection. I was able to GET ~2TB from S3 onto a Gluster cluster in a little more than an hour by using GNU Parallel to spread the GETs across 8 instances. Incredibly powerful, easy to use tool.

There are no comments on this comment yet.