News for hackers. Stuff that matters.

A comment on Evernote’s transition to Google Cloud Platform. Parent comment

Re:Evernote’s transition to Google Cloud Platform by aluminussoma on Thursday February 09, 2017 @03:17AM (#13605437)
I did try EMR + s3distcp a while ago. AWS Support recommended it and provided instructions in their docs. It was not very fast for me. The data I was trying to copy was about 400 GB of small 5-20 kb files - not ideal for fast copying. I gave up and used a different method.

I suspect Evernote, if using S3, would have a similar data profile of many small objects.

[ Parent | on HN ]
Re:Evernote’s transition to Google Cloud Platform by yeukhon on Thursday February 09, 2017 @11:03AM (#13607830)
Do you think you can share with the public this different method? If not, I completely understand.
[ Parent | on HN ]
Re:Evernote’s transition to Google Cloud Platform by aluminussoma on Thursday February 09, 2017 @02:27PM (#13610029)
No secret - I used s3s3mirror. Multithreaded copying.
[ Parent | on HN ]