My experience with using cp to copy 432 million files (39 TB)

Re:My experience with using cp to copy 432 million files (39... by fsniper on Friday September 12, 2014 @01:49AM (#8306742)

Also there is no mention of unrecoverable file analysis like error handling of cp operations in the article. And with this many files it would not be feasible without using an error log file.

So going with a simple block copy should suffice IMHO.

[ Parent | on HN ]

Re:My experience with using cp to copy 432 million files (39... by rbh42 on Friday September 12, 2014 @11:36AM (#8308585)

I'm the OP, so I can shred a bit of light on that: Dell's support suggested a file-level copy when I asked them what they recommended (but I'm not entirely sure they understood the implications). Also, time was not a big issue.

I did keep a log file with the output from cp, and it clearly identified the filenames for the inodes with bad blocks. Actually, I'm not sure how dd would handle bad blocks.

[ Parent | on HN ]

Re:My experience with using cp to copy 432 million files (39... by fsniper on Friday September 12, 2014 @06:06PM (#8310727)

Thank you for clarification.

I was about to bet on "read fail repeat skip" cycle for dd's behaviour but, looking into coreutil's source code at https://github.com/goj/coreutils/blob/master/src/dd.c , if I'm not mistaken , dd does not try to be intelligent and just uses a zeroed out buffer so It would return 0's for unreadable blocks.

[ Parent | on HN ]