Re:My experience with using cp to copy 432 million files (39...
by mnw21cam on Friday September 12, 2014 @11:46AM
(#8308667)
A while back I had a hard drive die with an unrecoverable error on the superblock (ext3) and a few other places. I sent the hard drive to a professional recovery firm who were as incompetent as incompetent can be. I got the duff drive back and decided to have a go myself. Recovering the superblock was easy - there are extra copies of that. I then did a full map of the drive listing bad blocks, and did a reverse map to work out which files contained those blocks. Luckily, the only file that contained any bad blocks was a swap file, so I recovered absolutely everything. Other bad blocks were in unallocated space. Ext3 is actually a sane enough disc layout that it is possible to work out what is going on manually. I would have been stuffed if it had been a more complicated filesystem.
Re:My experience with using cp to copy 432 million files (39...
by fsniper on Friday September 12, 2014 @01:49AM
(#8306742)
Also there is no mention of unrecoverable file analysis like error handling of cp operations in the article. And with this many files it would not be feasible without using an error log file.
So going with a simple block copy should suffice IMHO.
Re:My experience with using cp to copy 432 million files (39...
by rbh42 on Friday September 12, 2014 @11:36AM
(#8308585)
I'm the OP, so I can shred a bit of light on that: Dell's support suggested a file-level copy when I asked them what they recommended (but I'm not entirely sure they understood the implications). Also, time was not a big issue.
I did keep a log file with the output from cp, and it clearly identified the filenames for the inodes with bad blocks. Actually, I'm not sure how dd would handle bad blocks.
Re:My experience with using cp to copy 432 million files (39...
by fsniper on Friday September 12, 2014 @06:06PM
(#8310727)
Thank you for clarification.
I was about to bet on "read fail repeat skip" cycle for dd's behaviour but, looking into coreutil's source code at https://github.com/goj/coreutils/blob/master/src/dd.c , if I'm not mistaken , dd does not try to be intelligent and just uses a zeroed out buffer so It would return 0's for unreadable blocks.
I see that the choice of a file-level copy was deliberate: "I'd have copied/moved the files at block-level (eg. using dd or pvmove), but suspecting bad blocks, I went for a file-level copy because then I'd know which files contained the bad blocks."