Re:My experience with using cp to copy 432 million files (39...
by thaumaturgy on Thursday September 11, 2014 @08:40PM
(#8306052)
Good to know!
Also, re: bonage, I agree that it "shouldn't", but it definitely did. From my sysadmin notes file:
> The tar operation kicks off before ssh engages; having ssh ask for a password seems to intermittently cause problems with the tar headers on the receiving end. (It _shouldn't_, but it seems to.)
Re:My experience with using cp to copy 432 million files (39...
by pepve on Friday September 12, 2014 @08:28AM
(#8307625)
On modereately fast networks, instead of -z which uses a single threaded gzip implementation, parallel gzip may still give big improvements: http://zlib.net/pigz/
Also, don't use -z locally, or even over a moderately fast network. The compression is not that fast and almost always makes things slower.