News for hackers. Stuff that matters.
Re:My experience with using cp to copy 432 million files (39... by LeoPanthera on Thursday September 11, 2014 @07:08PM (#8305720)
The prompt goes to stderr, the pipe only pipes stdout, so a prompt should not cause excessive bonage, as long as you're there to respond to it.

Also, don't use -z locally, or even over a moderately fast network. The compression is not that fast and almost always makes things slower.

Re:My experience with using cp to copy 432 million files (39... by thaumaturgy on Thursday September 11, 2014 @08:40PM (#8306052)
Good to know!

Also, re: bonage, I agree that it "shouldn't", but it definitely did. From my sysadmin notes file:

> The tar operation kicks off before ssh engages; having ssh ask for a password seems to intermittently cause problems with the tar headers on the receiving end. (It _shouldn't_, but it seems to.)

Re:My experience with using cp to copy 432 million files (39... by pepve on Friday September 12, 2014 @08:28AM (#8307625)
On modereately fast networks, instead of -z which uses a single threaded gzip implementation, parallel gzip may still give big improvements: http://zlib.net/pigz/