Re:My experience with using cp to copy 432 million files (39...
by sitkack on Thursday September 11, 2014 @06:41PM
(#8305609)
If you are often restarting rsync jobs or they fail over low speed WAN connections
--partial keep partially transferred files
Is your friend. Esp if you are trying to sync directories with multi GB files. Also if your media is bad or there is a failing piece of network gear along the path --bwlimit=KBPS limit I/O bandwidth; KBytes per second
Can keep things running smoothly. It even works against disks, I have used it where an array would die if one attempted to read from it too quickly.
Rsync at least acts reasonably idempotent, so you can just run it again if it gets interrupted, which is usually why I use it for large copies.
I don't remember off the top of my head if tar handles hard links—it may be inappropriate for this usage.