Re:My experience with using cp to copy 432 million files (39...
by brazzy on Thursday September 11, 2014 @07:18PM
(#8305760)
No giant bookkeeping datastructures that end up with the process thrashing in swap, because tar is designed to work with so many files that you don't want to keep the metadata in memory.
[ Parent
| on HN ]
Re:My experience with using cp to copy 432 million files (39...
by tedunangst on Friday September 12, 2014 @11:44AM
(#8308651)
How does tar keep track of every hardlink without a giant hash table? It doesn't. It's not magic.
Re:My experience with using cp to copy 432 million files (39...
by makmanalp on Thursday September 11, 2014 @08:07PM
(#8305946)
This is true if the bookkeeping was irrelevant and unnecessary. I imagine the scenario where your copy gets stopped in the middle is worse when you use the tar method.
Re:My experience with using cp to copy 432 million files (39...
by brazzy on Friday September 12, 2014 @05:06AM
(#8307188)
It's not as if cp gives you anything more useful in that case. The bookkeeping that caused the problem was a hashtable of files that have been copied, necessary to handle hard links. Would be interesting how tar does that; if it can't do any better then the root comment and mine above are wrong.