Re:Show HN: Quickly copy a file between computers
by niggler on Saturday February 16, 2013 @11:33AM
(#5231965)
Better to do
(performance benefits more apparent with larger files)
[ Parent
| on HN ]
Re:Show HN: Quickly copy a file between computers
by HorizonXP on Saturday February 16, 2013 @11:56AM
(#5232069)
Care to explain why?
Re:Show HN: Quickly copy a file between computers
by niggler on Saturday February 16, 2013 @12:08PM
(#5232120)
(you can find more info in the manpage for your shell)
$ cat foo | bar
ends up creating two processes and a pipe. The foo file is first read by cat and then written onto the pipe (which bar then reads). $ <foo bar
is an input redirection: foo is opened for reading and bar's standard input fd is set to that open file (so the file's data is only read once)
Re:Show HN: Quickly copy a file between computers
by blibble on Saturday February 16, 2013 @12:42PM
(#5232265)
for reading files and transferring over the network: the limit is always going to be the IO devices, not an extra context switch and a few extra copies...
Re:Show HN: Quickly copy a file between computers
by 1337p337 on Saturday February 16, 2013 @01:26PM
(#5232408)
s/always/usually/. Depending on the disks involved and your network hardware (either a super-fast link, or even a system with a slow CPU and cheap network hardware that expects the drivers to do the heavy lifting in software), it actually could end up CPU-constrained.
But for the usual case where it is I/O-constrained, you can often get files across more quickly by throwing CPU at reducing the total amount of bandwidth required by, e.g., using something like bzip or pbzip:
pbzip2 < file | nc $host $port
And on the other side: nc -l $port | pbzip2 -d > file