News for hackers. Stuff that matters.
Re:Show HN: Quickly copy a file between computers by blibble on Saturday February 16, 2013 @12:42PM (#5232265)
for reading files and transferring over the network: the limit is always going to be the IO devices, not an extra context switch and a few extra copies...
Re:Show HN: Quickly copy a file between computers by 1337p337 on Saturday February 16, 2013 @01:26PM (#5232408)
s/always/usually/. Depending on the disks involved and your network hardware (either a super-fast link, or even a system with a slow CPU and cheap network hardware that expects the drivers to do the heavy lifting in software), it actually could end up CPU-constrained.

But for the usual case where it is I/O-constrained, you can often get files across more quickly by throwing CPU at reducing the total amount of bandwidth required by, e.g., using something like bzip or pbzip:

  pbzip2 < file | nc $host $port
And on the other side:

  nc -l $port | pbzip2 -d > file