Partial reads won't corrupt the data. Dd will issue other read() until 1MB of data is buffered. The iflag=fullblock is only useful when counting or skipping bytes or doing direct I/O. See line 1647: https://github.com/coreutils/coreutils/blob/master/src/dd.c#...

According to the documentation of dd, "iflag=fullblock" is required only when dd is used with the "count=" option.

Otherwise, i.e. when dd has to read the entire input file because there is no "count=" option, "iflag=fullblock" does not have any documented effect.

From "info dd":

"If short reads occur, as could be the case when reading from a pipe for example, ‘iflag=fullblock’ ensures that ‘count=’ counts complete input blocks rather than input read operations."

Thank you for the correction -- it is likely that I did use count= when I ran into this some 10 years ago (and been paranoid about ever since). I thought a chunk of data was missing in the middle of the output file, causing everything after that to be shifted over, but I'm probably misremembering.

thank you for bringing it up! i wasn't even aware of this potential problem, and I use bs= count= and skip= seek= (sk"i"p means "input") through pipes across the net aaaallll the time for decades.

it pretty much seems iflag=fullblock is a requirement if you want the counts to work, even though the failure times might be rare

Isn't `nc -l -p 1234 > /dev/nvme0nX` working by accident (relying on that netcat is buffering its output in multiples of disk block size)?

No — the kernel buffers non-O_DIRECT writes to block devices to ensure correctness.

Larger writes will be more efficient, however, if only due to reduced system call overhead.

While not necessary when writing an image with the correct block size for the target device, even partial block overwrites work fine:

  # yes | head -c 512 > foo
  # losetup /dev/loop0 foo
  # echo 'Ham and jam and Spam a lot.' | dd bs=5 of=/dev/loop0
  5+1 records in
  5+1 records out
  28 bytes copied, 0.000481667 s, 58.1 kB/s
  # hexdump -C /dev/loop0
  00000000  48 61 6d 20 61 6e 64 20  6a 61 6d 20 61 6e 64 20  |Ham and jam and |
  00000010  53 70 61 6d 20 61 20 6c  6f 74 2e 0a 79 0a 79 0a  |Spam a lot..y.y.|
  00000020  79 0a 79 0a 79 0a 79 0a  79 0a 79 0a 79 0a 79 0a  |y.y.y.y.y.y.y.y.|
  *
  00000200

Partial block overwrites may (= will, unless the block to be overwritten is in the kernel's buffer cache) require a read/modify/write operation, but this is transparent to the application.

Finally, note that this applies to most block devices, but tape devices work differently: partial overwrites are not supported, and, in variable block mode, the size of individual write calls determines the resulting tape block sizes.

Somehow I had thought even in buffered mode the kernel would only accept block-aligned and sized I/O. TIL.

> # yes | head -c 512 > foo

How about `truncate -s 512 foo`?

Your exact command works reliably but is inefficient. And it works by design, not accident. For starters, the default block size in most netcat implementations is tiny like 4 kB or less. So there is a higher CPU and I/O overhead. And if netcat does a partial or small read less than 4 kB, when it writes the partial block to the nvme disk, the kernel would take care of reading a full 4kB block from the nvme disk, updating it with the partial data block, and rewriting the full 4kB block to the disk, which is what makes it work, albeit inefficiently.

I would include bs=1M and oflag=direct for some extra speed.