News for hackers. Stuff that matters.
Re:Cloning a Laptop over NVMe TCP by dezgeg on Tuesday March 12, 2024 @04:20AM (#39677264)
Isn't `nc -l -p 1234 > /dev/nvme0nX` working by accident (relying on that netcat is buffering its output in multiples of disk block size)?
Re:Cloning a Laptop over NVMe TCP by jasomill on Tuesday March 12, 2024 @04:51AM (#39677430)
No — the kernel buffers non-O_DIRECT writes to block devices to ensure correctness.

Larger writes will be more efficient, however, if only due to reduced system call overhead.

While not necessary when writing an image with the correct block size for the target device, even partial block overwrites work fine:

  # yes | head -c 512 > foo
  # losetup /dev/loop0 foo
  # echo 'Ham and jam and Spam a lot.' | dd bs=5 of=/dev/loop0
  5+1 records in
  5+1 records out
  28 bytes copied, 0.000481667 s, 58.1 kB/s
  # hexdump -C /dev/loop0
  00000000  48 61 6d 20 61 6e 64 20  6a 61 6d 20 61 6e 64 20  |Ham and jam and |
  00000010  53 70 61 6d 20 61 20 6c  6f 74 2e 0a 79 0a 79 0a  |Spam a lot..y.y.|
  00000020  79 0a 79 0a 79 0a 79 0a  79 0a 79 0a 79 0a 79 0a  |y.y.y.y.y.y.y.y.|
  *
  00000200
Partial block overwrites may (= will, unless the block to be overwritten is in the kernel's buffer cache) require a read/modify/write operation, but this is transparent to the application.

Finally, note that this applies to most block devices, but tape devices work differently: partial overwrites are not supported, and, in variable block mode, the size of individual write calls determines the resulting tape block sizes.

Re:Cloning a Laptop over NVMe TCP by dezgeg on Tuesday March 12, 2024 @05:28AM (#39677606)
Somehow I had thought even in buffered mode the kernel would only accept block-aligned and sized I/O. TIL.
Re:Cloning a Laptop over NVMe TCP by neuromanser on Tuesday March 12, 2024 @12:22PM (#39681342)
> # yes | head -c 512 > foo

How about `truncate -s 512 foo`?

Re:Cloning a Laptop over NVMe TCP by mrb on Tuesday March 12, 2024 @04:43AM (#39677392)
Your exact command works reliably but is inefficient. And it works by design, not accident. For starters, the default block size in most netcat implementations is tiny like 4 kB or less. So there is a higher CPU and I/O overhead. And if netcat does a partial or small read less than 4 kB, when it writes the partial block to the nvme disk, the kernel would take care of reading a full 4kB block from the nvme disk, updating it with the partial data block, and rewriting the full 4kB block to the disk, which is what makes it work, albeit inefficiently.