News for hackers. Stuff that matters.
Re:Dd is not a disk writing tool (2015) by kosma on Sunday January 08, 2017 @07:11AM (#13349388)
There's a common idiom of skipping a file header and handing off processing to some other program, like this:

    cat foo | ( dd bs=$HEADERSIZE skip=1 of=/dev/null; process-foo-contents )
Re:Dd is not a disk writing tool (2015) by greenleafjacob on Sunday January 08, 2017 @08:32AM (#13349619)
Isn't this the same as:

    tail -c +$HEADERSIZE <foo
Re:Dd is not a disk writing tool (2015) by tspiteri on Sunday January 08, 2017 @10:06AM (#13349927)
The dd idiom can be used to split a file into parts with known block size, something like

    (dd bs=$SIZE1 count=1 of=file1; dd bs=$SIZE2 ...
Re:Dd is not a disk writing tool (2015) by fiddlerwoaroof on Sunday January 08, 2017 @10:07AM (#13349930)
Cool, although this doesn't really help with my most common-case, skipping the first line.

    (head -1 > /dev/null; cat -) < file
Re:Dd is not a disk writing tool (2015) by koala_man on Sunday January 08, 2017 @03:54PM (#13351958)
This is undefined behavior: while `head -1` will only output a single line, it may read more.

It happens to work on GNU head when stdin is seekable file, because GNU head specifically rewinds the stream before exiting:

    $ (strace  -e read,write,lseek head -1 > /dev/null; cat -) < file
    ...
    read(0, "hello\nworld\n", 8192)         = 12
    lseek(0, -6, SEEK_CUR)                  = 6    # <-- here
    write(1, "hello\n", 6)                  = 6
    +++ exited with 0 +++
If not for that explicit `lseek`, `head -1` would have skipped the entire 8k buffer.

As far as I know, this is exclusive to GNU cat. Neither Busybox nor OSX cat will do this, and will therefore throw away an entire buffer instead of just the first line. You can try it out:

(busybox head -1 > /dev/null; cat -) < file

Re:Dd is not a disk writing tool (2015) by fiddlerwoaroof on Sunday January 08, 2017 @05:25PM (#13352429)
Interesting. Is this true of `tail -n +2` as well? (On mobile, can't test at the moment).
Re:Dd is not a disk writing tool (2015) by ucs on Sunday January 08, 2017 @09:34PM (#13353549)
Tail employs a large read buffer as well, but it does not matter because you wouldn't use it in the same manner.

Tail is the right tool for the job here. But if you wish to stick with your idiom, read will reliably consume a single line of input, regardless of how it is implemented:

  (read -r; cat) < file
Re:Dd is not a disk writing tool (2015) by koala_man on Sunday January 08, 2017 @10:03PM (#13353685)
If anyone is wondering why `read` can reliably read a single line while head can't, it's because it reads byte by byte.

This is just as inefficient as it sounds, but it doesn't matter much in practice since you rarely read a lot with it.

Re:Dd is not a disk writing tool (2015) by koala_man on Sunday January 08, 2017 @09:22PM (#13353500)
That always reads to eof, so it can't be used in the same way.
Re:Dd is not a disk writing tool (2015) by RJIb8RBYxzAMX9u on Sunday January 08, 2017 @10:49AM (#13350181)
Unintuitively, tail is the utility you want:

  $ tail -n +2 file
From tail(1):

  -n, --lines=K
    output the last K lines, instead of the last 10; or use -n +K to output lines starting with the Kth
Re:Dd is not a disk writing tool (2015) by fiddlerwoaroof on Sunday January 08, 2017 @11:03AM (#13350277)
Yeah, I know about that, I just prefer not to use that option because `head -1 > /dev/null` is clearer
Re:Dd is not a disk writing tool (2015) by kosma on Sunday January 08, 2017 @10:23AM (#13350022)
head is line-wise. dd is byte-wise.
Re:Dd is not a disk writing tool (2015) by fiddlerwoaroof on Sunday January 08, 2017 @11:05AM (#13350300)
Yeah, I was just saying that generally, when I need to strip off a header, the header is the first line of the file