News for hackers. Stuff that matters.
Re:Sherver: Bash lightweight web server by junon on Wednesday December 22, 2021 @07:55AM (#29648641)
Cat is not a specified built-in but many implementations do build it in to speed up scripts that use it quite heavily.

Other examples include `which`, `time`, `echo`, etc. They're usually promoted to built-ins when it's clear that's the intended command to run, but there are still analogs on the PATH if need-be - I'm not sure exactly how bash would detect a non-standard executable there to know when it's unsafe to promote to a built-in, though.

For example with echo:

    $ bash -c 'echo hi'
    hi
    
    $ strace bash -c 'echo hi' | grep 'exec'
    execve("/usr/bin/bash", ["bash", "-c", "echo hi"], 0x7fffe264aad0 /* 24 vars */) = 0
    
    $ strace bash -c '$(which echo) hi' | grep 'exec'
    execve("/usr/bin/bash", ["bash", "-c", "$(which echo) hi"], 0x7fffe36c6cd0 /* 24 vars */) = 0
    execve("/usr/bin/echo", ["/usr/bin/echo", "hi"], 0x7fffe2e16ff0 /* 25 vars */) = 0
In fact for me, `cat` always gets inlined, even if I call it with its full path or with `which cat`.

There are cases where the built-in and the actual executable differ - the most prominent case being `time`. The built-in does not have the command line arguments to print out things like pre-empted scheduling (forced context swiches).

    $ time --help
    --help: command not found
    
    $ /usr/bin/time --help
       Usage: /usr/bin/time [-apvV] [-f format] [-o file] [--append] [--verbose]
          [--portability] [--format=format] [--output=file] [--version]
          [--quiet] [--help] command [arg...]
Re:Sherver: Bash lightweight web server by steerablesafe on Wednesday December 22, 2021 @08:55AM (#29649301)
"echo" and "time" are explicitly bash built-ins, "which" and "cat" are not, and you will always get a process spawned there.

Your second strace is misleading, you didn't trace child processes.

    $ strace bash -c '$(which echo) hi' 2>&1 | grep 'exec'
    execve("/bin/bash", ["bash", "-c", "$(which echo) hi"], 0x7ffddfcdaee0 /* 59 vars */) = 0
    execve("/bin/echo", ["/bin/echo", "hi"], 0x55eb565cb0b0 /* 59 vars */) = 0
    
    $ strace -f bash -c '$(which echo) hi' 2>&1 | grep 'exec'
    execve("/bin/bash", ["bash", "-c", "$(which echo) hi"], 0x7ffd09b3e228 /* 59 vars */) = 0
    [pid 81493] execve("/usr/bin/which", ["which", "echo"], 0x5560d172e0b0 /* 59 vars */) = 0
    execve("/bin/echo", ["/bin/echo", "hi"], 0x5560d172e0b0 /* 59 vars */) = 0
Re:Sherver: Bash lightweight web server by rascul on Wednesday December 22, 2021 @08:13AM (#29648841)
You had:

    $ strace bash -c 'echo hi' | grep 'exec'
    execve("/usr/bin/bash", ["bash", "-c", "echo hi"], 0x7fffe264aad0 /\* 24 vars */) = 0
Something seems off. strace(1) prints the trace to stderr not stdout, so you should need |& not | to catch it with grep. At least, that's how it works here.

Also, cat does not appear to be inlined for me:

    rascul@smarts:~> strace bash -c 'cat /etc/os-release' |& grep 'exec'
    execve("/usr/bin/bash", ["bash", "-c", "cat /etc/os-release"], 0x7ffe7b5334e0 /* 88 vars */) = 0
    execve("/usr/bin/cat", ["cat", "/etc/os-release"], 0x562c447076f0 /* 88 vars */) = 0*
Re:Sherver: Bash lightweight web server by junon on Wednesday December 22, 2021 @08:15AM (#29648863)
Yeah good catch, I forgot it since I didn't copy 1:1 the command line into vim when I formatted it for HN. I use fish + powerline so the command line doesn't copy correctly. In fish, it's `&|`, which would look weird anyway. I just forgot to add the `&` after the `|`.

The output was copied 1:1 though.

The inlining is an implementation detail and I've seen it happen differently across different platforms.

Re:Sherver: Bash lightweight web server by rascul on Wednesday December 22, 2021 @08:21AM (#29648947)
> The inlining is an implementation detail and I've seen it happen differently across different platforms.

Do you have more information on this? I would be interested to read more about it. As far as I was aware (and my knowledge is certainly not exhaustive), the only thing kinda related is as noted in the bash man page:

> The command substitution $(cat file) can be replaced by the equivalent but faster $(< file).

Re:Sherver: Bash lightweight web server by junon on Wednesday December 22, 2021 @08:26AM (#29648987)
I don't, sorry. I just know it's something some implementations opt-in to (especially drop-in replacements to bash). It's behavior I've observed quite a bit and have leveraged it in some cases to improve the performance of some critical bash scripts over the years. I'd also be interested in some prose about it.
Re:Sherver: Bash lightweight web server by prussian on Wednesday December 22, 2021 @08:13AM (#29648845)
Are you sure?

    strace -ff -e execve /bin/bash -c 'x="$(cat <<< "test")"'
    ... snip
    [pid 311214] execve("/usr/bin/cat", ["cat"], 0x56280da76fc0 /* 3 vars */) = 0
Re:Sherver: Bash lightweight web server by junon on Wednesday December 22, 2021 @08:16AM (#29648889)
Yep, I'm sure. It's not a guarantee (nor is it specified, to my knowledge) that the inlining happens. You also used a subshell there, which might make a difference.
Re:Sherver: Bash lightweight web server by BeefWellington on Wednesday December 22, 2021 @08:32AM (#29649043)
The original poster is using a subshell. Your example uses echo, which is both a built-in AND an external program.

They are not comparable. I've never seen a shell inline cat, probably because it would make no sense given there are POSIX builtins to do exactly what cat does.

Re:Sherver: Bash lightweight web server by junon on Wednesday December 22, 2021 @09:14AM (#29649535)
It of course makes sense since `cat` is widely used for the purpose of the original code, just not in a subshell. It's used often to output blocks of text to stdout/err for multiline output, namely --help text.

strace doesn't lie here, and I've seen it substitute cat in trivial cases quite often.

Re:Sherver: Bash lightweight web server by BeefWellington on Thursday December 23, 2021 @12:42AM (#29659015)
Indeed strace doesn't lie but you have yet to provide an example of a system where what you're saying about cat is true. You've shown a number of examples which do not actually back up the claim you are making, and have not responded to other posters pointing out the flaws in the examples you did provide.

I'm genuinely curious to see an implementation but I suspect you've misunderstood something in the past and your statement was incorrect.

Re:Sherver: Bash lightweight web server by frutiger on Wednesday December 22, 2021 @08:17PM (#29657628)
> They're usually promoted to built-ins

There’s no runtime “promotion”. You can read about it all in bash(1).