News for hackers. Stuff that matters.
Re:Sherver: Bash lightweight web server by _wldu on Wednesday December 22, 2021 @08:14AM (#29648854)
Here's my small webserver:

    #!/bin/bash

    while : ; do cat conference.txt | nc -l 80; done
Here's the story behind it:

https://www.go350.com/posts/finding-a-hacked-server/

Re:Sherver: Bash lightweight web server by Tepix on Wednesday December 22, 2021 @08:38AM (#29649107)
I liked the story and the impromptu web server!

Two comments:

- using "ping -a" creates an audible ping so you can find the server if you're by yourself without having to look at the screen.

- No need for cat:

    while :; do nc -l 80 < conference.txt; done
Re:Sherver: Bash lightweight web server by _wldu on Wednesday December 22, 2021 @08:56AM (#29649304)
Interesting. I don't hear anything when I use the -a flag.
Re:Sherver: Bash lightweight web server by thanatos519 on Wednesday December 22, 2021 @09:15AM (#29649545)
Let's check with "reveal codes":

    $ ping -a 127.0.0.1 | less

    PING 127.0.0.1 (127.0.0.1) 56(84) bytes of data.

    64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.029 ms^G

    64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.023 ms^G
Looks like you need a terminal that has a BEL sound.
Re:Sherver: Bash lightweight web server by vesinisa on Wednesday December 22, 2021 @09:15AM (#29649559)
It just echoes a Bell character to your terminal. Make sure your terminal emulator supports it, and is configured properly: https://en.wikipedia.org/wiki/Bell_character
Re:Sherver: Bash lightweight web server by rzzzt on Wednesday December 22, 2021 @03:38PM (#29654829)
Windows Terminal shows a "visual bell" in the tab bar whenever BEL is emitted (even though I have it set to "audible" in advanced settings of the profile, hmm).

Edit: Uhm, right. I have also silenced all system sounds in Windows.

Re:Sherver: Bash lightweight web server by dormento on Wednesday December 22, 2021 @09:36AM (#29649872)
Whoa, nifty.

Why does firefox shows the "chose an application to open this link" when you access the http://localhost link, but shows the content correctly when you access the 127.0.0.1/0.0.0.0 ip-based version?

Re:Sherver: Bash lightweight web server by hericium on Wednesday December 22, 2021 @12:07PM (#29652078)
Browser expects to see some HTTP headers before the content. Adding HTTP version with response code, content type declaration and empty line dividing headers and content should do the trick:

    HTTP/1.1 200
    Content-Type: text/plan;

    1st line of content
Re:Sherver: Bash lightweight web server by vincnetas on Wednesday December 22, 2021 @08:45AM (#29649196)
TIL. thanx for -a.
Re:Sherver: Bash lightweight web server by tessierashpool on Wednesday December 22, 2021 @11:17AM (#29651290)
for a more complete solution, check out Bash On Balls, a Rails clone written in bash:

https://github.com/jneen/balls

Re:Sherver: Bash lightweight web server by laputan_machine on Wednesday December 22, 2021 @10:44AM (#29650784)
I could read stories like this forever, they are educational and engrossing, thanks for sharing
Re:Sherver: Bash lightweight web server by BossingAround on Wednesday December 22, 2021 @08:14AM (#29648859)
> I didn't want to install and configure Apache or nGinx. In fact, I didn't want any configuration.

Personally, in that case, I'd use:

python3 -m http.server

Re:Sherver: Bash lightweight web server by nickjj on Wednesday December 22, 2021 @08:42AM (#29649155)
That's my goto solution for a simple server if I'm testing something locally but it has issues dealing with concurrent requests in its simple form.

For example I ran a simple hello world test service on Kubernetes once using Python's http.server.

Just having 3 Kubernetes health checks happen on regular intervals would routinely cause the server to respond with errors. I think what ends up happening is if you have 2 requests coming in within a short enough period of time one of them will fail. You have to use a more complicated threaded http server if you want a zero dependency solution.

Re:Sherver: Bash lightweight web server by rascul on Wednesday December 22, 2021 @08:18AM (#29648899)
There are many similar options. I think that awhile ago there may have been a list of one liner web servers posted to HN. Either way, I've been using python's for years and it's just muscle memory now.
Re:Sherver: Bash lightweight web server by em500 on Wednesday December 22, 2021 @08:21AM (#29648952)
Big list of http static servers:

https://gist.github.com/willurd/5720255

Re:Sherver: Bash lightweight web server by adamddev1 on Wednesday December 22, 2021 @01:58PM (#29653537)
`npm install -g live-server`

Also works great (but a bit bulkier sure).

Re:Sherver: Bash lightweight web server by throwaway984393 on Wednesday December 22, 2021 @07:27AM (#29648329)
They reinvented CGI scripts but didn't actually implement CGI. Could have implicitly supported a lot of apps by default. There's some pretty decent CGI libraries for Bash, too.

Personally I use busybox's httpd (which does support CGI) for a small portable web server.

Re:Sherver: Bash lightweight web server by zoomablemind on Wednesday December 22, 2021 @11:11AM (#29651201)
Just an idea, along the lines of a single binary webserver: one may use Fossil SCM [1] to easily spawn a webserver ('fossil server', or even 'fossil ui'), no need for root (port 8080 by default).

The pages could be served as part of a project (all version-controlled), or as documentation pages, or as wiki. Styling is somewhat limited, but could still be done creatively.

Additionally, a static website or even a whole application (PHP or the likes) could be rolled out via /ext, the 'extension' path.

[1]: https://www.fossil-scm.org

Re:Sherver: Bash lightweight web server by medv on Wednesday December 22, 2021 @07:26AM (#29648317)
It is a socat server, not bash. Basically a bash script calling socat.

Definitely not “pure”.

Re:Sherver: Bash lightweight web server by prussian on Wednesday December 22, 2021 @07:26AM (#29648326)
Someone beat me to it. When I think of pure bash, I don't really think of invoking anything that isn't a builtin or a grammatical feature of bash. If socat is fair-play, almost anything you can spawn on a shell should count then.

edit: on top of that, I already found a perl script utility in the repo: https://github.com/remileduc/sherver/blob/master/scripts/uti...

Re:Sherver: Bash lightweight web server by bluetomcat on Wednesday December 22, 2021 @07:38AM (#29648435)
Jokingly, "sudo service apache2 start" may also be considered pure Bash :-)
Re:Sherver: Bash lightweight web server by fragbait65 on Wednesday December 22, 2021 @07:41AM (#29648478)
Exactly!
Re:Sherver: Bash lightweight web server by fragbait65 on Wednesday December 22, 2021 @07:35AM (#29648404)
I think I'll just write something similar and wrap Apache with a shell scrip, claim that it's a web server written in bash, post it here and see what reaction I get, just for shits and giggles.
Re:Sherver: Bash lightweight web server by prussian on Wednesday December 22, 2021 @07:41AM (#29648473)
Here: https://github.com/remileduc/sherver/issues/1 that way you can have HTTP/1.1 support even with arbitrary output lengths. I was planning on sending this to the linked repo so the owner doesn't need to use HTTP/1.0.

EDIT: changed link to the github issue I created with inlined utility.

Re:Sherver: Bash lightweight web server by tyingq on Wednesday December 22, 2021 @09:08AM (#29649450)
Bash can't do listen sockets, so I don't think a pure bash web server is possible. You can make an "pure" gawk web server, though it works terribly https://gist.github.com/willurd/5720255#gistcomment-3143007
Re:Sherver: Bash lightweight web server by junon on Wednesday December 22, 2021 @07:51AM (#29648583)
~~Right. "Pure" bash would mean they somehow got sockets working in the scripting language itself, which is not possible unless you're on a system with `/dev/tcp` (which isn't many these days).~~

I am wrong! See response.

Re:Sherver: Bash lightweight web server by rzzzt on Wednesday December 22, 2021 @08:02AM (#29648737)
/dev/tcp is a Bash construct (mentioned as "pseudo-device file" in the manual): https://tldp.org/LDP/abs/html/devref1.html
Re:Sherver: Bash lightweight web server by junon on Wednesday December 22, 2021 @08:07AM (#29648784)
Oh huh, TIL it's only a Bash thing. I stand corrected - I thought it was a Linux thing that was removed for security reasons.

Thanks for correcting me :)

Re:Sherver: Bash lightweight web server by loeg on Wednesday December 22, 2021 @01:56PM (#29653501)
Re:Sherver: Bash lightweight web server by dang on Wednesday December 22, 2021 @01:38PM (#29653291)
Ok, we've impured the title above.
Re:Sherver: Bash lightweight web server by rascul on Wednesday December 22, 2021 @07:44AM (#29648513)
This seems like an odd usage of cat and command substitution to me:

    HEAD_TEMPLATE=$(cat <<EOF
        <title>Sherver example</title>
        <meta name="description" content="Sherver example">
    EOF
    )
https://github.com/remileduc/sherver/blob/master/scripts/ind...
Re:Sherver: Bash lightweight web server by rascul on Wednesday December 22, 2021 @08:06AM (#29648768)
I believe the pure bash way of assigning a multi line string to a variable, without dealing with quoting, might be as follows:

    IFS='' read -r -d '' HEAD_TEMPLATE <<'EOF'
        <title>Sherver example</title>
        <meta name="description" content="Sherver example">
    EOF
https://stackoverflow.com/questions/1167746/how-to-assign-a-...
Re:Sherver: Bash lightweight web server by throwaway984393 on Wednesday December 22, 2021 @08:31PM (#29657689)
I mean, if it's got no single quotes:

  HEAD_TEMPLATE='
  <title>Sherver example</title>
  <meta name="description" content="Sherver example">
  '
Re:Sherver: Bash lightweight web server by rascul on Thursday December 23, 2021 @07:10AM (#29661063)
Yes this is indeed simpler, however I noticed in some of the code single quotes were also used [0] so I wanted to use an example that would work for that.

[0] https://github.com/remileduc/sherver/blob/master/scripts/ind...

Re:Sherver: Bash lightweight web server by junon on Wednesday December 22, 2021 @07:50AM (#29648570)
It achieves multiline block text in bash, which is otherwise a PITA to reason about. It can keep certain scripts cleaner, especially since `cat` is often inlined, without process spawning.
Re:Sherver: Bash lightweight web server by steerablesafe on Wednesday December 22, 2021 @07:53AM (#29648608)
> especially since `cat` is often inlined, without process spawning.

That would be surprising. How would that work? `cat` is definitely not a bash built-in.

Re:Sherver: Bash lightweight web server by junon on Wednesday December 22, 2021 @07:55AM (#29648641)
Cat is not a specified built-in but many implementations do build it in to speed up scripts that use it quite heavily.

Other examples include `which`, `time`, `echo`, etc. They're usually promoted to built-ins when it's clear that's the intended command to run, but there are still analogs on the PATH if need-be - I'm not sure exactly how bash would detect a non-standard executable there to know when it's unsafe to promote to a built-in, though.

For example with echo:

    $ bash -c 'echo hi'
    hi
    
    $ strace bash -c 'echo hi' | grep 'exec'
    execve("/usr/bin/bash", ["bash", "-c", "echo hi"], 0x7fffe264aad0 /* 24 vars */) = 0
    
    $ strace bash -c '$(which echo) hi' | grep 'exec'
    execve("/usr/bin/bash", ["bash", "-c", "$(which echo) hi"], 0x7fffe36c6cd0 /* 24 vars */) = 0
    execve("/usr/bin/echo", ["/usr/bin/echo", "hi"], 0x7fffe2e16ff0 /* 25 vars */) = 0
In fact for me, `cat` always gets inlined, even if I call it with its full path or with `which cat`.

There are cases where the built-in and the actual executable differ - the most prominent case being `time`. The built-in does not have the command line arguments to print out things like pre-empted scheduling (forced context swiches).

    $ time --help
    --help: command not found
    
    $ /usr/bin/time --help
       Usage: /usr/bin/time [-apvV] [-f format] [-o file] [--append] [--verbose]
          [--portability] [--format=format] [--output=file] [--version]
          [--quiet] [--help] command [arg...]
Re:Sherver: Bash lightweight web server by steerablesafe on Wednesday December 22, 2021 @08:55AM (#29649301)
"echo" and "time" are explicitly bash built-ins, "which" and "cat" are not, and you will always get a process spawned there.

Your second strace is misleading, you didn't trace child processes.

    $ strace bash -c '$(which echo) hi' 2>&1 | grep 'exec'
    execve("/bin/bash", ["bash", "-c", "$(which echo) hi"], 0x7ffddfcdaee0 /* 59 vars */) = 0
    execve("/bin/echo", ["/bin/echo", "hi"], 0x55eb565cb0b0 /* 59 vars */) = 0
    
    $ strace -f bash -c '$(which echo) hi' 2>&1 | grep 'exec'
    execve("/bin/bash", ["bash", "-c", "$(which echo) hi"], 0x7ffd09b3e228 /* 59 vars */) = 0
    [pid 81493] execve("/usr/bin/which", ["which", "echo"], 0x5560d172e0b0 /* 59 vars */) = 0
    execve("/bin/echo", ["/bin/echo", "hi"], 0x5560d172e0b0 /* 59 vars */) = 0
Re:Sherver: Bash lightweight web server by rascul on Wednesday December 22, 2021 @08:13AM (#29648841)
You had:

    $ strace bash -c 'echo hi' | grep 'exec'
    execve("/usr/bin/bash", ["bash", "-c", "echo hi"], 0x7fffe264aad0 /\* 24 vars */) = 0
Something seems off. strace(1) prints the trace to stderr not stdout, so you should need |& not | to catch it with grep. At least, that's how it works here.

Also, cat does not appear to be inlined for me:

    rascul@smarts:~> strace bash -c 'cat /etc/os-release' |& grep 'exec'
    execve("/usr/bin/bash", ["bash", "-c", "cat /etc/os-release"], 0x7ffe7b5334e0 /* 88 vars */) = 0
    execve("/usr/bin/cat", ["cat", "/etc/os-release"], 0x562c447076f0 /* 88 vars */) = 0*
Re:Sherver: Bash lightweight web server by junon on Wednesday December 22, 2021 @08:15AM (#29648863)
Yeah good catch, I forgot it since I didn't copy 1:1 the command line into vim when I formatted it for HN. I use fish + powerline so the command line doesn't copy correctly. In fish, it's `&|`, which would look weird anyway. I just forgot to add the `&` after the `|`.

The output was copied 1:1 though.

The inlining is an implementation detail and I've seen it happen differently across different platforms.

Re:Sherver: Bash lightweight web server by rascul on Wednesday December 22, 2021 @08:21AM (#29648947)
> The inlining is an implementation detail and I've seen it happen differently across different platforms.

Do you have more information on this? I would be interested to read more about it. As far as I was aware (and my knowledge is certainly not exhaustive), the only thing kinda related is as noted in the bash man page:

> The command substitution $(cat file) can be replaced by the equivalent but faster $(< file).

Re:Sherver: Bash lightweight web server by junon on Wednesday December 22, 2021 @08:26AM (#29648987)
I don't, sorry. I just know it's something some implementations opt-in to (especially drop-in replacements to bash). It's behavior I've observed quite a bit and have leveraged it in some cases to improve the performance of some critical bash scripts over the years. I'd also be interested in some prose about it.
Re:Sherver: Bash lightweight web server by prussian on Wednesday December 22, 2021 @08:13AM (#29648845)
Are you sure?

    strace -ff -e execve /bin/bash -c 'x="$(cat <<< "test")"'
    ... snip
    [pid 311214] execve("/usr/bin/cat", ["cat"], 0x56280da76fc0 /* 3 vars */) = 0
Re:Sherver: Bash lightweight web server by junon on Wednesday December 22, 2021 @08:16AM (#29648889)
Yep, I'm sure. It's not a guarantee (nor is it specified, to my knowledge) that the inlining happens. You also used a subshell there, which might make a difference.
Re:Sherver: Bash lightweight web server by BeefWellington on Wednesday December 22, 2021 @08:32AM (#29649043)
The original poster is using a subshell. Your example uses echo, which is both a built-in AND an external program.

They are not comparable. I've never seen a shell inline cat, probably because it would make no sense given there are POSIX builtins to do exactly what cat does.

Re:Sherver: Bash lightweight web server by junon on Wednesday December 22, 2021 @09:14AM (#29649535)
It of course makes sense since `cat` is widely used for the purpose of the original code, just not in a subshell. It's used often to output blocks of text to stdout/err for multiline output, namely --help text.

strace doesn't lie here, and I've seen it substitute cat in trivial cases quite often.

Re:Sherver: Bash lightweight web server by BeefWellington on Thursday December 23, 2021 @12:42AM (#29659015)
Indeed strace doesn't lie but you have yet to provide an example of a system where what you're saying about cat is true. You've shown a number of examples which do not actually back up the claim you are making, and have not responded to other posters pointing out the flaws in the examples you did provide.

I'm genuinely curious to see an implementation but I suspect you've misunderstood something in the past and your statement was incorrect.

Re:Sherver: Bash lightweight web server by frutiger on Wednesday December 22, 2021 @08:17PM (#29657628)
> They're usually promoted to built-ins

There’s no runtime “promotion”. You can read about it all in bash(1).

Re:Sherver: Bash lightweight web server by IceWreck on Wednesday December 22, 2021 @07:43AM (#29648493)
If this is pure bash then so is

``` caddy file-server --browse --listen :80 ```

Re:Sherver: Bash lightweight web server by remote on Wednesday December 22, 2021 @07:32AM (#29648381)
Kudos for scripting a great hack in BASH.

Since the title reads "pure bash" however I was expecting to see an implementation using builtin bash features such as /dev/tcp.

Have you considered solving this problem using BASH builtin support for TCP?

Re:Sherver: Bash lightweight web server by rascul on Wednesday December 22, 2021 @07:43AM (#29648499)
I recall that bash can't create a server socket, because bash doesn't bind().
Re:Sherver: Bash lightweight web server by MisterTea on Wednesday December 22, 2021 @10:10AM (#29650315)
Plan 9's rc-httpd is quite useful, supports cgi and serves many plan 9 web sites, many of which run werc http://werc.cat-v.org/

It can also run on Unix via plan 9 port (p9p) or derivatives.

https://github.com/9front/9front-test/blob/master/rc/bin/rc-...

Re:Sherver: Bash lightweight web server by Brian_K_White on Wednesday December 22, 2021 @08:01AM (#29648718)
Ah, no. Bold claim unrealized. Immediately I see many unnecessary forks and externals to do things that bash actually can do itself.
Re:Sherver: Bash lightweight web server by montroser on Wednesday December 22, 2021 @07:28AM (#29648343)
For the occasional service where you are essentially just wrapping an existing binary, a bash server behind uwsgi can be totally sensible.
Re:Sherver: Bash lightweight web server by montroser on Wednesday December 22, 2021 @07:24AM (#29648308)
Fantastic. For a more sinatra-inspired take, see also https://github.com/guigo2k/ganesh
Re:Sherver: Bash lightweight web server by notRobot on Wednesday December 22, 2021 @09:32AM (#29649818)
See also:

darkttpd: When you need a web server in a hurry.

https://github.com/emikulic/darkhttpd

Re:Sherver: Bash lightweight web server by laumars on Wednesday December 22, 2021 @07:28AM (#29648340)
> I didn't want to install and configure Apache or nGinx. In fact, I didn't want any configuration.

If you ignore all of the manual installation instructions and any dev time you took to create this, then sure.

I’m not going to argue that the author should have used something else instead though. It’s a personal project used purely on an internet system of his house. For those kinds it projects “because I could” is as good a reason as any. I just think more authors should be honest and say “I just did it because it seemed fun at the time”

Re:Sherver: Bash lightweight web server by tjoff on Wednesday December 22, 2021 @11:36AM (#29651576)
Manual instructions and dev time doesn't necessarily pollute the state of the machine though and are harder to replicate, which is how I would interpret it.
Re:Sherver: Bash lightweight web server by laumars on Wednesday December 22, 2021 @11:59AM (#29651949)
Not having your application packaged up for quick removal strikes me as more polluting. And I don’t really see how an independent solution with almost no documentation and zero configuration management support (eg Ansible) is easier to replicate than a battle tested industry standard with widespread support.

Don’t get me wrong, I have nothing against building things for fun. But the technical arguments made for why this is preferable over an established solution doesn’t stack up.

Re:Sherver: Bash lightweight web server by tjoff on Wednesday December 22, 2021 @12:13PM (#29652183)
I do.

Just as I would use this (would look into it more first though), the netcat trick mentioned here or python3 -m http.server over ever considering installing Ansible and/or nginx/apache.

Totally different use case though.

Re:Sherver: Bash lightweight web server by laumars on Wednesday December 22, 2021 @12:25PM (#29652351)
If I was rolling my own and concerned about polluting, I’d favour a language like Rust or Go because I then don’t need to install the Python runtime, have random undocumented dependencies for the bash etc.

Or I’d use Docker.

But as said before, there’s no wrong answer here since it’s just a hobby project running on a HTPC.

Re:Sherver: Bash lightweight web server by tjoff on Wednesday December 22, 2021 @01:33PM (#29653217)
With that I agree, I would only use python if it was already installed, and the assumption with bash was that everything needed was already installed.
Re:Sherver: Bash lightweight web server by adamddev1 on Wednesday December 22, 2021 @01:53PM (#29653463)
Very cool. Was hoping to see Sean Connery in the README.
Re:Sherver: Bash lightweight web server by pkrumins on Wednesday December 22, 2021 @10:10AM (#29650310)
Also `php -S` is very handy.
Re:Sherver: Bash lightweight web server by vkazanov on Wednesday December 22, 2021 @07:18AM (#29648262)
One those things that is s scarily beautiful
Re:Sherver: Bash lightweight web server by moondev on Wednesday December 22, 2021 @08:38AM (#29649108)
go run github.com/patrickhener/goshs@latest