News for hackers. Stuff that matters.
Re:The Bash Hackers Wiki by ofrzeta on Friday February 21, 2020 @07:36PM (#22388505)
I've come to use more and more Python for tasks I would have used Shell in the past. Because of weird syntax with anything but simple variables or endless fiddling with quoting.

You can get a lot done quite easily and concise in Python 3, for instance with the pathlib module that has methods for recursive globbing and stuff like that.

https://docs.python.org/3.7/library/pathlib.html

Re:The Bash Hackers Wiki by badsectoracula on Friday February 21, 2020 @11:54PM (#22389506)
The nice thing about Bash which can obviously not be said about Python is that Bash scripts will keep working (assuming whatever they call is also there of course) since Bash goes out of its way to preserve backwards compatibility. Considering its legacy, Bash can run scripts more than a decade before Python even existed.

The only other scripting language i know of that has similar backwards compatibility is Tcl. Perhaps Perl too (ignoring Perl 6 that was renamed to something else again due to it being practically a different language).

Re:The Bash Hackers Wiki by Doingmything123 on Saturday February 22, 2020 @12:10AM (#22389542)
I guess it's the trade off between backwards compatibility and readability/correctness/maintainability. Both languages will have technical debt overtime. In my opinion, I would much rather have to rarely update python scripts in exchange to for peace of mind that I can fix/update the script as need be. In my experience, *sh scripts can be fragile and hard to ensure correctness.
Re:The Bash Hackers Wiki by Aloha on Saturday February 22, 2020 @03:54AM (#22390062)
Having migrated something recently from python2 to python3, I'll take bash. I actually ported two of the three scripts to bash, and the third one I reluctantly moved to python3 (I needed python-pil). String handling in python3 in my opinion is a tire fire compared to bash.
Re:The Bash Hackers Wiki by lizmat on Saturday February 22, 2020 @06:17AM (#22390439)
> Perl 6 that was renamed to something else

Indeed, it has been renamed to Raku (https://raku.org using the #rakulang tag on social media).

Re:The Bash Hackers Wiki by ekianjo on Saturday February 22, 2020 @06:47AM (#22390541)
Python needs to be installed, maintained, and every single library will have its list of dependencies (along with breaking API changes once in a while). There's a reason Bash exists: it's everywhere by default and you know it will still work as it works now in x years.
Re:The Bash Hackers Wiki by epr on Saturday February 22, 2020 @07:07AM (#22390594)
I'm sorry, but for the type of work he is suggesting (bash script equivalent), you don't need anything besides the standard library. I cannot think of a single system I have used in recent decades that didn't come with python installed by default. For all intents and purposes, Python is as ubiquitous as bash at this point.
Re:The Bash Hackers Wiki by vaingloriole on Saturday February 22, 2020 @10:10AM (#22391363)
Balderdash and hokum. z=1 ; while test ${z} -lt 4 ; do a=${RANDOM:1:3} ; dd if=/dev/urandom count=12 bs=4 of=${a} && b=$(cat ${a}) && echo ${b:$(expr 1 + ${RANDOM} % 4):12} ; ((z++)); done

You have one tool that you know well. I have thousands of tools that I've learned daily and I wrap with a bash script..except when I need python/tcl or C. Most of what I do in python/tcl/C is truly complicated, needs to be fast or use a data structure other than an array: as it should be. Most of what I do in the shell is as simple as the userspace tools and bash will let it be. Write the one liner above in python without importing a library.

Re:The Bash Hackers Wiki by orf on Saturday February 22, 2020 @07:07PM (#22394201)
Pretty sure it's just this:

> python -c "for i in range(4): print(__import__('os').urandom(12)[__import__('random').randint(1,4):12])"

Or something like it, that bash one liner is pretty unreadable and full of side effects that may or may not be needed. A small 3 line python file would be cleaner and more readable.

Re:The Bash Hackers Wiki by qilo on Saturday February 22, 2020 @07:49PM (#22394368)
TIL about Python __import__ function. Thanks!
Re:The Bash Hackers Wiki by ofrzeta on Saturday February 22, 2020 @10:14AM (#22391386)
What's the point of not importing a library/module? It's not like cheating or something.
Re:The Bash Hackers Wiki by dijit on Saturday February 22, 2020 @01:26PM (#22392452)
Eh. Pros and cons.

I think most people are spoiled by being able to install anything; I work in a locked down environment and getting anything non-sanctioned in is very difficult, and the sanctioning process is tedious if nothing else.

That, and the fact that libraries are evolving- which is why we end up with horrible things like package/library version pinning.

Re:The Bash Hackers Wiki by ofrzeta on Saturday February 22, 2020 @01:42PM (#22392534)
Sure, but we're talking (with the problem at hand) about Python standard libraries that get shipped with the Python distribution and not some NPM or pip universe.

For instance the "os" module has an "urandom" function that does essentially the same as the "dd" in the bash script. Of course you could also read from the device so you wouldn't need to import "os". You would need to import "random" to get access to the random number generator, though, but it's still part of the standard library.

Re:The Bash Hackers Wiki by thewhitetulip on Saturday February 22, 2020 @09:11AM (#22391104)
We use AIX at work and it didn't have Python installed.

We had to install it for a code review tool I built.

But despite Bash's weird syntax, I love how we can do very complex tasks within a few lines.

Re:The Bash Hackers Wiki by tjoff on Saturday February 22, 2020 @07:41AM (#22390711)
My phone and router have bash.

Even if pyhton is installed it isn't certain if it is v.2 or 3.

Re:The Bash Hackers Wiki by labawi on Saturday February 22, 2020 @08:31AM (#22390884)
My router has busybox (ash), but not bash.

Neither bash nor python seem to be preinstalled on BSDs.

Re:The Bash Hackers Wiki by nchelluri on Saturday February 22, 2020 @08:12AM (#22390810)
Sounds like Stockholm syndrome to me!
Re:The Bash Hackers Wiki by mhd on Saturday February 22, 2020 @10:55AM (#22391592)
That's great, as long as you're happy with the POSIX Unix utilities (probably, wouldn't bet my life on it) and whatever Bash 3.x provides.

This can break quite fast if you rely on something from Bash 4 (released 11 years ago, still not on OS X by default), or some command-line tools that's either not installed or uses switches/functions only available in their GNU incarnations.

For anything that involves more than one file or with a line count going into triple digits, I wouldn't recommend bash. And heck, for those kind of scripts, I'd go one step further and stick with plain 'sh', just to be really sure. Bash adds little. GNU missed the train of transitioning to something like ksh93, probably because RMS thought that everyone would be using Guile once Hurd is out. (I once worked with a pretty large piece of software for packaking/administration mostly written in ksh93, as this was more palatable for commercial Unix users -- not as important these days where you won't run into AIX, HP-UX and Irix anymore)

For anything more involved than a few straightfoward lines, I'd probably still pick Perl 5. Very common, especially as some operating systems/distros have it in their core tooling (e.g. CentOS) and because it's included in git, so even available on Windows. Standard functionality is good enough and it's no big jump from sh/awk/sed. Rather do that before I depend on some obscure gawk function or gsed switch, especially as it's really hard to tell whether that's proprietary or in what version it was first introduced. Cross-browser checking is easier these days than cross-distro/unix.

Python does have a better/bigger standard library, but isn't as common and you might even run into Python 2/3 shenanigans these days.

Still, the situations where you have something moderately complex and can't install stuff should be pretty rare. Might not be able to do it for the whole system, but things like perlbrew/rbenv/nvm/pyenv-du-jour solve that mostly. At that level, Go might be an option, too.

Re:The Bash Hackers Wiki by hannob on Saturday February 22, 2020 @10:36AM (#22391494)
Actually I have recently analyzed some python code that does some shell calls and if I can eliminate them with pure python.

It was quite frustrating. A lot of stuff that the shell has simple functionality for can't be done easily in python.

Examples:

* Call to chown -R. Sure, python can change file owners (os.chown()). But it has no parameter to do this recursively. You need to build a loop around it over some file iteration function.

* Call to setfacl. Can't be done with the python standard library, needs pylibacl and you can't rely on it being installed.

* Call to killall -HUP [processname]. You can send kill signals with python, but only to pids, not to process names. You'd have to parse the system process list or something like that in order to simulate.

In all cases I decided to keep the shell call, it just wasn't worth it converting that to less readable python code.

Re:The Bash Hackers Wiki by ofrzeta on Saturday February 22, 2020 @11:04AM (#22391637)
Fair enough, choosing the right tool for the job. On my own servers I can take care that the requires modules are installed and even use virtual envs for stable library versioning. This is how we do Let's Encrypt renewal because I can use a Python library for Route53 authentication that works.

Your first example could be solved with something like

[os.chown(p, uid, gid) for p in pathlib.Path('.').rglob('*')]

Obviously this is more verbose than chown but I think it's ok in a script.

Re:The Bash Hackers Wiki by ShorsHammer on Saturday February 22, 2020 @12:42AM (#22389630)
Having watched people hack together utterly bizarre bash scripts with awk and grep to handle json responses I'd very much agree here.

And before someone raises jq, what % of servers already have python installed vs jq?

Re:The Bash Hackers Wiki by heavyset_go on Saturday February 22, 2020 @12:55AM (#22389657)
> And before someone raises jq, what % of servers already have python installed vs jq?

How many of those servers have something like requests installed out of the box vs needing to fetch it via a package manager or PyPI?

jq is available in Ubuntu's universe repository along with requests.

Re:The Bash Hackers Wiki by ShorsHammer on Saturday February 22, 2020 @02:24AM (#22389848)
it's python, requests is one of it's many libraries for the purpose, would actually be quite concerned hiring someone who doesn't know the multiple stdlib solutions for this and still claims ability in python.

import json

import urllib.request

tough hey

Re:The Bash Hackers Wiki by heavyset_go on Saturday February 22, 2020 @02:55AM (#22389920)
I'd be as concerned with someone not opting to use what the documentation recommends[1] in lieu of the stdlib, as I would someone who puts together an awkward Bash script to parse JSON instead of just installing the 50kb package for jq.

> The Requests package is recommended for a higher-level HTTP client interface.

[1] https://docs.python.org/3/library/urllib.request.html

Re:The Bash Hackers Wiki by ShorsHammer on Saturday February 22, 2020 @05:33AM (#22390298)
It's a lot easier to hire people than fire them friend.

The documentation recommends nothing of the sort, it's a direction for consumers. We all live in varied and different worlds, how many varied server environments have you been responsible for all at once? Is it really beyond your amazing programming skills to use the tools at hand in every circumstance? Perhaps it's a chance to demonstrate yet another 3rd party tool which fixes non-existent problems.

It's a few bytes difference in your script that is using requests over the numerous and plentiful stdlib options there.

Re:The Bash Hackers Wiki by vaingloriole on Saturday February 22, 2020 @10:23AM (#22391427)
Would never use python in lieu of curl/jq for json handling unless I am writing something more complicated than a shell script. catch 22.
Re:The Bash Hackers Wiki by ShorsHammer on Saturday February 22, 2020 @09:42PM (#22394749)
If it gets the job done good on you.

Each to their own. Simply saying what I would look for.

Have any examples of your apt install jq script? How would you install jq on a few hundred machines also?

I'd be happy to share an example of an out of the box python script that does the same job and works virtually everywhere.