News for hackers. Stuff that matters.
Re:My deployment platform is a shell script by anonzzzies on Wednesday April 10, 2024 @07:20AM (#39989406)
I use similar things for bigger (multi-server) deploys too. It's light and it just works and works for decades without changes/updates. People say it's brittle; I have a proof of n>0 that this is not the case compared to many other solutions, this post making that point too. Sh/bash/perl(8) have been around forever, they don't break after update etc.

I sadly don't recommend it for my day job, simply because of liability. When something messes up with ansible, terraform, docker, cloudformation etc, no-one gets any blame because 'complex systems', 'it happens' etc etc; with a simple script going wrong, they would hang me high even though it probably saves a crap load in maintenance, compute etc over the 10s of years. Same reason we use clusters and IaC while of course nothing we do needs it; if an aws cluster goes down, no-one but aws gets blamed, while if the $2 postgres vps@cheapafhosting (with an higher uptime than that aws cluster by the way; human error downed it a few times, short, but still) is down even for a ping, everyone is upset and pointing fingers.

Re:My deployment platform is a shell script by cqqxo4zV46cp on Wednesday April 10, 2024 @08:30AM (#39989950)
Sounds like you’ve got an axe to grind. You’re saying nothing about what happens when you leave and nobody wants to maintain your mess of shell scripts.
Re:My deployment platform is a shell script by runlaszlorun on Wednesday April 10, 2024 @09:09AM (#39990296)
I don’t think ‘axe to grind’ was what I took away from the comment. And if it’s a ‘mess of shell scripts’, I can’t imagine any of the mentioned solutions to this being better.

I think we lose focus of the fact that we invented a whole new career track- devops. And it basically is about getting code from developers’ machine or environments to production. And we’re still terrible at it.

Perhaps that should be a signal that we’re doing it wrong…

Re:My deployment platform is a shell script by candiddevmike on Wednesday April 10, 2024 @09:57AM (#39990810)
There is a constellation of companies (living and dead) that have attempted to solved this problem in different ways because everyone thinks we're doing it wrong. The reality is every company has different needs. There isn't (will never be?) a one-size-fits-all approach to deployments because they're the fun intersection of people/process/technology.

The best deployment pipeline is the one that no one notices. Everything else is yak shaving/bike shedding.

Re:My deployment platform is a shell script by mattbuilds on Wednesday April 10, 2024 @09:04AM (#39990252)
It’s funny because in my many years of development I don’t think I’ve ever encounter a “mess of shell scripts” that was difficult to maintain. They were clear, did their job and if they needed to be replaced it was usually simple and straightforward.

Can’t say the same for whenever the new abstraction of the day comes along. In my experience what the OP is saying is exactly my experience. The abstractions get picked not because they are best but because they reduce liability.

Re:My deployment platform is a shell script by from-nibly on Wednesday April 10, 2024 @09:14AM (#39990337)
Hello. I have found the mess of shell scripts. Please don't do this.

I was able to deal with the weird skaffold mess by getting rid of it, and replacing it with argocd. I was able to get rid of jenkins by migrating to github actions. I have yet to replace the magic servers with magic bash scripts. They take just enough effort that i can't spend the time.

Use a tool i can google. If your bash script is really this straight forward and takes you from standard A to standard B, and it's in version control then bash is AMAZING. Please don't shove a rondom script that does a random thing on a random server.

Re:My deployment platform is a shell script by Danjoe4 on Wednesday April 10, 2024 @04:05PM (#39995157)
Bash is good but can grow out of control. The problem is solo engineers and managers who push/approve 500+ line bash scripts that do way too much. A good engineer will say it's getting too complicated and reimplement it in Python.
Re:My deployment platform is a shell script by ffsm8 on Wednesday April 10, 2024 @08:30PM (#39997209)
Wasn't there a rule about that?

Something like "in software development the only solution that sticks is the bad one, because the good ones will keep getting replaced until it's so bad, nobody can replace it anymore"

Re:My deployment platform is a shell script by kragen on Wednesday April 10, 2024 @09:52AM (#39990752)
i have encountered messes of shell scripts that were difficult to maintain; in my first sysadmin job in 01996 i inherited a version control system written as a bunch of csh scripts, built on top of rcs

but they were messy not because they lacked 'abstractions' but because they had far too many

i think shell scripts are significantly more bug-prone per line than programs in most other programming languages, but if the choice is hundreds of thousands of lines in an external dependency, or a ten-line or hundred-line shell script, it's easy for the shell script to be safer

Re:My deployment platform is a shell script by chasil on Wednesday April 10, 2024 @10:42AM (#39991240)
If it was in RCS, then you could directly move the archives under a CVSROOT and use them natively.

CVS had been out since Brian Berliner's version of 1989.

I actually moved a PVCS archive into RCS->CVS this way, and I'm still using it.

Re:My deployment platform is a shell script by kragen on Wednesday April 10, 2024 @03:51PM (#39995035)
that version control system provided a number of facilities cvs didn't (locking, and also a certain degree of integration with our build system permitting the various developers to only recompile the part of the system they were working on, which was important because recompiling the whole thing usually took me about a week, once a month), but it had never actually occurred to me that turning an rcs repository into a cvs repository like that was a possibility. also i never realized pvcs used rcs under the covers. thank you very much
Re:My deployment platform is a shell script by chasil on Wednesday April 10, 2024 @04:57PM (#39995637)
PVCS did not use the RCS format, but the RPM distribution included a perl script to convert the archives.

  $ rpm -ql cvs | grep pvcs
  /usr/share/cvs/contrib/pvcs2rcs
Re:My deployment platform is a shell script by kragen on Wednesday April 10, 2024 @05:45PM (#39996042)
ooh. that would have been very useful two jobs later when i got stuck with pvcs
Re:My deployment platform is a shell script by redman25 on Wednesday April 10, 2024 @09:53AM (#39990767)
Shell seems great until your tens of lines in googling every other line of obscure error-prone syntax.
Re:My deployment platform is a shell script by anonzzzies on Wednesday April 10, 2024 @01:21PM (#39993284)
… or maybe you are not proficient at shell scripting? I never had this issue, including large projects written in tcl, bash or perl in the 90s when it was more normal to do so.

The modern answer seems to be some kind of dsl with yaml syntax mixed with Unix (and thus bash) snippets which are often incredibly verbose and definitely not easier to read than a well written bash script. The only thing I think of when I see those great solutions is; another greenspun’s tenth rule in action.

Re:My deployment platform is a shell script by red-iron-pine on Wednesday April 10, 2024 @02:32PM (#39994132)
bash and other sh related approaches have a lot of "foot guns". python, or powershell, or even C++ are often easier to read and follow.

> are often incredibly verbose and definitely not easier to read than a well written bash script

define well written -- getting into no true scotsman here.

bash is fine for what it was and what it did, and i'm glad to know enough sed and awk to be dangerous, but it's a PITA unless we're forced to use it

Re:My deployment platform is a shell script by forinti on Wednesday April 10, 2024 @09:26AM (#39990468)
Just yesterday I was working on a monitoring agent that was a python script deployed through gitlab-ci, and executed inside a docker container through logstash.

I bet you could do the same with about 5 lines of shell (wget|grep|curl -XPOST). It would be simpler, easier to modify, and it wouldn't ever break.

Shell scripts aren't necessarily messy or complex.

Re:My deployment platform is a shell script by kragen on Wednesday April 10, 2024 @09:54AM (#39990781)
python is a particularly bad choice for things that are not supposed to break; the python maintainers have adopted a new policy of intentionally breaking new things every release. recent casualties include asyncore, distutils, and imp

https://docs.python.org/3/whatsnew/3.12.html#removed

there are people who foolishly depend on python for things that need to work. i hope that at some point they band together behind a fork that doesn't pull this kind of nonsense, but for the time being, that hasn't happened. if you wrote stuff in python in 01995 or 02000 or 02005 it probably still worked in 02015, but today, basically no python from before 02015 works

Re:My deployment platform is a shell script by icedchai on Wednesday April 10, 2024 @11:22AM (#39991739)
Perhaps don't upgrade your Python willy-nilly? That seems like a sane solution to this problem.
Re:My deployment platform is a shell script by kragen on Wednesday April 10, 2024 @03:58PM (#39995095)
debian stable no longer has python 2. debian's pypy package has its own fork of cpython 2 in order to be able to build itself. numerous other projects are doing similar things

debian's versioning system for packages doesn't really contemplate needing to install old versions of a package because the maintainers are deliberately breaking new versions of it, so in 'bookworm' debian 12.5, the only cpython available to install is python 3.11. so if you want to 'not upgrade python' you are going to have to take on the maintenance burden of the old versions yourself; debian isn't going to help

some other package managers handle this kind of situation better (nix is notably excellent at it) but that doesn't really help with the maintenance burden of keeping up with security fixes

Re:My deployment platform is a shell script by icedchai on Wednesday April 10, 2024 @04:12PM (#39995228)
My "willy-nilly" comment meant "don't upgrade it without planning in advance" not "don't upgrade it, period." Part of that planning can be choosing to stay on an older LTS distro until you have time to properly migrate your code to a Python released in this decade.
Re:My deployment platform is a shell script by kragen on Wednesday April 10, 2024 @04:34PM (#39995424)
or you could just write the code in the first place in a language that doesn't have maintainers that intentionally break old code. that's what you would do if you aspire to write code that constitutes part of the intellectual heritage of humanity, as opposed to being an expedient way to resolve a problem you have this week, to be thrown away in a month or two. you can still compile c code that uses gets(), you know. a couple of years ago i fixed a perl cgi script i'd written in 01998 to work with current perl; it took me half an hour. and euclid's proofs are still valid 2500 years after he wrote them down
Re:My deployment platform is a shell script by icedchai on Thursday April 11, 2024 @09:29AM (#40001920)
C is a very slow moving language, but platform-specific APIs (meaning outside of stdlib) change more frequently. "C" does not exist in isolation, and outside of toy applications, not useful without the underlying platform APIs. If you were migrating C code, from say, Mac OS 7, or even Mac OS X 10.1, you would have to make many, many changes. If you're migrating code across Unix platforms (say, 1990's SunOS to modern Linux), you'll require changes, too.
Re:My deployment platform is a shell script by kragen on Thursday April 11, 2024 @12:56PM (#40004314)
sure, but the gratuitous breakage i'm complaining about in python is inside of stdlib, and even in the language syntax itself. and i've frequently taken c code from 1990s sunos and compiled it on modern linux, and yes, it's true that it usually requires a few changes where it interacts with the outside world—but only a few

it's true that very small programs are often little more than glue between the underlying platform apis, but large programs tend to have a much lower surface-to-volume ratio. that's why τεχ still runs, virtually unchanged since 01990, and you can easily render plain τεχ documents from the 80s or 90s with τεχ today—even with pdfτεχ

Re:My deployment platform is a shell script by adamtaylor_13 on Wednesday April 10, 2024 @09:04AM (#39990250)
If we’re using the script in the post as an example, that’s hardly a “mess” of shell scripts. I’d rather maintain that than almost any other build system I’ve ever seen.
Re:My deployment platform is a shell script by yjftsjthsd-h on Wednesday April 10, 2024 @11:00AM (#39991456)
Are you sure you're not the one grinding an axe? Shell scripts are a mess but YAML isn't? On the happy path they can both be fine, and on the unhappy path they both suck.
Re:My deployment platform is a shell script by icedchai on Wednesday April 10, 2024 @11:20AM (#39991711)
Most CI/CD configurations are also a mess. Often YAML, shell command snippets, weird conditional syntax... I can't say it's better.
Re:My deployment platform is a shell script by mdekkers on Wednesday April 10, 2024 @12:56PM (#39992930)
> nobody wants to maintain your mess of shell scripts.

Have you seen the scripts in question? Are you in any kind of way able to make such a value judgement over this persons work? Have you considered the impact of your choice of words on others?

Re:My deployment platform is a shell script by dang on Wednesday April 10, 2024 @10:49PM (#39997941)
You've been breaking the site guidelines frequently lately by posting much too aggressively. We have to ban accounts that won't stop doing this, so if you'd please stop doing this, that would be good.

HN is for thoughtful, respectful, curious conversation—not attacking other users or elbowing.

https://news.ycombinator.com/newsguidelines.html