You can use Ruby in Make. I don't know Ruby but here's some Python:
SHELL = python
.SHELLFLAGS = -c
.ONESHELL:
.DELETE_ON_ERROR:
foo.txt: bar.json
import json
with open('$<') as input, open('$@', 'w') as output:
output.write(json.load(input)['the_text'])
(Caveat: I've never actually tried this at scale.)You can even use different languages for different recipes with per-recipe variable settings.
At that point I think you might as well just use a language-native one.
Great pull, though.
I think you're (understandably) misinformed about what Makefiles do because you've run into some bad ones. The thing they're doing is managing a N-level deep dependency tree in a declarative way. So if A->B->C you can run something to generate C, then B can run, and finally A, and this can all be done in parallel for hundreds of files.
On the individual rule level this is really simple, e.g. just turning a .c file into a .o file, then finally some rule that depends on all *.o being generated creates a program out of them.
The language-native ones are usually much worse. They're easier to use at the outset because they don't make you create this dependency graph, but that also means that they can't run in parallel on your N cores, and they usually suck at incrementally updating the build.
I'm not sure what you mean? How would this allow you to use, say, Python as the language for recipes? Just having Make drop straight into Python kind of defeats the purpose of Make.
The point is that people conflate these two things. They open some random Makefile and see that it's mostly doing stuff with shellscripts, and think "oh I should do this all in Python", and then write some monstrosity that doesn't make a dependency DAG and thus can't run in parallel or be easily understood.
Instead they should have split the actual logic they found in shellscripts in to Python scripts the Makefile invokes.
Is it that hard to learn shell? Why is it so painful? What makes it the "lowest common denominator"? I use it all the time, but I admit at work I am one of the few.
> Expecting everything to at all times be written in and for that lowest common denominator is not reasonable. We're a tool-using species and we refine tools over time to make them better.
This is too vague. What makes a Ruby based build or JS based build a more "refined" tool? It sounds like familiarity is the real issue here.
> If old tools were sufficient, people would use them because learning new ones is hard.
How many people even know Makefiles these days anyway? The "modern" approach seems to be, learn a programming language and then try to do everything inside of it. Some languages are more interested in this cloistered philosophy than others (like JS).
If anything, I think the reason these build tools keep being proliferated is because nobody wants to learn anything more than the bare minimum to "be productive" (which, depending on what you're working on, can be anything from pushing out customer demos for a company that will never sell, to microservices operating at scale). Learning a language and never leaving its paradigms/comforts is easy.
That said, Maven is incredibly suck-tastic.
(You may be a wizard and have a reason to do them, for sure--but that's what writing Maven plugins is for. Or not using Maven. You've got choices.)
https://github.com/ingydotnet/json-bash
source json.bash
json='{"name":"Jason","friends":["Jimmy","Joe"]}'
JSON.load "$json"
joe=$(JSON.get /friends/1)
JSON.put /friends/2 Jeff
new_json=$(JSON.dump)
It's still not as trivial as javascript, but "tooth-pullingly difficult" is a little unfair.That is blatantly false, but I get the premise behind it. Writing and maintaining complex stuff in bourne or bash is not fun or easy.
I always use bourne until certain level of complexity or awkwardness is reached. It's pretty easy to write and dead simple to troubleshoot.
And yeah, I did say `sh`, because that is what you can practically be expected to have kicking around alongside `make` on a system where I can't just install something worthwhile. If I have them, then there's no reason to write much harder to troubleshoot shell scripts (and, thank you quotebarf, more likely to be incorrect) than to open Pry.
> --show me its arrays
declare -a
> Show me hashes. declare -A
> Show me sets.See above
> Show me the basic building-block primitives of software,
thing() {
local localVar
# Yes, it would be nice to have lexical scope as well as
# dynamic scope, but...
}And besides: if I can install bash, I can install something specifically better for whatever I'm doing instead.
What are these build systems where you need to install Bash? Bash 4 was released in 2009: it's been in every major Linux distribution for at least two major versions, ditto for FreeBSD... heck, even Solaris ships with it now.
And Busybox is commonplace. Systems that include Busybox usually do not include `bash`. And so, for my purposes, if I can specify `bash`, I can also specify, say, Ruby, which--while by no means perfect--makes life much, much easier.
"All of which are clunky, opaque, and harder to work with than any other language I have used in the last decade."
Wrong question. A better one: Is it more clunky, opaque, and harder to work with than every other language that's appeared in the last decade? Because no one seems to agree on what is specifically better.
(And I am what a current sysadmin would be if we did not call ourselves "devops engineers" now.)
You seem to be the sort of developer that developers love and the sort of sysadmin that gets fired in the first week Just my .02 after two and a half decades.
Not to say that you are entirely wrong or your approach doesn't have merit in the new world (especially SV). But it doesn't work for sysadmins and production environments anywhere but your bubble. Not yet.
Anyway, the core mission of anybody touching the stack is to enable the business to achieve its goals. Nothing, and I mean nothing, more. "Restricting that environment" is appropriate in some environments, and a number of my clients bring me in to help with that. Facilitating developer velocity--and, yes, developers do tend to like me, because I'm good at this while achieving goals around security and uptime--is appropriate in, probably, more. Pays better, too, even if it shouldn't.
1. Design. 2. Discipline.
Where these two values are dispensable long term devops and the new world shine through. I've worked in both worlds and the only mistake is assuming one size fits all.
There were guys like this that were there before and helped everyone along: https://www.computerhope.com/people/don_libes.htm
I'm still not certain what point you were trying to make with "neithere here nor there", however.
I love Bash and use it way more often than I should, but Bash is not a friendly language. Mocking and stubbing functions is really hard to do, which makes tests awkward, even with BATS. And you need to watch the file descriptor to which text is sent to avoid things get globes unexpectedly. And you need to properly scope your vars to avoid unexpected collisions. And everything regarding sourcing and importing dependencies. Etc.
This is an anti-feature. When you need this, your build is too complicated.
Million+ line codebases got errm made with make. In my experience people most sneering at it have far lesser demands on their tool...
The world advances.
I use Rake (or Gulp, or whatever) because then I can use Ruby (or JavaScript, or whatever). Shell plumbing is fine for informal and small-scale stuff, and I make my code conform if somebody down the line (who may be me) wants to get out their duct tape, but the world is more complex than what /bin/sh can see. Shell is the lowest common denominator. Expecting everything to at all times be written in and for that lowest common denominator is not reasonable. We're a tool-using species and we refine tools over time to make them better. The profusion of tools happens because they iterate on each other to be better. If old tools were sufficient, people would use them because learning new ones is hard.
So, yes, you do need to learn those tools. Or invent a shell that isn't tooth-pullingly difficult to use with a JSON file (and do not say `jq`, I love `jq` as an inspector but it does not step to `JSON.parse` and a working subscript operator). Or change `make` so that a git checkout won't trigger a full rebuild. Lots of baseline, stump-simple things that `make` is just not going to do for you because it's built for a frankly outmoded method of development.