This is a great question that I will make sure to answer in my Turing-completeness post in a week.
But to answer now, it's because of multi-project builds, where projects can be in multiple languages.
If you have build systems that are not independent from their language, how do you get them to play nice?
I designed Rig to be able to act as a babysitter and make them play nice.
If I play my cards right, Rig will be the reason other build systems become irrelevant. :)
https://github.com/aappleby/Hancho
In all seriousness, I think both our projects are aiming in the right direction, mine is just more focused on minimalism at the cost of having extra-pointy corner cases.
End users get to pick a familiar language to them.
Similar to how Pulumi is available in multiple languages.
Then code both builds and meta build concerns as you would code anything else.
EDIT: s/Skylark/Starlark/
The thing about Starlark is that it is effectively Turing complete.
You can simulate a while loop with a long enough foreach, an if statement, and a break statement.
And by "long enough," I mean long enough to make anyone just stop the build, but hey, if I stack the foreach loops, I can make the inner loop exponentially long, even though ints are limited to 32 bits.
By not taking out the break keyword, Starlark remains Turing-complete.
And yes, I tested this.
A primitive recursive language gets you most of the benefits of a general recursive one _without_ needing to be _actually_ Turing complete. So this isn't "bad news" for me. I'm saying that I think you should avoid using a general recursive (i.e. Turing complete) language for a build system and instead have guarantees that the system will terminate by using a primitive recursive language of some kind because you literally, as you did, need to go out of your way to get code written in such a fashion to run for an inordinate amount of time. And yet, an inordinate amount of time isn't forever and unlike a program written in an actually Turing complete language, you can use static analysis to detect that kind of thing if only primitive recursion is supported.
If it's only guaranteed to terminate after the heat death of the universe, it won't terminate, so I would call that "forever."
> unlike a program written in an actually Turing complete language, you can use static analysis to detect that kind of thing if only primitive recursion is supported.
And this is exactly why I said that break was necessary.
Sure your static analysis may detect that the loop goes 2^256 times, which would be "forever," and so it returns the answer, "Does not halt."
Ah, but with the break keyword, it still could. In fact, it might always halt rather quickly.
Rule of thumb is that if static analysis can be wrong, it is Turing-complete.
I want my build system to be independent of the programming language for two reasons:
1) What gavinhoward said: I regularly use several different languages, sometimes in the same project, sometimes not. But in all cases, I want to use the same build system in order to minimize my environmental complexity. Much like I want to use the same IDE regardless of language.
2) I don't want to risk version dependencies. I want to be able to use the same build system even when I'm compiling something using an old version of a compiler, and I don't want to be forced into a build system upgrade just because I want to upgrade one of my compilers.
For me, the separation of the two roles is very important. If they were tied together, it would mean that I'd have to add complexity elsewhere to cover my use cases. Languages that include their own package managers (like Rust, Go, etc.) give me headaches.
It’s not obvious to me that one can solve the problem of making N languages co-operate by adding another. Why not just use the most powerful language to orchestrate everything?
> I don't want to risk version dependencies. I want to be able to use the same build system even when I'm compiling something using an old version of a compiler, and I don't want to be forced into a build system upgrade just because I want to upgrade one of my compilers.
That begs the question of whether compiler upgrades should force build system upgrades. In a mature programming language, this should never be the case, although practically speaking it’s definitely possible (e.g. ASDF 3.3, released earlier this year but built on a language which has been static for thirty years, adds compatibility for Allegro Common Lisp!).
Incidentally, ASDF is pretty cool: https://asdf.common-lisp.dev/
I usually do everything in a single language, but sometimes that's not the best solution. It's not a matter of how powerful the language is in a general sense, but how good it is at a particular kind of task.
But even ignoring my projects that involve multiple languages simultaneously, I still regularly use several languages in total, even if only one per project. So having a build system that's independent of the language is still highly desirable.
> That begs the question of whether compiler upgrades should force build system upgrades.
My answer to that is "no, never", of course. That leads to increased complexity for me, in exchange for no gain. The build system should have no special knowledge or expectation of what language or language tools are being used.
Ah, that’s a difference between us: I prefer to always use one primary language, and all others are secondary languages selected per-project. So it makes sense for me to just use a system built in my primary choice.
> My answer to that is "no, never", of course.
Agreed, too! In principle upgrading the compiler should have no effect whatsoever on correct, portable programs. OTOH, there sure do seem to be an awful lot of incorrect and unportable programs out there!
Rails
Bundler
Vagrant
Chef
Opsworks configuration
JavaScript bundling
Generating PDFs
Probably other things
The system was basically frozen in time because too many cross-dependencies couldn't update in sync to something newer (the opsworks cookbook in chef were the worst offenders iirc).It was so bad when I started that I couldn't even get the system up and running locally. I ended up tossing hundreds of lines of random ruby code and with a few lines of docker commands had it up and running.
Since then I've preferred keeping things nicely not dependent on each other whenever possible.
It shouldn’t. The only issue is that most folks aren’t using powerful enough languages. As you note, using C++ as a scripting language would be pretty crazy. So don’t do that!
Please no. I don't want to handle filename and other string manipulation in C.
What about packages for *nix based OS's? dnf, pacman, apt, etc. essential have there own "meta" build systems to create packages for there particular package flavor weather that be rpm, deb, tar.gz etc.
C/C++ are big time numerous and serious real. And they need all the help they can get to do builds more like golang
If turing-completeness is necessary, why should there even exist a build system independent from a programming language now?
Isn't the obvious solution to script your language's default build system in precisely the language you're trying to build from?
So when you build Rig, build it in C++ and make C++ your scripting ... Ok, that left a bad taste in my mouth.
Seriously, though. Build systems as the author describes them are going to compete against the project managers that come with languages (Cargo, Go). The only real use case for Rig would be legacy languages that don't come with language-central packaging repositories, i.e., Fortran, C++, C.
While I applaud the author for the attempt to improve this ecosystem, I really hope it will become less relevant some day.