News for hackers. Stuff that matters.
Re:Write Gitlab CI Pipelines in Python Code by pydry on Friday April 30, 2021 @05:57AM (#26991546)
This is deeply misguided, I think. Dhall is a neat hack, but it's still a hack.

Any and all attempts to munge YAML or to turn YAML into something Turing complete are signals that a scripting language would have been a better fit for the problem.

The exit to this cycle is:

* to use configuration languages for things that are genuinely configuration now and always

* scripting language APIs for things that are genuinely not configuration (e.g. build systems).

* provide both for domains that are genuinely mixed (a lot of kubernetes probably fits into this category - some things are configuration for some people and dynamic or generated for others).

It sounds simple but it is hard - the correct border between config and code is usually hard to perceive and how your tools end up being used is inherently unpredictable.

Re:Write Gitlab CI Pipelines in Python Code by infogulch on Friday April 30, 2021 @10:23AM (#26993844)
You just described the cycle, not a path to exit it. There is no such thing as "genuinely configuration now and always", nor "genuinely not configuration".

Systems change, and requirements of those systems change. Oh now someone wants to do more things with what used to be 'just configuration': scale it up and to work across many instances by scripting configuration templates with parameters. Look, all of this scripted automation is doing the same things that can be neatly divided into buckets, can we simplify it and turn it into a few configuration options? And so the cycle repeats. The churn is not reducible to labeling it as pointless make-work, its born of changing and growing systems.

Re:Write Gitlab CI Pipelines in Python Code by protomikron on Friday April 30, 2021 @07:25AM (#26992049)
Dhall is not turing-complete. That's the whole point.
Re:Write Gitlab CI Pipelines in Python Code by pydry on Friday April 30, 2021 @08:14AM (#26992425)
That was incidental to dhall. Dhall doesn't and was never intended to help clean up the mess made by having ansible code be Turing complete (for example).

I'm pretty sure the whole point of dhall is to DRY out configuration files and to hack on type guarantees to config files.

I played with it for a bit and those are the only two use cases I could find. Those two use cases usually come encumbered with several others which it does not help with at all, but a scripting language would.

A good, refactored-to-be-DRY config file that is typesafe completely obliviates any need for dhall and if you've got a Turing complete YAML monstrosity it's not going to help much.

It reminds me a bit of XSLT (except that was accidentally Turing complete of course).

Re:Write Gitlab CI Pipelines in Python Code by infogulch on Friday April 30, 2021 @10:31AM (#26993959)
This comment doesn't make any sense to me

> A good, refactored-to-be-DRY config file that is typesafe completely obliviates any need for dhall and if you've got a Turing complete YAML monstrosity it's not going to help much.

Dhall is a "refactored-to-be-DRY config file that is typesafe", and the host of safety-related features ( https://docs.dhall-lang.org/discussions/Safety-guarantees.ht... ) make this practical to implement, even while evaluating untrusted & potentially malicious dhall code. These guarantees are much stronger than what nearly any other config or scripting language provides.

Re:Write Gitlab CI Pipelines in Python Code by pydry on Friday April 30, 2021 @12:34PM (#26995691)
The point is you could all get that with YAML without plonking dhall on top if you had a properly designed schema.

Hence it's a hack to circumvent badly designed / no schema.

Re:Write Gitlab CI Pipelines in Python Code by infogulch on Friday April 30, 2021 @03:35PM (#26998217)
Lets look at a specific example. Take Kubernetes: everything is yaml, with complete schemas, all the way down. From your perspective this is configuration utopia, right? Meanwhile back in reality k8s is the poster child of "yaml hell". From the day it was released, people took one look at it, gave it a giant NOPE and instantly spawned half a dozen templating languages. The most popular of these is helm, which has a terrible, no good, very bad design: full of potential injection attacks from purely textual string substitution, manually specified indentation to embed parameterized blocks, virtually no intermediate validation, no way to validate unused features, etc etc

Compare to dhall which publishes a complete set of dhall-k8s schema mappings which enables you to factor out any design you want down to as few configuration variables as you like, while validating the configuration generators themselves at design time. https://github.com/dhall-lang/dhall-kubernetes#more-modular-...

Re:Write Gitlab CI Pipelines in Python Code by pydry on Friday April 30, 2021 @03:44PM (#26998346)
>Lets look at a specific example. Take Kubernetes: everything is yaml, with complete schemas, all the way down. From your perspective this is configuration utopia, right?

LOL! No, their schemas are horribly designed and half of their YAML should really be APIs.

But it's hella popular.

>Compare to dhall

Which isn't.

Re:Write Gitlab CI Pipelines in Python Code by infogulch on Friday April 30, 2021 @02:10PM (#26997135)
Your argument is akin to "why do you need a high-level language? just write it in assembly, C is just a hack to circumvent badly designed assembly". If a 'properly designed yaml schema' was sufficiently powerful for all these use-cases then why does everyone and their third cousin come up with new configuration languages every week? Clearly they do, thus I must reject the null hypothesis and conclude that plain yaml schemas are not sufficient.

These arguments aren't holding any water.

Re:Write Gitlab CI Pipelines in Python Code by pydry on Friday April 30, 2021 @03:47PM (#26998382)
>If a 'properly designed yaml schema' was sufficiently powerful for all these use-cases then why does everyone and their third cousin come up with new configuration languages every week?

Coz they fucked up/didn't fix their schema designs and people keep trying to work around that?

I thought I was being very explicitly obvious about this.

Programming is a history of bad designs being worked around with awkward hacks. This is nothing new.

Re:Write Gitlab CI Pipelines in Python Code by infogulch on Friday April 30, 2021 @03:52PM (#26998446)
Ah so your argument is that they just aren't doing it right, that a true yaml schema designer wouldn't make this mistake. It's just a mistake.. which keeps being made over and over by everyone that does configuration. Sounds like a no-true-scotsman and "it's the children who are wrong" to me. Can you give any examples of a popular correctly designed yaml schema?
Re:Write Gitlab CI Pipelines in Python Code by pydry on Sunday May 02, 2021 @05:33PM (#27019250)
I'm saying that a schema designers reaction to their users using jinja2 or dhall on their YAML ought to be to fix their schema so it's not necessary.

There are plenty of properly designed YAML schemas where people never feel the need to use something like dhall or jinja2.