News for hackers. Stuff that matters.
Re:Pkl, a Programming Language for Configuration by avianlyric on Saturday February 03, 2024 @10:00AM (#39240916)
Guess the obvious question is why don’t you want types in your config language? Pushing all the validation to parsing state just makes it hard to write valid config, because you only know if the config is valid when you feed it into your program.

Having the ability to pull all that validation forward, and into your IDE, means you can be told about invalid config as you’re writing it. To me the idea of only wanting to validate config at the last possible moment, is a bit like advocating for only using simple text editors, and replying purely on build failures for feedback. Sure you can do it, but why would subject yourself to that?

Pkl is interesting because it makes it possible to describe not just the types, but also valid configs. Effectively allowing you to ship your applications config parsing and validation into Pkl, so your code can just expect valid config. Then Pkl can expose all that parsing and validation logic into your IDE, so you get pointers as you type. Just like you do in any modern IDE for any modern language.

Re:Pkl, a Programming Language for Configuration by hgs3 on Saturday February 03, 2024 @11:30AM (#39241862)
> Guess the obvious question is why don’t you want types in your config language?

The disadvantage of typed configuration languages is they make assumptions about the format of the data. For you a "date" type might mean ISO 8601, but for me it might mean RFC 3339. Config languages that make assumptions are coupling the language and schema validation together. The alternative is to decouple them: offer a flexible configuration language with a separate schema language. The latter would let you define the schema specifically for your data for validation.

Re:Pkl, a Programming Language for Configuration by avianlyric on Saturday February 03, 2024 @11:43AM (#39242004)
> For you a "date" type might mean ISO 8601, but for me it might mean RFC 3339

A generic date type doesn’t come with any specific string format. ISO 8601 and RFC 3339 are both ways of representing a date as a string. Which has little to do with Date as a type.

There’s also perfectly good solutions to those problems. Use a type alias to create a date type backed by a string, with formatting constraints (such as a regex). Then people can define dates using any string representation they want!

Incidentally this is exactly what Pkl lets you do. You can use Pkl schema to define in detail how you want to validate your data using various primitives like regex. And then separately create a config template that uses those custom data types. As a dev you can choose how tightly bound your config template is to a data validation schema, define all the validation in line, or import an external module to provide you with richly validated data types.

Re:Pkl, a Programming Language for Configuration by hgs3 on Saturday February 03, 2024 @12:16PM (#39242422)
> ISO 8601 and RFC 3339 are both ways of representing a date as a string. Which has little to do with Date as a type.

Tell that to the TOML authors [1].

It's good Pkl has string validation, but what if I don't want a string? What if I want my own literal datetime syntax, like TOML? The grammar for a configuration language could be made flexible enough to accept most anything as a value. In such a design the "string" type itself could be a regex rule defined by a schema.

Keep in mind datetime literals are just an example, the actual number of potential types is unbound.

[1] https://toml.io/en/v1.0.0#offset-date-time

Re:Pkl, a Programming Language for Configuration by samatman on Saturday February 03, 2024 @02:18PM (#39243738)
This choice in TOML is a mixed bag. I would say it's substantially mitigated by the fact that strings must be quoted, so you don't have strings magically turning into datetimes if they look like a date. But it does add substantially complexity to an otherwise simple language.

The near-compatibility of ISO 8601 and RFC 3339 is a rich source of bugs, but that's hardly TOML's fault, and the standard is perfectly clear that the latter is used. TOML, like many configuration and data transport languages, is defined in terms of its syntax, so an abstract Date type doesn't make sense for it.

Providing a datetime format for TOML was probably the right decision, I think there are more people who complain about JSON lacking one than there are who complain about TOML having one.

Re:Pkl, a Programming Language for Configuration by cpeterso on Saturday February 03, 2024 @05:51PM (#39245584)
Seems like someone should create a new datetime standard, documenting the safe intersection of ISO 8601 and RFC 3339. I did find this useful comparison: https://ijmacd.github.io/rfc3339-iso8601/
Re:Pkl, a Programming Language for Configuration by Too on Sunday February 04, 2024 @01:28AM (#39248084)
Even more reason to standardize the format used in configurations and to validate it early, rather than at runtime.
Re:Pkl, a Programming Language for Configuration by usrusr on Saturday February 03, 2024 @11:09AM (#39241653)
One argument I might put on the cons side of AOT type validation for configs is that there will always be some invalid inputs that can't be statically checked (e.g. port number already taken), and not failing simple type errors before runtime helps keeping in mind and view whatever feedback channel exists for the runtime failure. I wouldn't consider that a winning argument, but it's not entirely without merit.
Re:Pkl, a Programming Language for Configuration by avianlyric on Saturday February 03, 2024 @11:36AM (#39241921)
That’s not a reason for giving up on all config validation before runtime. Just because we can’t solve a problem in every possible situation doesn’t mean shouldn’t solve the problem for any situation.
Re:Pkl, a Programming Language for Configuration by srcreigh on Saturday February 03, 2024 @11:21AM (#39241767)
duplicate port number can be checked in the type system if you’re using racket, at least
Re:Pkl, a Programming Language for Configuration by usrusr on Saturday February 03, 2024 @12:47PM (#39242774)
Duplicate port number in the config might be checked ahead of time, but port number already taken by something unrelated in the environment deployed to can't. I'm sure that the scope of pkl isn't intended to setting up clean slate containers and nothing else, ever.
Re:Pkl, a Programming Language for Configuration by youerbt on Saturday February 03, 2024 @10:34AM (#39241262)
My take on this is that there is not obvious reason not to, but it just so happens that typed configuration languages are not rich enough and not integrated enough to be that useful.

Those languages that arrived with the JSON hype train like yaml or toml might be great for dynamic languages, where you can load them to some native object. But in statically typed languages you are gonna declare your types, in code, anyway. So configuration providing types doesn't really do much.

Re:Pkl, a Programming Language for Configuration by avianlyric on Saturday February 03, 2024 @10:43AM (#39241369)
Ah! Well this is the hole that Pkl does a very good job of filling!

Being able to use Pkl code-gen to create language bindings means you can take any arbitrary Pkl schema and turn it into native structures in your language, typed based on the Pkl schema. Then you can let Pkl do all the heavy lifting of parsing and validating a specific config, and turning it into native objects in your language.

So no need for double declaring types. Declare them once in Pkl, generate the equivalent types in your language, then just start using those types like any other primitive. The Pkl bindings will handle the loading glue for you.

Re:Pkl, a Programming Language for Configuration by electroly on Saturday February 03, 2024 @11:00AM (#39241565)
If you replace "Pkl" with "XML", this is all exactly true for XML. Ten years ago we were generating C# classes, typed validators, and automatic parsers from XSD schemas, with automatic IDE integration and IntelliSense completions when editing the XML documents--is this just XSD for the younger JSON generation? I shipped multiple megabytes of complex manually-written XML configuration this way and it was delightful. We never would have pulled it off without XSD.
Re:Pkl, a Programming Language for Configuration by sanderjd on Saturday February 03, 2024 @05:24PM (#39245382)
What you're expressing here is that many of the ideas from that period of xml-centricity were quite good and useful!

But xml itself was not a good language for this, because its legibility is terrible. It's just not a good format for human reading and editing. (But it also isn't a great format for machine interaction either...)

So yeah, I see it as a good thing that this seems to be able to do all that useful stuff you were doing with xml and xsd a decade (and more) ago. But it's (IMO) a much nicer way to do it.

Re:Pkl, a Programming Language for Configuration by youerbt on Saturday February 03, 2024 @10:59AM (#39241555)
To be fair if your config is just a structure with strings then you declare your types only once, too. Minus the codegen, but also minus the editor integration.

I'm not hating on Pkl here, we deserve better in this space, so I'm happy with more developments.

Re:Pkl, a Programming Language for Configuration by jen20 on Saturday February 03, 2024 @10:42AM (#39241357)
The whole _point_ of Pkl is that it is both rich enough and integrated enough though?
Re:Pkl, a Programming Language for Configuration by youerbt on Saturday February 03, 2024 @10:51AM (#39241451)
Oh, I didn't comment on the Pkl, just on the status quo. My bad for not making that clear.

"Enough" is the keyword here, time will tell I guess.

Re:Pkl, a Programming Language for Configuration by riffraff on Saturday February 03, 2024 @02:39PM (#39243973)
Yaml predates toml by ten years or so _and_ has an extensible schema to define types.

Sadly, nobody ever cared about that.

Re:Pkl, a Programming Language for Configuration by jmull on Saturday February 03, 2024 @10:59AM (#39241552)
Yeah, generally, you want to validate as early as practical... catching problems sooner is better than later.

I think the problem might be separation of concerns...

pkl comes in early, and by design is separated from your app and the details thereof. It seems good for validating high-level or external configuration constraints. But suppose you have some constraints based on implementation details of your app. Now you face a choice: eschew pkl and put that validation logic in the app where you lose the benefits of early validation or put it in pkl (if that's even possible) which implicitly makes it dependent on application implementation details. Of course, we devs aren't great at stopping to consider the deep implications of each config param we add, so which one happens in practice in each case probably depends on the dev or dev group and how much they've bought in to pkl... some will pretty much always add a config to pkl because that's what it's there for, while others will ignore pkl whenever they can. I think this is inherent in the ambiguity the choice presents. There's probably a right choice in each case, but devs will only sometimes be that careful about the location of each config validation.

That's my guess anyway, as to why the previous post wants to just put all the validation at the level it's used. If that's your rule the ambiguity is resolved and it works perfectly for config dependent on specific app concerns and pretty well for config that also has high-level or external concerns, since those are less volatile and when they do change, it generally implies app changes in any case.

My gut says pkl is over engineered for the vast majority of cases and people should not reach for it unless they have a specific problem that it will solve for them.

Re:Pkl, a Programming Language for Configuration by lucianbr on Saturday February 03, 2024 @12:00PM (#39242214)
I am convinced the Pkl config will grow in complexity until it has a yaml or json config for the configuration program.
Re:Pkl, a Programming Language for Configuration by garethrowlands on Saturday February 03, 2024 @03:52PM (#39244627)
Not in my experience.
Re:Pkl, a Programming Language for Configuration by seer on Saturday February 03, 2024 @11:36PM (#39247659)
Hmm isn’t pickle designed for exactly this use case? External config module deps you pull for overall config validation.

E.g. so you always write valid k8s manifests.

And then you can extend them with your own additional validation rules for what you think your app needs? I’ve just skimmed the docs but it seems it allows you to be as loose or as precise as possible, plus packaging and publishing those rules for others to use.

Seems kinda awesome.

Re:Pkl, a Programming Language for Configuration by HelloNurse on Monday February 05, 2024 @08:37AM (#39261064)
Fixing the app implementation details so that the configuration stays "clean", or at least forcibly documenting them so that the configuration can be written correctly, is vastly better than allowing the app to make undocumented surprises with its private validation.
Re:Pkl, a Programming Language for Configuration by maximus-decimus on Saturday February 03, 2024 @11:21AM (#39241774)
Because with a real programming language you get an actual IDE, auto complete, a debugger, sane compiler errors instead of a vague helm error "invalid thingy at line 4", you can log a bad config the same way you log stuff for the rest of your program and you can't guarantee your config is valid anyway if your config language can't see what class you're going to feed it to.
Re:Pkl, a Programming Language for Configuration by avianlyric on Saturday February 03, 2024 @11:34AM (#39241898)
That’s a consequence of configuration languages not have proper type systems, and robust was to manipulate data.

Helm isn’t a configuration language, it’s a dressed up string templating system with ability to do kubectl apply.

> you can't guarantee your config is valid anyway if your config language can't see what class you're going to feed it to.

Obviously, but that’s hardly an insurmountable problem. We’ve had code gen and language introspection for decades.

Re:Pkl, a Programming Language for Configuration by Too on Sunday February 04, 2024 @01:34AM (#39248117)
Helm is not a proper configuration language. It just does string replace. Unbelievable that people actually use it.

A config language that does have the type information can give proper errors. Try for example terraform.

Re:Pkl, a Programming Language for Configuration by wtetzner on Saturday February 03, 2024 @03:00PM (#39244167)
> Guess the obvious question is why don’t you want types in your config language?

Where do you store the schema?