Yes, the windows registry, the peak of our craft. No need to innovate, this one’s solved, wrap it up and move on

Try taking more time to consider the problems you don’t have that others do, instead of writing anything off that doesn’t make sense to you (and simultaneously gatekeeping an entire industry)

Creating complex layers upon layers just to "solve" a very simple problem is not innovation. It's shittification.

If your configurations are simple, you don’t need this. If they are not, you might.

Because you have not run into the problems this has addressed in your career, does not mean that you know better than Apple how to solve them. In fact, it means like you are uniquely unqualified to solve them. Acting like you are in a severely condescending dismissal of the engineers who worked on this makes for boring conversation.

Well, Apple is the gold standard in creating new proprietary programming languages for often just evil reasons - typically for vendor lock-in.

They invented the Objective C and Swift, and made it pretty much impossible to use any standard language to target their platforms in a cross-platform way. I can access the Windows API with any language I want. I can access the Linux Kernel API with any language I want. I can access the BSD API with any language I want. I even can use any language I want on the C64 to call native operating system functions.

So, yes, I am taking the liberty to believe I know better than Apple. TBH I don't regard this conclusion as rocket science, so no ego involved here.

There's no "walled garden" here. We want Pkl to be useful for many developers, for a variety of purposes. Hopefully that's clear enough with Java, Kotlin, and Go being amongst the various supported language bindings. We also already support Linux, and plan on supporting Windows.

It's also open sourced under the Apache 2.0 license, which grants you the right to distribute and modify it.

Cheers

I appreciate you having good intentions. I don't mean to be disrespectful.

This however does not change my position that we have more than enough scripting languages focused on string operations, and don't need another language. And especially not a bloated one that itself depends on huge frameworks. If the same task can be solved using a 76KB standard Unix tool, creating a custom language with a huge bloated runtime simply can not be justified. It's bringing a sledgehammer for a task where a swiss pocket knife would be appropriate.

And my position also stands that it would make far more sense to finally agree on a config file standard format, with native parsers existing in every major programming language, so people stop re-inventing the wheel again and again and again, with the wheel becoming heavier and bulkier and less round every time.

> And my position also stands that it would make far more sense to finally agree on a config file standard format

It would also make far more sense for humans to agree on most things and work towards a common goal that benefitted us all, but that’s just as much of a pipe dream.

Proposing something that will never happen is not a practical solution.

https://xkcd.com/927/

communism immediately comes to mind. beautiful idea, but also the idea with the largest body count in the human history.

> They invented the Objective C and Swift

Apple had nothing to do with the invention of Objective-C. It had been out for 12 years when Apple first started using it.

Apple did not invented Objective-C. They took it over from NeXT. Which also didn't invent it.

Also, since Objective-C is basically just C, you could actually target their platform with plain C if you know what you're doing. Some of the underlying frameworks (e.g. Core Foundation) are actually C APIs.

Maybe give us an example?

It's an interesting question - just how complex are our biggest configuration problems today?

Kubernetes config is a decent example. I had ChatGPT generate a representative silly example -- the content doesn't matter so much as the structure:

https://gist.github.com/cstrahan/528b00cd5c3a22e3d8f057bb1a7...

Now consider 100s (if not 1000s) of such files.

I haven't given Pkl an in depth look yet, but I can say that the Industry Standard™ of "simple YAML" + string substitution (with delicate, error prone indentation -- since YAML is indentation sensitive) is easily beat by any of:

- https://jsonnet.org/

- https://nickel-lang.org/

- https://nixos.org/manual/nix/stable/language/index.html

- https://dhall-lang.org/

- (insert many more here, probably including Pkl)

You're absolutely right, k8s is a pig.

Especially since k8s allows using arbitrary labels for plugins, effectively creating „stringly typed“ programming.

Nobody is stopped from compiling code, converting it to base64 and storing it into a label for later execution.

Arbitrary parameters like this are the opposite of a unifying abstraction.

That ingress-behavior wasn‘t defined but pluggable to suit existing load balancers also broke the abstraction.

I do feel there's a simple abstraction under there somewhere - k8s config is just a big tree, after all. And adding pkl's loops on top of that would probably reduce a lot of duplication.

Maybe not always, but surely in most cases of too complex a config, it is a case of ad-hoc grown config, representing what one wants to actually configure badly, and/or underlying abstractions of the thing one wants to configure matching badly what one wants to do. In most cases it would be good to take a step back, or multiple ones at that, and really ask oneself: "What is it, that I actually want to configure here?" and think about why it cannot be a simpler config. What abstractions would actually make expressing that config easy.

Often one will get to a very simple config format in the end. Of course, when one has to deal with very complex formats created by others, already widespread in use, on cannot easily change the format. Maybe that is the reason we get these meta config tools.

Sure. There are undoubtedly a lot of config formats that are overly complex.

But sometimes the complexity is irreducible. Kubernetes is one such case. The model is very well thought out, and just about as simple as it could get without removing functionality. It has sensible defaults, built-in versioning, well-defined schema etc. But if you want to describe a complete installation of a distributed system with many heterogenous processes, spread across many hosts, communicating in specific ways, with specific permissions, persistence, isolation, automatic scaling, resilience, etc, there are a lot of details. I've worked with systems that have thousands of lines of configuration, and honestly that's not extraordinary. Many people on this site will rightly scoff and say, "psshh, that's nothing."

Configuration languages are a really important area of research in the tech industry right now, and every time someone posts one on here, there are a huge number of dismissive comments. Fine. Not everyone has this problem, but it's a real problem, and solving it represents a real advance in the state of the art.

Sounds like the case where a picture is worth a thousand words

A team of 6 responsible for overseeing the deployment of all software in a department of 1000 to any number of deployment environments with tight budgets on durability, delivery time, and the autonomy of the teams that are writing the configuration may reach for this, or something similar, over the Windows Registry

I just run a count, on the infrastructure configuration for a relatively small, but already public company. 13k lines of YAML alone. Some of it is generated (from "YAML with macros") by AWS CDK parody I invented before CDK became a thing (and being an old school Unix pervert I did it with awk and m4). There's also 1400 lines of written or generated JSON that still needs to be managed. And that's a successful, but single-product company. I also worked for a company which had a single product but ran it for their clients heavily (very heavily) customized, you can easily 10x the numbers above and you still have to do that with a single team and go back to your family every night and not riding an ambulance to the nearest mental facility. YAML is like the object code, it's not a configuration language. (and that's why I prefer JSON, it's less error prone and easier to generate). Writing YAML manually is like programming in machine code back in the 80s. Possible, but why?

Misappropriating a trendy word to use as a descriptor for anything you don’t like. This is how someone speaks immediately before taking a look in the mirror and realising that they sound like their parents. You are not the be-all end-all and your drive-by analysis of a technology is going to be biased by its suitability for what you do day-to-day. No amount of experience justifies your attitude.

I am sorry if the wording of my opinion has offended you.

My parents most likely would have complained that they'd need 58 floppy disks to run this programming language. :)

It's not that it's offensive, it's just not conducive to constructive dialog. "Enshittification" has rapidly come to mean nothing more than "changing in ways I don't like", just crasser.

Ok, I'll avoid that term in the future, then.

It is no different when you are trying to solve a complex problem and complex layers upon layers obfuscate and confound you when you are trying to figure out why some component ends up with configuration values that are nowhere to be found in your input files.

I am taking your comment as a joke because the Windows Registry is a joke.

I think they are being sarcastic about windows registry.

Pointing out that a proposed solution to a problem creates more problems is not the same as saying the original problem doesn't exist or isn't worth solving. Nor does it imply that some random other solution is better.

How about .ini/.cfg files? Simple and portable, allowing comments, unlike JSON.

Not scalable though, like JSON with nested objects.

Scalable config files? I'm confused. What needs to scale?

That strikes me as a very uncurious perspective. There’s absolutely nothing that could be improved about configuration files? An 87MB binary feels like a very arbitrary measure of worthiness.

ONE actual standard that everyone uses would be fine. For non-tree (nested) configurations, an ini file would do. It's a standard that worked well in 1990, and still does today.

For nested data, it doesn't matter to me if it's JSON, YAML, TOML or whatever. Just agree on ONE format.

TOML also is a good example of "creating problems instead of solutions": They deliberately (!) broke compatibility to the INI format due to "I can't stand unquoted strings". Yeah, emotional feelings about CONFIG FILE FORMATS.

And again: If your config files are so complex, how about just creating a TUI/GUI so the user can configure your program in an accessible fashion?

Anyway, yes, there is something that can be improved: Create a standard instead of adding another layer of complexity for something that should be very simple in the first place.

Yes, binary size may feel arbitrary, but it often gives a hint about the hammer's size.

> TOML also is a good example of "creating problems instead of solutions": They deliberately (!) broke compatibility to the INI format due to "I can't stand unquoted strings". Yeah, emotional feelings about CONFIG FILE FORMATS.

INI is hard to parse because without quotes the parser would not know whether it’s a “True” as a string or True as Boolean value. Formal parser that can be included into a program as a library and would reliably generate internal config representation (e.g., read config file into an object in application memory that will be used to modify the behaviour) is a good thing and TOML helps with that while INI does not.

Yes, agreed. On the other hand, there are quite a few people using Javascript, a language where this distinction also isn't really done in a clean way.

If you would de-serialize an ini-file into Objects in memory you would typically have meta-data about your object (RTTI/Reflection for compiled languages, given for scripting languages).

But yes, INI isn't without flaws. Parsers typically will need to accept everything from "true", "True", True, true, 1, "1" for boolean fields.

Schemas are often better than syntax typing. In other words the program decides what the type is, not the file.

If the schema allows union types, I don't want to deal with unquoted strings.

You don’t deal with syntax period. Union types are not desirable with configuration with the exception of |null perhaps, which should receive a default.

It’s pretty clear that you are putting “things that I [think I] understand above things that are better. Have you ever written an ini parser? If you have, I doubt it was good, because none of them are, because ini is a flawed format.

No, I am a Pascal guy. Turbo Pascal had an INI parser, Delphi has one, FreePascal has one, and they all work flawlessly.

From what I see, 20+ years proven parsers are available for pretty much any language there is.

And yes, the format is a bit flawed. But it gets the job done.

The Python parser has worked well for decades.

> They deliberately (!) broke compatibility to the INI format

There is no "ini format" though? INI is a weak set of conventions that different programs parse in a completely different way, it's the opposite of standardization.

> ONE actual standard that everyone uses would be fine

Sounds like a social problem. How do you get _everyone_ to agree on anything?

I mean, there is something to be said against using the equivalent of 87 novels worth of text to produce a note’s worth of configuration.

25 years ago admin would be responsible for one server, that they would setup and then never touch until (and sometimes after) someone hacked it.

Now people are responsible for thousands of containers, and GUI simply does not scale. And the configuration of those containers is domain-specific and/or varies enough so that you cannot write a single config file and copy it over. This is why configuration management systems first came with template engines and now require a separate tool to render the configuration.

Nobody would develop anything like that if it was not needed.

Oh yes, people DO develop stuff that is not needed. Back in the 90ies pretty much every nerd sooner or later had to write his own programming language (myself included). The difference between then and today: We did throw it away afterwards. These days every week someone spews out a new "programming language", claiming that the re-invented wheel is so much more round than the others.

If fully agree that you might need to automate editing and deploying config files. But you don't need a new programming language for that. Just use one of the thousands of existing languages and tools.

You can for example use sed. A standard since 1973. Available everywhere. 76KB in size. Extremely fast.

Or use AWK. Exists since 1977.

Or you can use Perl. Natively able to read/write ini files, and in general very good at string processing. 6MB in size (or 250KB for embedded versions).

Or use Javascript, PHP, shell scripting, whatever.

But no, there is absolutely no need to invent a new programming language for a task where there are very established tools for.

I stopped using sed because it behaved differently on my coworker's machine than it did on mine.

Same, actually a lot of the listed things are too old-school for me.

You are proposing to use general propose language vs domain-specific ones. I think, this discussion has been settled quite some time ago. For example, awk itself is domain-specific, built for processing text (or filed-separated) files. On the other had, m4 is as old as awk. m4 was heavily used even before autoconf (sendmail configuration is probably the best-known and possibly the ugliest application of it), for configuration, so one can argue, the need for macro expansion and rendering of configuration (as well as othe types) files was recognised even back then.

m4 just shows its age, and we’ve learned what is good (being Turing-complete) and what’s less useful (multiple output streams).

m4 could’ve evolved into m5, m6, etc., but nobody did it, instead people developed tools like jinja. One could argue, this is wrong and should’ve been evolutionary, but the point still is, domain-specific language to render configuration existed because there was use for them. Perl indeed could be such, I guess it did not happen for totally unrelated reasons. If Perl worked, Python would not gain popularity.

Pkl is not conceptually a new invention, it’s a new attempt to solve a real problem.

I'm surprised there's no widely-adopted library for general-purpose languages that handles user-friendly object input and outputs static config (perhaps in JSON) for machines to read. Instead, there are several DSLs that act as dynamic config, and even some that generate dynamic config, or something in the middle like k8s YAMLs.

Have I just not thought hard enough about this problem yet, and a DSL is really needed for it? Or should I go write that lib?

There is PowerShell: `ConvertTo-Json`

100%. Containers are essentially language-agnostic software libraries, and the config we write for them is analogous to the wiring we'd normally do in a to wire libraries together. Yet, our current tools aren't adapted to write and distribute the "programs" formed from gluing them together.

(Mostly-shameless plug, this is what we're trying to solve with Kurtosis: https://github.com/kurtosis-tech/kurtosis )

25 years ago, you didn't have thousands of websites and applications supporting billions of users and millions of TPS. In hindsight, the innovation around DevOps is nothing short of a marvel IMO. 87 MB in the context of a $100 16GB stick of RAM is basically cheap in relative terms.

> manually crate a ... config file.

Wouldn't you want to self host the config?

25 years ago, the only way to get two pieces of software to talk to each other was to get their authors to talk to each other.

I'm quite happy that remix culture has finally made it to software tooling. The extra layer of abstraction is a small price to pay for the power it brings.

(I don't know Pkl, but it seems like it scratches part of the same itch that you might use nix for).

> So what we are missing now is a 500GB framework that can write the config file for the programming language that is writing a config file for the actual program I wish to use.

That exists since 1960. It's called LISP. The e.g. https://guix.gnu.org/ uses with great success, the Guile Scheme dialect of LISP, to be precise. And FYI the "framework" is:

  $ ls --human-readable --size $(readlink $(which guile))
  16K /gnu/store/1gd9nsy4cps8fnrd1avkc9l01l7ywiai-guile-3.0.9/bin/guile

Yes, only sixteen kilos, not gigas.

Guix/Nix can use dynamic linking without risk since they know the exact dependency closure. It's not fair to compare a statically linked executable to a dynamic one.

Right, so its still just about 9 MB:

    guile=$(readlink -f $(which guile))
    sizes=$({ echo $guile; ldd $guile|grep /|sed 's+^[^/]*\(/[^ ]*\).*+\1+'; }|xargs -n 1 readlink -f|xargs du -ab|cut -f 1)
    sumsize=0
    for size in $sizes; do
        sumsize=$((sumsize+size));
    done
    echo $sumsize

Gives 9041632 here.

Thanks for the snippet. Now I have a recipe how to calculate the size of a binary file, plus the shared objects, i.e. the so-file(s), it requires directly.

Now I just need to extend your snippet so that it works recursively. A shared object can require other shared objects.

The general shift to infrastructure-as-code is tied to a number of new trends and requirements:

1) Code review (config code can be reviewed by humans)

2) Automated pre-submit checks (config code can passed through automated pre-submit checks - such as preventing huge changes, or giving you a nice diff to look at

3) Auditability / history-tracking (you can look at the history of a config file to see who, when, why changes were made - this may even be necessary for compliance reasons)

4) De-duplication (you can extract common components into templates/functions - this supports variation across envs/regions/customers and repetition across tasks/machines/DBs)

All of these features help build large-scale systems in modern corporate environments.

40 years ago I wrote Assembly that ran directly on the CPU. You could modify the registers directly. Then these crazy people came along and made a compiled language? And it outputs... Assembly?

I'm sorry, but a huge chunk of todays "developers" really are in the business of creating problems.

except when you moved to C there was no corresponding increase in fixed time which you must spend on tool setup just to program

Yeah, it’s really great that compiling C is a straightforward endeavor that doesn’t require any tools, and definitely doesn’t depend on executables that generate or parse configuration files. I’m confident that no C developers have ever spent time on tool setup just to program.

gcc main.c

Of course tinkerers made some tools you can tinker with as time went on, but you don’t have to let those interfere with velocity, just like I will not be using configuration languages.

Do you mind linking me to some large C projects that don’t use make/autotools/etc?

stb libraries

So… novelty single-file libraries? I’m not sure that “it works if you only have one file” truly supports your argument.

I'm convinced some developers are copying the lawyers old scam of charging by the word why else would we have this nonsense

I think there are plenty of people who have the same uncomfortable feeling.

I used to see this sort of thing a lot at Apple and I used to classify the people concerned as those that work AT Apple rather than people that work FOR Apple.