this exists for k8s[0]. there have been other users based on the same library[1], I heard reddit did something similar internally
[0] - https://github.com/cruise-automation/isopod [1] - https://github.com/stripe/skycfg
Programs take inputs and can output arbitrary data such as resources. However they can do so with type safety, and everything else a programming ecosystem can achieve.
For asset distribution it uses wasm, and that's it!
If you want to check it out its here: github: (https://github.com/davidmdm/yoke) docs: (https://davidmdm.github.io/yoke-website)
I like that you said: > I think proper programming language support is the way to go.
I think we need to stop writing new ways of generating yaml since we already have the perfect way of doing so. Typed languages!
This means general purpose languages do not qualify, and more generally, no general recursion
To protect against somebody writing a non-terminating program?
General programming languages come with a lot of general purpose benefits from their ecosystems like package managers npm, cargo, go modules, etc.
They have test runners, and control flow.
Lots of them already have type definitions for kubernetes and if you are working in Go you have access to almost the entire kubernetes ecosystem.
Maybe we are throwing the baby out with the bath water when we disqualify general purpose languages?
You speak like turning incomplete languages cannot have the control structures, tooling, and ecosystems we enjoy elsewhere, which would be the wrong assessment. I recommend you take a look at CUE to see how this can be true
The OpenAPI specs are probably better than the Go language types for k8s. They have more of the validation information and you can get at the CRDs / versions actually running in the cluster.
However there’s no reason we should rule out general purpose languages.
We have a lot of configuration based IaC and configuration tooling a la jsonnette and cue and yet these are riddled with their own problems and DX issues.
Anyways we don’t need to see eye to to eye on this but I respect your position.
We’ve learned the hard way general purpose languages are poor for configuration at scale. I know first hand having worked on some of the larger prod infrastructures out there.
At scale, the best SRE’s out there still have trouble reasoning about the system and end up pushing bad config that takes down prod.
Languages like CUE really are different and better. CUE in particular hits the right balance for configuration of millions of lines of k8s yaml.
CUE was created by the same person who wrote the Borg precursor and also worked on BCL & GCL.
However there's something about a full blown general purpose language that is so much more flexible.
I don't think that the fact that people can and do write bad programs disqualifies general purpose languages from being great tools to build packages.
I am sure there is just as equally bad CUE, Jsonette, PKL, etc out there.
Other than CDK8s I don't know of other tools that have tried in this space to use general purpose languages to define their packages, and I think CDk8s uses are generally happy. Much more so than helm users at least.
I am not sure I can agree with this statement > We’ve learned the hard way general purpose languages are poor for configuration at scale
I think we've just assumed this, or seen a pulumi project we didn't like working in.
I believe and hope there will be plenty of room to experiment and innovate in this space!
CUE is also pragmatic in that it has integrations with yaml, json, jsonschema, openapi, protobuf
See also: KCL, which is very similar and might _actually_ be "the winner". Time will tell.
Having used CUE, mainly outside of Kubernetes, I cannot see myself switching to KCL. I really like having a configuration language that isn't so tied to a specific system and which I can use with the Go SDK
Personally, I would prefer a SQLite database. Ok I'll show myself out.
If your config is the source of truth of what your infra should be, then you can use source control tools to roll back to a known good state, or to binary search for when a problem was introduced.
If you use programmatic config, then you can't find out the intended state of your system without executing a program. You can't grep through program executions in a meaningful way, especially at scale. So you can't do even simple things like search for a string.
Guaranteeing termination is helpful, but it doesn't solve the main problem that programmatic config puts a large complexity barrier between you and the ability to understand your infrastructure.
Tools like Helm give up a fair amount of this declarative benefit. And IMO that's one of the reasons why it's almost always a worse experience to use a helm chart than to just render the chart once and for all and forget Helm ever existed.
This is how Pants (build system) works. You have declarative Starlark which supports basic programming semantics and this generates a state the engine reads and tries to produce.
I've been meaning to dive into jsonnet for a while but it'd be good to have a higher level representation that didn't rely on sophisticated templating and substitution engines like current k8s.
Compare k8s to Terraform where you have modules, composability, variables. These can be achieved in k8s but you need to layer more tooling on (kustomize, helm, etc). There could be a richer config system than "shove it in YAML"
Things like explicit ordering and dependencies are hard to represent in pure yaml since they're ",just text fields" without additional tools
Languages can be declarative or imperative. For instance, Pulumi and CDK are declarative.
> If you use programmatic config, then you can't find out the intended state of your system without executing a program. You can't grep through program executions in a meaningful way, especially at scale. So you can't do even simple things like search for a string.
I don’t understand — nothing stops a language from having an intermediate compilation step that shows the intended state and is searchable. Beyond that, programmatic config means you can add in hooks to make plans or runs more interrogatable.
> Guaranteeing termination is helpful, but it doesn't solve the main problem that programmatic config puts a large complexity barrier between you and the ability to understand your infrastructure.
It seems like this is untrue — having seen templated IaC that is hundreds of thousands of lines and cdk that defers that complexity to an abstraction that I have to understand once, I’d always take the latter.
Agreed that helm use is a giant mistake and liability.
For example: You have a problem that is suitable for some message queue -> Apache Kafka. Now you have 7 new problems and the complexity warrants perhaps 3 other services, and on, and on.
Nowadays you can also start with a light weight MQ like Rabbit MQ and decouple your service just into a hand full components. This will set you up for scalability without introducing massive overheads.
In end it is also always a knowledge game. How experienced are you or how much time are you willing to invest into learning and understanding a technology.
Here's how I use it: https://github.com/shepherdjerred/homelab/tree/main/cdk8s
I think proper programming language support is the way to go.
Ideally a static type system that isn't turing complete and guaranteed to terminate. So something like starlark with types.