News for hackers. Stuff that matters.
Re:Validating Kubernetes YAML for best practice and policies by default-kramer on Sunday August 09, 2020 @10:42AM (#24099921)
> Config-lint is a promising framework that lets you write custom checks for Kubernetes YAML manifests using a YAML DSL. But what if you want to express more complex logic and checks? Isn't YAML too limiting for that? What if you could express those checks with a real programming language?

Having recently worked a little bit with YAML for Kubernetes and HCL for Terraform, I really wish they had both just used "a real programming language" right from the start. I'll choose Racket because I know it best, but there are probably many languages that would work well. You could expose very nearly the same configuration language, but backed by a real programming language. I bet this would make some of the tools the author lists at the end (eg copper, config-lint) much easier to write, or perhaps not necessary at all.

And the author didn't mention Helm, but I will. The part of Helm I saw seemed to be a lot of work just to add "functions with parameters" to Kubernetes YAML, something we could have had for free using "a real programming language" from the start: https://helm.sh/docs/chart_template_guide/functions_and_pipe...

Why are so few configuration languages not backed by a real language?

Re:Validating Kubernetes YAML for best practice and policies by Znafon on Sunday August 09, 2020 @10:58AM (#24100015)
> Why are so few configuration languages not backed by a real language?

In many cases, not having a full featured language is helpful as you have some additional guarantees that comes with a non Turing complete language like guaranteed completion.

In some cases though, you do need a full pledged programming language. For those cases, HashiCorp recently announced CDK support for Terraform: https://www.hashicorp.com/blog/cdk-for-terraform-enabling-py...

Re:Validating Kubernetes YAML for best practice and policies by zelphirkalt on Sunday August 09, 2020 @11:32AM (#24100228)
You can bring that argument, but only until you decide to use YAML, instead of something declarative and simple like JSON.

Once you switch to something as powerful as YAML, you might as well reach for a real programming language.

Re:Validating Kubernetes YAML for best practice and policies by tyfon on Sunday August 09, 2020 @11:51AM (#24100369)
Also, YAML is kind of a hack.

Living in Norway I've had quite a few of these situations where no is parsed to False [1].

[1] https://hitchdev.com/strictyaml/why/implicit-typing-removed/

Re:Validating Kubernetes YAML for best practice and policies by ithkuil on Sunday August 09, 2020 @01:58PM (#24101477)
It has been fixed since yaml 1.2

That said, yeah :facepalm:

Re:Validating Kubernetes YAML for best practice and policies by ithkuil on Sunday August 09, 2020 @01:57PM (#24101464)
> powerful as yaml

Can you make an example of the power of yaml that makes it more like a programming language (in particular vs JSON)?

Re:Validating Kubernetes YAML for best practice and policies by riffraff on Sunday August 09, 2020 @02:04PM (#24101537)
not OP but: tags (custom data types), more builtin data types (dates etc), more syntax variations (heredocuments, multiple ways to write a string,a boolean, etc), comments, references+aliases.

The language is vast, and most people use it without knowing it.

I love YAML, but I wish there was a "strict&sane yaml subset".

Re:Validating Kubernetes YAML for best practice and policies by ithkuil on Sunday August 09, 2020 @03:37PM (#24102230)
All those are good examples of yaml complexity but none of that is anywhere near what even a simple programming language can unleash (variables, loops, functions, recursion etc)
Re:Validating Kubernetes YAML for best practice and policies by Znafon on Sunday August 09, 2020 @06:53PM (#24103641)
You can do recursion in YAML, and it ends badly: https://en.wikipedia.org/wiki/Billion_laughs_attack#Variatio...
Re:Validating Kubernetes YAML for best practice and policies by ithkuil on Monday August 10, 2020 @03:59AM (#24106706)
Recursion without termination doesn't produce a useful computation
Re:Validating Kubernetes YAML for best practice and policies by default-kramer on Sunday August 09, 2020 @11:08AM (#24100059)
Cool, I hadn't seen CDK for Terraform before. But doesn't this support my point? If they had just used Typescript from the start, they wouldn't have to add Typescript support later.

I suppose guaranteed completion matters if you are running untrusted code, but wouldn't sandboxing solve that? Are there any other guarantees that sandboxing wouldn't solve?

Re:Validating Kubernetes YAML for best practice and policies by Znafon on Sunday August 09, 2020 @11:19AM (#24100122)
> I suppose guaranteed completion matters if you are running untrusted code, but wouldn't sandboxing solve that? Are there any other guarantees that sandboxing wouldn't solve?

There is more benefits than just that, by restricting the possibilities you know there won't be unbounded loops, analysis and code review is easier (and infrastructure teams are often seriously lagging in this regard), it can be easier to maintain, update and test.

In some cases, you can have a project where this is seriously limiting though because you have some very complex and specific thing you need to express. For this you can use CDK. I would say both approach are complementary, not exclusive.

In my experience I would say nearly all infrastructure projects can be expressed as Terraform rather easily, but YMMV.

Re:Validating Kubernetes YAML for best practice and policies by rectang on Sunday August 09, 2020 @01:46PM (#24101356)
Can a config mechanism where shell commands are embedded as strings still be considered as offering these additional guarantees?

Perhaps certain things are easier: it's easier to parse, to isolate the code portions, and to read data portions. But then validating what's inside those code portions is a challenge.

It seems like there's an opportunity for a programming language purpose built for the task of configuration. Its chief feature would be the ability to provide a lot of information when read in "inert data" mode, yet also provide full programming language power when read in "run" mode.

ETA: Perhaps embed Python inside something similar to YAML, and support active code via a "lambda" type. Use indentation for delimiting.

   project: "Foo"
   version: "1.0.1"
   email: "[email protected]"
   load:
     lambda file_path:
       pass
Re:Validating Kubernetes YAML for best practice and policies by pjmlp on Sunday August 09, 2020 @11:16AM (#24100099)
Because when you do that, things always go astray as not everyone is that disciplined in what they put on those files.

Tcl, Perl, Python based config files experience in production, means I only touch Gradle when Google forces me to do so on Android.

Re:Validating Kubernetes YAML for best practice and policies by lazyant on Sunday August 09, 2020 @01:36PM (#24101244)
> I really wish they had both just used "a real programming language" right from the start

There are currently several options, like https://github.com/stripe/skycfg

> Why are so few configuration languages not backed by a real language?

There are tons of tools that work with YAML/JSON but I imagine you mean more like a first-class citizen programming language specific or good at writing static configuration. The other side of the coin is that configuration in many cases have a different audience than programmers (for ex, end users or operations team) and it's a good option to have static configuration key=value than any human can (more or less) read easily without having to run programming code in your head. Plus programming, besides being harder to read and share, introduces bugs. So I suppose the preference between code writing configuration and stand-alone configuration depends on who the consumer is and how complex the configuration is.

Re:Validating Kubernetes YAML for best practice and policies by swiley on Sunday August 09, 2020 @12:10PM (#24100562)
This so much.

Every time you have to use another configuration language you have to learn all the quirks (ex: "" == undef in puppet), Give up all your powerful tools (like a debugger) and learn new abstractions. It's extremely counterproductive and it usually would have been better to have simple data structure that you generate from a program written in a traditional language (which the team can artificially restrict to avoid recursion if they want.)

Also IMO there's no such thing as a "declarative language." These are just languages where almost everything has a side affect of mutating a data structure that you don't have an easy way of inspecting or debugging.

Re:Validating Kubernetes YAML for best practice and policies by nrvn on Monday August 10, 2020 @01:45AM (#24105960)
using a "real" Turing complete language for configuration is overkill.

But there are two promising projects that could finally make a difference (at least one of them).

https://cuelang.org

https://dhall-lang.org

Re:Validating Kubernetes YAML for best practice and policies by harpratap on Sunday August 09, 2020 @11:26AM (#24100182)
Everything mentioned in the article can be simply done by writing a kubernetes validation webhook in the language of your choice. Why would you specifically need the configuration to be a real language?
Re:Validating Kubernetes YAML for best practice and policies by runamok on Sunday August 09, 2020 @02:24PM (#24101700)
That's one of the things I like about chef. You have an extensible DSL and templating but you can also write Ruby if needed.
Re:Validating Kubernetes YAML for best practice and policies by riverdroid on Sunday August 09, 2020 @02:34PM (#24101774)
k-rail does realtime validation and enforcement with policies written in Go: https://github.com/cruise-automation/k-rail
Re:Validating Kubernetes YAML for best practice and policies by shahsyed on Sunday August 09, 2020 @11:00AM (#24100023)
I'm having some arguments with other developers (devs) on whether or not this is important. I'm gonna finally try to implement this for my own pipeline this week, hopefully.

I would much rather have devs double check/validate things locally before they edit changes.

Modifying config files by using the edit text feature in GitHub (GH), doesn't enable you to do that.

& Devs are lazy. I'm lazy. They want things easy. Me too.

So let's make it easy. Modify your CI/CD pipeline to validate YAML configs on any file changes (use GH hooks for example)

Now devs can do whatever they want - if their pre-deployment checks fail, go back and fix it!

Re:Validating Kubernetes YAML for best practice and policies by Znafon on Sunday August 09, 2020 @11:23AM (#24100151)
This is a very sensible approach. One pro of having the checks automated instead of just having the developers check carefully their changes is that onboarding a new developer is easier, you will spend less time on very small and specific details and you won't forget to tell some detail.
Re:Validating Kubernetes YAML for best practice and policies by JamesSwift on Sunday August 09, 2020 @12:02PM (#24100486)
This is a good approach because it focuses on the desired outcome ('no invalid configs get deployed'), and doesn't try to use a proxy ('you have to validate locally') to get there.
Re:Validating Kubernetes YAML for best practice and policies by gk1 on Sunday August 09, 2020 @01:31PM (#24101205)
You're basically describing Sentinel for Terraform (https://www.hashicorp.com/sentinel/) or Datree for Kubernetes (https://www.datree.io). There are also a bunch of tools popping up in this space that focus on catching security issues rather than misconfigurations.
Re:Validating Kubernetes YAML for best practice and policies by jasonlotito on Sunday August 09, 2020 @09:12AM (#24099393)
You are using YAML already. How much do you care about best practices.

Only partially kidding here.

Re:Validating Kubernetes YAML for best practice and policies by EdwardDiego on Sunday August 09, 2020 @10:06AM (#24099718)
You're currently being downvoted, but I agree, YAML is kinda terrible, not sure why anyone thought Python's syntactically relevant whitespace was ideal for a config file.

Classic example:

  - containerPort: 7173
    name: http
I think that's an object in a list? But it's not overly clear.And if I indented any of those lines wrong...
Re:Validating Kubernetes YAML for best practice and policies by pydry on Sunday August 09, 2020 @10:36AM (#24099887)
>not sure why anyone thought Python's syntactically relevant whitespace was ideal for a config file.

Because it reduces syntactic noise and increases informational density.

Most other serialization formats end up putting in indentation anyway to make them readable (JSON, TOML).

Re:Validating Kubernetes YAML for best practice and policies by dmitriid on Sunday August 09, 2020 @10:21AM (#24099814)
Don't forget the plethora of truthy values that put Javascript to shame.

   - country: NO
Of course it's boolean
Re:Validating Kubernetes YAML for best practice and policies by riffraff on Sunday August 09, 2020 @02:16PM (#24101641)
that was changed in YAML 1.2[0], which I think is many years old, but people don't specify a version on config files anyway and I'm not even sure most parsers respect it, so it keeps popping up, which is sad.

[0] https://yaml.org/spec/1.2/spec.html#id2803629

Re:Validating Kubernetes YAML for best practice and policies by dmitriid on Monday August 10, 2020 @02:30AM (#24106221)
That's the problem with bad formats: they live forever. They and any number of parsers for any version between version 0 and now.
Re:Validating Kubernetes YAML for best practice and policies by aliswe on Sunday August 09, 2020 @10:58AM (#24100010)
That's a map as a list element, iirc the terminology. But check out this.

When studying the Yaml spec I discovered that a map property (key: value) can have not only a string as its key, but any value. Even a list. (cue screams)

Re:Validating Kubernetes YAML for best practice and policies by shahsyed on Sunday August 09, 2020 @11:03AM (#24100034)
Here's another reason why YAML is bad:

the fact that I have to read the spec to figure out what's going on.

It should have been fairly simple to do this.

Why do I have to know the difference between ":" and "=" ...?

Re:Validating Kubernetes YAML for best practice and policies by Already__Taken on Sunday August 09, 2020 @11:58AM (#24100440)
Why didn't anyone learn from CSS when we all made pre-processors like stylus, less and sass that added white-space sensitivity. It's not great.

I thought HCL was one of the best attempts at a terse config language. Shortening foo { bar { bazz {} }} to foo bar baz{} cleans up plenty.

Re:Validating Kubernetes YAML for best practice and policies by corty on Sunday August 09, 2020 @10:17AM (#24099785)
also, keywords of the underlying software mechanisms are decoupled, so uncheckable for plain YAML tooling. Whereas with XML you can at least infer a lot about the desired structure and keywords from the schema. Deeper checks are only possible with "real" programming languages, preferrably statically typed ones. I'm wondering when that wisdom trickles down to configuration languages.
Re:Validating Kubernetes YAML for best practice and policies by corty on Sunday August 09, 2020 @10:07AM (#24099727)
YAML is unusable without a sufficiently capable linter. Syntax is very fragile and completely decoupled from function.
Re:Validating Kubernetes YAML for best practice and policies by shahsyed on Sunday August 09, 2020 @11:01AM (#24100028)
You're partially kidding but you're 100% right.

Someone in another thread mentioned using some LISP or LISP-like language to solve this problem.

Would something like TOML make things better here?

What would have been the solution?

Re:Validating Kubernetes YAML for best practice and policies by rgoulter on Sunday August 09, 2020 @12:53PM (#24100903)
Dhall is an example of a configuration language. Its programs must terminate. It aims for safety, claiming that it can support safe evaluation of untrusted code. https://dhall-lang.org/

TBH, I have no experience with it. But, it sounds like if you need a configuration language with programmatic features, it would be more suited to the job than a general purpose programming language.

Re:Validating Kubernetes YAML for best practice and policies by pjmlp on Sunday August 09, 2020 @11:18AM (#24100107)
XML, but I guess I won't get many friends for asserting that.
Re:Validating Kubernetes YAML for best practice and policies by juped on Sunday August 09, 2020 @01:48PM (#24101374)
I really regret disliking XML all those years ago, now that I've seen the alternative.
Re:Validating Kubernetes YAML for best practice and policies by secondcoming on Sunday August 09, 2020 @11:38AM (#24100277)
Nope. Json has schema validation too, and every language has a json parser
Re:Validating Kubernetes YAML for best practice and policies by pjmlp on Sunday August 09, 2020 @11:44AM (#24100320)
JSON doesn't do comments.
Re:Validating Kubernetes YAML for best practice and policies by secondcoming on Sunday August 09, 2020 @12:16PM (#24100605)
Not officially, but it's fakeable
Re:Validating Kubernetes YAML for best practice and policies by pjmlp on Sunday August 09, 2020 @12:28PM (#24100694)
Which for me is a design smell, why bother with workarounds when XML already offers me all the best tooling out of the box.
Re:Validating Kubernetes YAML for best practice and policies by secondcoming on Sunday August 09, 2020 @09:23PM (#24104554)
For me, xml is just too much, and when working with C or C++ (not t sure about rust) it's just a pain. Can your json schema file not serve as the documentation?
Re:Validating Kubernetes YAML for best practice and policies by pjmlp on Monday August 10, 2020 @01:42AM (#24105938)
Why should it be a pain? Parsing XML is no different than parsing JSON, there are plenty of robust parsers available since around 2000.

Mature libraries with plenty of experience deploying into production.

Plus for small enough stuff, like configuration files, one can just plug XPATH as well, making the search for items even easier.

Re:Validating Kubernetes YAML for best practice and policies by jayd16 on Sunday August 09, 2020 @01:08PM (#24101027)
If something like k8s implemented comments as specific comment fields that would actually be pretty useful. The fields could be parsed and show in GUIs.
Re:Validating Kubernetes YAML for best practice and policies by secondcoming on Sunday August 09, 2020 @09:12PM (#24104487)
That is the workaround, just add it as "comment":"comment text". Having one comment for each field in an object would get unwieldy though. Anther place for them is possibly in the json schema file.
Re:Validating Kubernetes YAML for best practice and policies by jiangplus on Sunday August 09, 2020 @12:41PM (#24100798)
Then use HTML and extend the functionality with JSX
Re:Validating Kubernetes YAML for best practice and policies by pjmlp on Sunday August 09, 2020 @01:13PM (#24101064)
Useless work when XML already has all the tooling available and battle tested in production.
Re:Validating Kubernetes YAML for best practice and policies by pbiggar on Sunday August 09, 2020 @02:31PM (#24101751)
I recently had a bug that wouldn't have happened if I'd had these in place: https://dev.to/darklang/a-fun-bug-55cl. I added similar checks (and kube-score and polaris seem like good tools - I might try adding them).
Re:Validating Kubernetes YAML for best practice and policies by shimont on Sunday August 09, 2020 @09:22AM (#24099451)
I think that this is a great approach to test out the files. Mistakes in those files can cause a production outage. I like doing those tests once a PR is open and before it is merged into master and executed on the production cluster. (Disclaimer i am a co-founder of datree.io)
Re:Validating Kubernetes YAML for best practice and policies by pjmlp on Sunday August 09, 2020 @11:13AM (#24100084)
And so the circle of schema validation does yet another turn, now back to those XML config files.
Re:Validating Kubernetes YAML for best practice and policies by juped on Sunday August 09, 2020 @01:47PM (#24101363)
Every time I see a JSON config file I miss those kilobytes of XML.
Re:Validating Kubernetes YAML for best practice and policies by EdwardDiego on Sunday August 09, 2020 @10:08AM (#24099734)
I find Intellij's K8s plugin really helpful for identifying issues within a single K8s YAML file, but it won't find things like a deployment.yaml without a pdb.yaml but it's a good start.