News for hackers. Stuff that matters.
Re:Why are we templating YAML? by MrStonedOne on Thursday February 07, 2019 @04:50PM (#19108930)
The real question is why are we using yaml at all?
Re:Why are we templating YAML? by deathanatos on Thursday February 07, 2019 @05:35PM (#19109315)
Easier to read and write than JSON.

JSON requires constant quoting, can't support multiline strings, has no comments, has no/little typing (e.g., no datetime type). It's not good if a human needs to encode data.

For configs, I think TOML beats YAML hands down; I think YAML's spot is at encoding data structures that humans need to read/write.

I do agree that YAML, the spec, is fairly complicated. But YAML, as used in most projects, by most people, is not, and can be picked up fairly quickly. It is easier to visually read as it removes much of the clutter that would exist in the comparable JSON. It isn't typically necessary to know the entirety of the YAML spec to be useful with YAML, and most of the parts you won't know will get introduced by an obvious-looking sigil, which can be used to figure out what you're dealing with.

When I've actually sat with folks struggling with YAML, it's almost always in configuration tools, and it's also always around the templating bits. Ansible, in particular, has a bizarre templating: it happens after YAML parsing, which is not the mental model most people use when approaching it. I've also found that most of the people I've spoken to intertwine YAML and Ansible's templating functions, thinking they're one in the same.

I do not think Ansible makes good use of YAML: I would rather write task files in an actual programming language, since they are — at their core — a program. (The tasks do have some metadata attached to them, but the core task itself is a program. A function in some real language can get metadata attached to it in a number of ways, and that would be a better solution.)

Re:Why are we templating YAML? by erik_seaberg on Thursday February 07, 2019 @10:49PM (#19111363)
YAML quoting is only optional if you're obsessive enough to always remember to quote the country code for Norway.
Re:Why are we templating YAML? by pytester on Thursday February 07, 2019 @08:18PM (#19110463)
>For configs, I think TOML beats YAML hands down

If you compare a medium sized, 3 level deep TOML document with an equivalent YAML document you'll find that the TOML document is up to 50% longer.

None of that additional verbosity adds meaning - or readability.

Part of this is the extra syntactic noise but most of it is because all of the key names have to be defined explicitly above the key values for each value whereas in YAML you just need to put the values below an indent.

TOML is tolerable for small, very lightly nested config files but even medium sized TOML configurations get ugly very fast.

Re:Why are we templating YAML? by guitarbill on Thursday February 07, 2019 @07:23PM (#19110082)
It's worth giving Ansible some credit here. Alternatives like Chef, can and do contain arbitrary Ruby logic which can make Chef "cookbooks" horrible to figure out and debug.

So while Ansible may not be great, other solutions also have drawbacks, and it isn't quite as easy as do <x>.

Re:Why are we templating YAML? by rbankston on Thursday February 07, 2019 @09:18PM (#19110884)
I've worked with both. The simple items are nicer in ansible but when there are a lot of custom roles and functions ansible isn't much better than chef from what I've seen in practice.
Re:Why are we templating YAML? by jszymborski on Thursday February 07, 2019 @05:27PM (#19109258)
I really like TOML as a compromise between "looks pretty" and "parses well".

That being said, I find it funny that the author is crying afoul, saying that YAML isn't as well suited to be templated as JSON is, when XML had schemas sorted out ages ago.

People ran away from XML, saying it was this verbose, ugly, bandwidth hungry (back when we were sending XML over AJAX) behemoth (which it totally is)... but I think when your usecase is complicated enough to worry about templating, you should take a hard look at XML and ask whether it might suit your better.

Re:Why are we templating YAML? by quickthrower2 on Thursday February 07, 2019 @04:55PM (#19108969)
Baffles me. I don't like any language or file format where whitespace matters. Even Haskell bothers me in this regard. To me white space shouldn't add cognitive load. I want to look at the symbols not the formatting of the antisymbols to understand what is going on.

Write JSON and use your editor tools to format it with nice indentation, and you are sweet!

That said Yaml makes an excellent format for reading, but not for writing.

Re:Why are we templating YAML? by wolph on Thursday February 07, 2019 @05:03PM (#19109028)
I personally love Python where whitespace matters a lot too. But I regularly find myself fighting with the whitespace in yaml for some reason.

To me yaml (especially with templating) seems like a very contrived way of avoiding having to program, while still effectively programming. I much prefer json, or if more intelligence is required, actual javascript objects.

The only big downside of json where yaml does shine is the support for comments within your files.

Re:Why are we templating YAML? by amanzi on Thursday February 07, 2019 @04:57PM (#19108987)
White space matters with Python but I rarely run into issues with indentation in Python. But YAML on the other hand, I have nothing but nightmares.
Re:Why are we templating YAML? by kccqzy on Thursday February 07, 2019 @05:03PM (#19109033)
I have the same experience. I just think think it is possible to design easy-to-write indentation-sensitive formats but YAML is not. For example is always baffles me that

    a:
    - b
    - c
has a list inside an object but the list is not further indented. There's in fact a hierarchy relationship but absolutely no indentation.
Re:Why are we templating YAML? by Rapzid on Thursday February 07, 2019 @05:08PM (#19109084)
You can indent it though, but the hyphen already means a list item.

That is a preferred syntax for many that are used to YAML though.

Re:Why are we templating YAML? by XorNot on Thursday February 07, 2019 @08:51PM (#19110676)
The hyphen is associated with the a though. In what's posted, a is a list, not a map - the type of the parent is being declared by that hyphen.
Re:Why are we templating YAML? by kccqzy on Thursday February 07, 2019 @05:14PM (#19109137)
That's kind of my point. Indentation is optional, so many don't indent, making the resulting document harder to understand.
Re:Why are we templating YAML? by emmelaich on Thursday February 07, 2019 @08:23PM (#19110496)
FWIW, I think yaml / ansible linters and style checkers typically require indentation.
Re:Why are we templating YAML? by mikepurvis on Thursday February 07, 2019 @05:09PM (#19109090)
That's at your option, though. It's perfectly valid yaml to indent the list as many spaces as you want.
Re:Why are we templating YAML? by setr on Thursday February 07, 2019 @05:26PM (#19109248)
I think its because python has very strict and very simple indentation rules (a nested block must be indented, and the file requires consistent indentation. Is there anything else?)

YAML gives you options, or varies necessity somewhat arbitrarily on the structures, which is (marginally) good for reading, but a lot of headache in writing

Re:Why are we templating YAML? by amanzi on Thursday February 07, 2019 @05:32PM (#19109292)
Also because the longer the YAML file, the bigger your indentation gets. It gets exponentially worse as you go.
Re:Why are we templating YAML? by setr on Thursday February 07, 2019 @09:08PM (#19110809)
In python that’s been a non-issue, but I’m not sure YAML can reset indentation as python does with functions
Re:Why are we templating YAML? by crdoconnor on Thursday February 07, 2019 @05:10PM (#19109100)
I never quite understood this attitude. Not treating whitespace as meaningful means it's necessary to have a longer, syntactically noisier file.

That's fine if you're primarily concerned with computers exchanging data (JSON) but where readability and writability matters, extra syntactic weight is a headache.

I can't imagine writing JSON by hand but I write plenty of YAML by hand every day without issues.

Re:Why are we templating YAML? by quickthrower2 on Thursday February 07, 2019 @09:11PM (#19110833)
Because deleting a tab could entirely change the meaning of the file, breaking everything, with no compile time errors. IDE tooling or linting might go somewhat to make me more comfortable with YAML.
Re:Why are we templating YAML? by emmelaich on Thursday February 07, 2019 @08:22PM (#19110482)
> I don't like any language or file format where whitespace matters. .... To me white space shouldn't add cognitive load.

1. there is no language where whitespace does not matter

2. the very purpose of indentation is to ease cognitive load.

Now, you may not want it to be inflexible or mandatory but your argument needs elucidating.

Re:Why are we templating YAML? by quickthrower2 on Thursday February 07, 2019 @09:09PM (#19110825)
Ok: I don't like any language of file format where the regex s/\s+/\s/g would change the meaning of the program except in parts of the program within string delimiters, except for the parts of string delimiters which contain literal code e.g. ins ES6 `Hello ${world}`.
Re:Why are we templating YAML? by andybak on Friday February 08, 2019 @12:18AM (#19111745)
So you prefer languages way that regex only appears to change the meaning?

Non-semantic white space can be deceptive. Semantic whitespace isn't.

Re:Why are we templating YAML? by emmelaich on Friday February 08, 2019 @12:05AM (#19111708)
OK, but I'd restrict the set even more to exclude voluntary indentation. I mean, you'd want that caught by style checkers anyway, even if the language didn't require it.

So we're left with the cases where you'd like compact code / one-liners but are forced to use multiple lines for language syntax reasons.

I'd say that that's a pretty small set of cases and long way from the phrase "I don't like whitespace sensitive languages" which you (and many others!) use.

I conceded it can be annoying in Python and I do wish there was an escape route sometimes.

Re:Why are we templating YAML? by h1d on Thursday February 07, 2019 @11:36PM (#19111562)
Experienced programmers use decent editors to indent their code anyway but at least you don't see codes with indentation going left and right at random locations with Python.
Re:Why are we templating YAML? by Rapzid on Thursday February 07, 2019 @05:04PM (#19109049)
The best thing that ever happened to Cloud Formation was the new YAML syntax.

Writing YAML is easy with a good editor like VSCode. Install the YAML code outline extension for sugar; works great for OpenAPI specs. YAML flow style offers some good options for keeping the file compact.

Re:Why are we templating YAML? by SamWhited on Thursday February 07, 2019 @05:21PM (#19109191)
I'm all for good editor support, but you shouldn't need it to write a simple document. Complicated editors should be supporting tools at best, because you won't always be in a position to use one. The things we write should be simple enough to be written and understood by hand with minimal mistakes (then we can make it even easier in some cases using nice editors and tooling).
Re:Why are we templating YAML? by shirakawasuna on Thursday February 07, 2019 @05:32PM (#19109297)
I don't think I've ever used anything under than vanilla vim for editing YAML files... with next to zero issues.

The problem for me is that the multiple ways to do the same thing result in something that's pretty opaque to clear specifications that aren't just very simple examples / structures.

Re:Why are we templating YAML? by SamWhited on Thursday February 07, 2019 @05:58PM (#19109533)
Me too, except it's just always wrong the first time… drives me nuts. Maybe it's just me.
Re:Why are we templating YAML? by mixmastamyk on Thursday February 07, 2019 @04:57PM (#19108982)
It's a lot easier for end-users to read and write by hand. If that is not a requirement, a simpler machine-friendly format is likely better.
Re:Why are we templating YAML? by amanzi on Thursday February 07, 2019 @05:00PM (#19109013)
Writing YAML is not easy for end-users. The indentation--especially in large files--is really difficult to keep track of, and some really basic stuff like when to quote strings is not consistent at all.
Re:Why are we templating YAML? by shirakawasuna on Thursday February 07, 2019 @05:31PM (#19109280)
It definitely is. You have no idea how intimidating an equivalent JSON file would be for a new user creating a pretty simple k8s setup.

YAML hides most of the scary brackets and quotes so that new users can focus on copy-and-pasting semantic tidbits.

That said... I don't like YAML...

Re:Why are we templating YAML? by mixmastamyk on Thursday February 07, 2019 @05:15PM (#19109150)
Nope, indentation is not a problem with indentation guides. A misplaced comma in a JSON file however kills it, gets me every time I have to write it by hand.

Also, it is true YAML has too many features, though I find they are typically ignored or disabled.

Re:Why are we templating YAML? by krapp on Thursday February 07, 2019 @05:20PM (#19109181)
A misplaced comma is also not a problem in an editor with syntax highlighting, but between the two, indentation could take longer to fix.
Re:Why are we templating YAML? by mixmastamyk on Thursday February 07, 2019 @05:25PM (#19109240)
Syntax highlighting doesn't count commas. The problem is JSON doesn't allow trailing commas, so adding or reordering frequently results in a difficult to diagnose error.
Re:Why are we templating YAML? by krapp on Thursday February 07, 2019 @05:33PM (#19109306)
Finding out if a set ends in a comma or not still makes it easier to debug than YAML.
Re:Why are we templating YAML? by mixmastamyk on Thursday February 07, 2019 @05:46PM (#19109428)
Not my experience, 10x more trouble with long JSON.
Re:Why are we templating YAML? by krapp on Thursday February 07, 2019 @05:02PM (#19109023)
But the simpler format of JSON should be easier to write and read.
Re:Why are we templating YAML? by deathanatos on Thursday February 07, 2019 @05:27PM (#19109259)
No comments, constant need to quote every string, inane comma requirements, no multiline string, little typing support.
Re:Why are we templating YAML? by krapp on Thursday February 07, 2019 @05:37PM (#19109332)
Many language require strings to be quoted, and calling the comma requirements inane is an opinion, not an objective fact.

You're not saying anything about the readability of one versus another, you're just listing gripes you have about JSON.

Re:Why are we templating YAML? by deathanatos on Thursday February 07, 2019 @07:17PM (#19110043)
> Many language require strings to be quoted

The languages we're comparing here are JSON and YAML. The latter does not require quoted strings, except in ambiguous cases. (And even then, actually, it technically isn't required, though it is usually the easiest thing to do.) The absence of these quotes makes the syntax get out of the way of the reader, and makes YAML comparatively easier to read.

> comma requirements inane is an opinion, not an objective fact.

I boiled it down to a simple statement, but disallowing a comma at the end of repeated grammars means that adding something to that line causes unnecessary noise in the diff. For example, say I add a single item to the end of a JSON list. The diff I would love to see is:

   [
      a,
      b,
  +   c,
   ]
which makes it rather straight-forward to the code reviewer / reader of the diff that we're simply adding a single item c. But JSON's grammar forces this diff:

   [
      a,
  -   b
  +   b,
  +   c,
   ]
which makes it harder to see what the semantic change is, because simple syntactic changes are now clouding the picture. Also, I find that people's mental model of the task ("add item to end of list") causes them to forget that they need to add a comma to the item above it, resulting in syntax errors down the road.

(Better diff tooling can help here, but often I find we have to work with the most primitive of tooling; if the grammar can lend itself towards such simple tooling, s.t. that tooling is more effective despite its simplicity, why not? And here, we can: grammatically, whether the list ends or does not end in a comma is rather meaningless, and JSON (and for a long time, JS) were pretty alone in this opinion. Most other languages allow that trailing comma. The only other one I can think of is SQL.)

> You're not saying anything about the readability of one versus another, you're just listing gripes you have about JSON.

The gripes I have about JSON don't apply to YAML. Hence, that's why I prefer using YAML, which was the original question.

Re:Why are we templating YAML? by afiori on Friday February 08, 2019 @08:32AM (#19113925)
> The latter does not require quoted strings, except in ambiguous cases.

I believe it is the right choice to always quote strings, there should be various quoting formats as no single one is perfect (normal strings with reasonable escapes,raw string, multiline-with-indentation, multiline raw strings...) and you should be able to use unquoted keys (TOML has a good balance on this problem), but unquoted strings as a default are unnecessary and problematic.

Re:Why are we templating YAML? by StevePerkins on Thursday February 07, 2019 @09:00PM (#19110749)
You post a comment describing one alternative as "simpler" and "easier to read" compared to another.

Respondent lists five specific reasons in support of the opposite conclusion.

You dismiss those as "opinions, not objective facts". Ignoring that the same could be said for your parent comment, to a much greater degree.

Re:Why are we templating YAML? by krapp on Friday February 08, 2019 @12:11AM (#19111728)
No comments is... fair. That does put a limit on the legibility of JSON files, unless you want to count dumb hacks like comment strings.

Having to quote strings in JSON, however is still simpler than the multitude of ways strings can be declared in YAML. You know a string is a string in JSON because of the quotes... knowing whether something is a string or not in context is more difficult in YAML because of the more complex syntax.

You can learn the entire syntax of JSON in minutes. Objects, arrays, string keys, and a few primitive types... that's it. How long would it take to learn all of YAML? How explicit is its syntax versus JSON? Of course JSON is far simpler, and being simpler, it's easier to read.

I dismissed "inane comma requirements" as an opinion, not everything the commenter said. The only reason it's "inane" is because the commenter doesn't like it personally.

Not having multiline strings, to me, doesn't affect readability much at all, although it is unfortunate. Turn on text wrapping in your editor, it's the same thing.

Typing support doesn't affect readability either. I'd like to have a date type in JSON too but

    2013-03-01T16:15:09+01:00
or whatever is just as readable as

    "2013-03-01T16:15:09+01:00"
Re:Why are we templating YAML? by h1d on Thursday February 07, 2019 @11:43PM (#19111595)
Parent's points are exactly the reason I'd never consider JSON as config format.
Re:Why are we templating YAML? by pdonis on Friday February 08, 2019 @02:19AM (#19112173)
Not when you take into account all the clutter in JSON: braces, brackets, quotes. Those are extra things to type and extra things to have to filter out when you read.
Re:Why are we templating YAML? by mixmastamyk on Thursday February 07, 2019 @05:16PM (#19109154)
See sibling comment.
Re:Why are we templating YAML? by IshKebab on Thursday February 07, 2019 @05:04PM (#19109048)
I think it is partly momentum - at this point most CI systems use YAML in spite of its insanity - even Azure Pipelines!

And partly it's because the syntax for multiline string literals is very minimal, which is kind of a nice feature for CI since you tend to have a lot of them.

It's still insane though. TOML is much more reasonable.

Re:Why are we templating YAML? by nightfly on Thursday February 07, 2019 @04:55PM (#19108971)
I've taken the strategy of emitting JSON, but accepting YAML input (maybe a restricted subset in cases where it's untrusted data). YAML can function as a super-set of JSON, so you can have comments in hand-written/modified data this way while emitting a simpler to parse data format.
Re:Why are we templating YAML? by pavon on Thursday February 07, 2019 @05:44PM (#19109405)
Because json is poorly suited for config files, and it is very convenient to have config files that closely mirror apis, like kubernetes does. I've seen too many teams who try to use json for config end up developing yet another custom superset of json to allow comments or multiline strings or such. If you are going to use a superset of json might as well use an existing one like yaml or json5.

That said I agree with all the criticisms of templating yaml. We have to do the same (with helm and other tools), and I have pushed hard to adopt conventions that we only use flow-style and not block-style to avoid all the white space problems when splicing together chucks of yaml. And on the plus side we get trailing commas and other such niceties which don't exist in json and make it harder template.

Re:Why are we templating YAML? by atsjie on Thursday February 07, 2019 @05:05PM (#19109058)
Anyone working with cloudformation would wonder why we ended up writing json?