News for hackers. Stuff that matters.
Re:Kubernetes Failure Stories by malkia on Sunday January 20, 2019 @08:00PM (#18956725)
It took me a while to get comfortable in Borg (and in general that your binary can take hundredths of verbosely written command-line arguments (coming from gamedev, I was in a bit of shock state for a while)... But then got used to it - still I felt I could never fully internalize the evaluation rules - but the other tooling (diffing) really helped in that respect.

One thing I've really appreciated, was how one could enable/disable things based on the binary version rolled in, and if it's rolled back the state goes back.

Basically something like this:

    {
       new_exp_feature = (binary_compiled_after_changelist( 123456789 ) || binary_compiled_with_cherrypicks( { 123456795, 1234567899 } )
    }
Since piper is changelist based (like perforce/svn), each "CL" goes up atomically, so you can use this to say - this specific flag should get turned ON only if my binary has been compiled with base CL > 12345789 or if it was compiled with earlier, had these cherrypicks (e.g. individual Changelists) built with it. But this was heavily integrated with the whole system - e.g. each binary would basically be built at some @base_cl and additional @{cherry_pick_cl1, chery_pick_cl2, ..} maybe applied. For example the team decides to release with verison @base_cl, but during the release bugs were found, and rather than rolling to a new @base_cl, just individual cherry picks maybe be pushed - so basically you can then control (in your configuration) how to act (configuration could be pushed indepedntly of your binary, ... though some systems would bundle them together)... And then if you have to rollback, the Borgcfg would re-evaluate all this, and decide to flip the switch back (that switch would simply emit something like --new_exp_feature=true or --new_exp_feature=false (or --no-new_exp_feature, it was long time ago so I could be wrong)).

With git/hg - you no longer have such monotonic order, but also that monotonic order worked best with monorepos (or maybe I'm just too narrow-sighted here)...

Re:Kubernetes Failure Stories by akhilcacharya on Monday January 21, 2019 @01:08AM (#18958028)
From this comment thread I’m beginning to think I’m one of the few people on HN that hasn’t used Borg.

All of this seems way more complicated than the tools we use at my company. Is there a specialized need here I’m not seeing?

Re:Kubernetes Failure Stories by justicezyx on Sunday January 20, 2019 @09:25PM (#18957122)
You seem confuse Borg and borgcfg.

The evaluation rules are merely a borgcfg artifact.

Disclaimer: I maintain borgcfg.

Re:Kubernetes Failure Stories by ithkuil on Monday January 21, 2019 @02:20AM (#18958372)
I found https://jsonnet.org/ to fix several issues with the Borg configuration language.

https://github.com/ksonnet/kubecfg is an attempt to reboot the borgcfg experience with k8s + jsonnet

Re:Kubernetes Failure Stories by arianvanp on Monday January 21, 2019 @04:15AM (#18958928)
There is also Https://GitHub.com/dhall-lang/dhall-kubernetes which is built on top of dhall, another competitor of jsonnet
Re:Kubernetes Failure Stories by justicezyx on Monday January 21, 2019 @02:24AM (#18958396)
There is no such thing as borgcfg experience. It’s just a configuration dsl applied to producing borg specs.
Re:Kubernetes Failure Stories by ithkuil on Monday January 21, 2019 @07:21AM (#18959854)
You might take it for granted, and thus not experience it as an "experience", but if you use other tools that are popular in the k8s world (such as helm) you might feel a tinge of nostalgia.

For example, {borg,kube}cfg allow you to import an existing config and override it so you can adapt it to another scenario (different things in different clusters, like prod vs staging, or a cluster has a new feature while another one doesn't etc).

Furthermore, the overrides are described with the same "shape" as the things they override, and can override things that weren't necessarily marked as overridable.

Compare this with the current state of affairs with helm, where the only way for users to be able to inject e.g. the resources requests and limits in a pod spec is for the original template author to have foreseen that need and explicitly added a hook in the template so that values from another file (the values.yaml) can be injected into it.

        spec:
          imagePullSecrets:
            - name: {{ .Values.image.pullSecret }}
          containers:
            - name: {{ template "mycontainer.name" . }}
              ......
              resources:
    {{ toYaml .Values.resources | indent 12 }}
          volumes:
              ......
Re:Kubernetes Failure Stories by malkia on Monday January 21, 2019 @02:56AM (#18958557)
Sure yes - I meant borgcfg, not borg - stupid me...