News for hackers. Stuff that matters.
Re:How we use HashiCorp Nomad by ianlevesque on Saturday June 06, 2020 @12:18PM (#23440212)
After trying out most of the kubernetes ecosystem in the pursuit of a declarative language to describe and provision services, Nomad was a breath of fresh air. It is so much easier to administer and the nomad job specs were exactly what I was looking for. I also noticed a lot of k8s apps encourage the use of Helm or even shell scripts to set up key pieces, which defeats the purpose if you are trying to be declarative about your deployment.
Re:How we use HashiCorp Nomad by alledm on Sunday June 07, 2020 @04:38AM (#23445809)
https://kapitan.dev/ is the one-stop shop that covers for true declarative configuration with either jsonnet, python (kadet) and jinja, amazing secret management with support of gkms, awskms, gpg, vault.

It is simpler than other tools, because you can get started without even touching jsonnet or python or anything else, when using our generators.

It does more than all the other tools combined, as it replaces helm+helmfile+gitcrypt or kustomize.

It's universal, so you can use it on non-kubernetes situations where other tools leave you high and dry.

With the new generator library, you can have 1 template and use it to configure many services. Check our examples at https://github.com/kapicorp/kapitan-reference

We have just released: * https://github.com/kapicorp/kapitan-reference a repo with examples for quick-start. It includes our generator as explained in https://medium.com/kapitan-blog/keep-your-ship-together-with.... * https://github.com/kapicorp/tesoro a secret "webhook controller" to seamlessly handle Kapitan secrets in your cluster. Better than sealed-secrets because there is no need to convert secrets and it supports KMS like google and aws together.

Get started with our blog: https://medium.com/kapitan-blog or join our kubernetes slack on #kapitan

Re:How we use HashiCorp Nomad by ognyankulev on Saturday June 06, 2020 @12:24PM (#23440273)
Helm charts are declarative way of deploying app(s) and their accompanying resources.
Re:How we use HashiCorp Nomad by q3k on Saturday June 06, 2020 @12:26PM (#23440283)
Helm, however, is objectively terrible with its yaml-based templating language and zero practical modularity.
Re:How we use HashiCorp Nomad by speedgoose on Saturday June 06, 2020 @12:46PM (#23440445)
Indeed. Helm offers great features but it suffers from the kubernetes unnecessary complexity and by using golang templates in YAML.

When I started with kubernetes I converted my small Docker compose files to kubernetes files. Later I rewrote everything in helm charts. Now it's almost more YAML and golang templates lines than business logic lines in my applications.

I'm considering to go back to Docker compose files. It's simple, readable, and easy to maintain.

Re:How we use HashiCorp Nomad by q3k on Saturday June 06, 2020 @12:48PM (#23440465)
Highly recommend trying Jsonnet (via https://github.com/bitnami/kubecfg and https://github.com/bitnami-labs/kube-libsonnet) as an alternative. It makes writing Kubernetes manifests much more expressive and integrates better with Git/VCS based workflows. Another language like Dhall or CUE might also work, but I'm not aware of a kubecfg equivalent for them.

Jsonnet in general is a pretty damn good configuration language, not only for Kubernetes. And much more powerful than HCL.

Re:How we use HashiCorp Nomad by threentaway on Saturday June 06, 2020 @02:58PM (#23441477)
If you like those, I'd take a look at Grafana's Tanka [0]. It also uses jsonnet but has some additional features such as showing a diff of your changes before you apply, easy vendored libraries using jsonnet-bundler, and the concept of "environments" which prevents you from accidentally applying changes to the wrong namespace/cluster.

[0] https://github.com/grafana/tanka

Re:How we use HashiCorp Nomad by q3k on Saturday June 06, 2020 @03:41PM (#23441713)
I looked at it, I don't like it for the same reason as I dislike many other tools in this space: it imposes its own directory structure, abstraction (environments) and workflow. I'm a fan of the kubecfg-style approach, where it lets you use whatever sort of structure makes sense for you and your project.

It's a 'framework' vs 'library' thing, but in the devops context.

Re:How we use HashiCorp Nomad by dilyevsky on Sunday June 07, 2020 @03:45AM (#23445552)
I worked on this at previous gig - https://github.com/cruise-automation/isopod

Same use-case but uses Starlark dsl instead of jsonnet

Re:How we use HashiCorp Nomad by kraig on Saturday June 06, 2020 @01:11PM (#23440655)
Funny, I thought jsonnet was an even worse experience than templating yaml
Re:How we use HashiCorp Nomad by p_l on Saturday June 06, 2020 @05:18PM (#23442303)
The problem with templating YAML is that you're templating text, in a very sensitive to whitespace syntax. By definition, Jsonnet avoids that because it operates on the data structure, not on their stringified representation.
Re:How we use HashiCorp Nomad by q3k on Saturday June 06, 2020 @01:21PM (#23440738)
My experience with jsonnet varied: there's good jsonnet code, and there's bad one, too. Just like with any programming language, you have to apply good software engineering practices. Text templated YAML, however, is terrible by design.
Re:How we use HashiCorp Nomad by AlphaSite on Saturday June 06, 2020 @03:36PM (#23441690)
Ive found the whole k14s eco system is pretty great to (ytt + kbld + kapp).
Re:How we use HashiCorp Nomad by q3k on Saturday June 06, 2020 @03:43PM (#23441727)
ytt is even more templating-yaml-with-yaml, so it all ends up being a bargain bin Helm. There's no reason to do this over just serializing plain structures into YAML/JSON/...
Re:How we use HashiCorp Nomad by AlphaSite on Saturday June 06, 2020 @05:21PM (#23442325)
It avoids the whole set of issues you get with templating yaml with helm since its structure aware.

Also you can actually write pure code libraries in starlark (basically a subset of Python)

Re:How we use HashiCorp Nomad by q3k on Saturday June 06, 2020 @06:42PM (#23442964)
I'm aware, but I just don't understand the point of it. How is it in any way better than a purpose-specific configuration language like Dhall/CUE/Jsonnet? What's the point of having it look like YAML at all?
Re:How we use HashiCorp Nomad by sandGorgon on Saturday June 06, 2020 @03:35PM (#23441679)
We just moved to the kubernetes ecosystem - specifically for integration with spot instances of AWS (and cut our costs by 60%).

But I hate the UX of kubernetes. Compose was beautiful.

I have hope - since compose files have become an independent specification. https://www.docker.com/blog/announcing-the-compose-specifica...

Kubernetes distro built around the compose file specification would be a unicorn.

Re:How we use HashiCorp Nomad by ramaro on Sunday June 07, 2020 @05:17AM (#23445964)
https://kapitan.dev/ is the one-stop shop that covers for true declarative configuration with either jsonnet, python (kadet) and jinja, amazing secret management with support of gkms, awskms, gpg, vault. It can also render helm charts!

It is simpler than other tools, because you can get started without even touching jsonnet or python or anything else, when using our generators. It does more than all the other tools combined, as it replaces helm+helmfile+gitcrypt or kustomize. It’s universal, so you can use it on non-kubernetes situations where other tools leave you high and dry.

With the new generator library, you can have 1 template and use it to configure many services. Check our examples at https://github.com/kapicorp/kapitan-reference

We have just released: * https://github.com/kapicorp/kapitan-reference a repo with examples for quick-start. It includes our generator as explained in https://medium.com/kapitan-blog/keep-your-ship-together-with...

* https://github.com/kapicorp/tesoro a secret “webhook controller” to seamlessly handle Kapitan secrets in your cluster. Better than sealed-secrets because there is no need to convert secrets and it supports KMS like google and aws together. Get started with our blog: https://medium.com/kapitan-blog or join our kubernetes slack on #kapitan

Re:How we use HashiCorp Nomad by orthoxerox on Sunday June 07, 2020 @05:27AM (#23446001)
It is. But k8s has no convenient way of parameterizing releases that can beat Helm. A simple stateless application needs:

- a deployment - a service - an ingress - a config map (or several) - a secret (or several)

It's even worse for stateful applications.

And each of the resource definitions is 60% boilerplate, 35% application-specific and 5% release- or environment-specific.

Helm would probably be a nice and neat tool if it had stopped at maintaining a simple map of variable names to values. But since applications need things like "if the user said SQLite, add a pvc, a configmap and a secret and refer to them in the ss, if she said Postgres, go pull another chart, deploy it with these parameters, then add this configmap and this secret and refer to them in the ss", Helm is an overcomplicated mess.

Re:How we use HashiCorp Nomad by f355 on Thursday June 11, 2020 @04:24AM (#23486424)
k8s has a very convenient built-in way of parametrizing releases called kustomize, it is supported by kubectl since quite a few versions ago.
Re:How we use HashiCorp Nomad by namelosw on Saturday June 06, 2020 @03:31PM (#23441657)
Sometimes I even wish they could embed a JavaScript interpreter... After all, YAML is almost equivalent to JSON, which the perfect templating language for JSON is -- JavaScript tbh.

Or people have to keep inventing half baked things.

Re:How we use HashiCorp Nomad by q3k on Saturday June 06, 2020 @03:46PM (#23441737)
The problem isn't JSON or YAML: it's text templating serialization formats, instead of just marshalling/serializing them.
Re:How we use HashiCorp Nomad by AaronFriel on Saturday June 06, 2020 @05:39PM (#23442446)
There is a tool called Pulumi that does exactly that, and it's excellent.
Re:How we use HashiCorp Nomad by justinsaccount on Saturday June 06, 2020 @08:05PM (#23443479)
json is already valid yaml
Re:How we use HashiCorp Nomad by rayanimesh on Saturday June 06, 2020 @01:08PM (#23440629)
> Helm charts are declarative way of deploying app(s) and their accompanying resources.

How do you make helm chart deployment declarative? `helm install` is not declarative (in my understanding `kubectl apply` is declarative and `kubectl create` is not. Let me know if my understanding of declarative is wrong). Thanks.

Re:How we use HashiCorp Nomad by meitham on Sunday June 07, 2020 @02:42AM (#23445299)
I think you’re confusing “declarative” with “idempotent”! The helm equivalent to “kubectl apply” is “helm upgrade —-install”.
Re:How we use HashiCorp Nomad by rayanimesh on Sunday June 07, 2020 @04:09AM (#23445662)
Yes. Thank you.
Re:How we use HashiCorp Nomad by sciurus on Saturday June 06, 2020 @07:21PM (#23443227)
`helm template | kubectl apply` ?
Re:How we use HashiCorp Nomad by rayanimesh on Sunday June 07, 2020 @01:10AM (#23444920)
This is one way of doing declarative using helm, but this doesn't guarntee resource creation order. `helm install` creates resources in order while `kubectl apply` creates in order of alphabetically filename. This may lead to some unexpected issues.
Re:How we use HashiCorp Nomad by phlakaton on Saturday June 06, 2020 @12:48PM (#23440461)
Helm charts, round these parts, are a glorified yet byzantine templating system to spit out K8s manifests.
Re:How we use HashiCorp Nomad by ianlevesque on Saturday June 06, 2020 @12:47PM (#23440452)
Maybe I was doing it wrong but every guide was verb based - “helm install X”. My declarative ideal ended up being a text file full of helm install commands and that wasn’t what I wanted.
Re:How we use HashiCorp Nomad by rochacon on Saturday June 06, 2020 @03:11PM (#23441543)
Quick-start guides takes the easiest path to get something running, which is `helm install` in the Helm world.

If you want to have complete control of what you're pushing to the API, use Helm as an starting point instead, run `helm template` and save the YAML output to some file, publish it using `kubectl` or some other rollout tool. I recommend using `kapp` [1] for rollouts.

[1] https://get-kapp.io/

Re:How we use HashiCorp Nomad by halbritt on Saturday June 06, 2020 @10:39PM (#23444335)
This is precisely how Spinnaker behaves with Helm. It renders the template into manifests and then deploys them.

Kinda loses the benefit of Helm hooks, but if one is using Spinnaker, there are other ways to do the same thing hooks can do.