Our team tried to migrate to AWS ECS a few times and found it much harder to abstract stuff from devs / create self-service patterns.
That said it's not a walk in the park. You will need to scratch your head a little bit to setup consul + nomad + vault + a load balancer correctly.
This is thing I like about the HashiCorp tools. You don't have to eat the whole cake in a single sitting.
To avoid the pain of managing a CA and passing out certificates for TLS between services, I use a wireguard mesh and bind nomad, consul and vault to these wg interfaces. This includes all the chatter of these components, as well as the services I deploy with nomad. It's configured such that any job can join the "private" wireguard network or "public" internet gateway.
It takes a few days to set up, but it's very easy to manage.
I’m a freelancer that hosts client stuff and I need something between “SSH into server” and “kubernetes.”
No, I never did buy the docker hype. Seem to be doing okay.
I've been wondering, would it make sense to try to package all that into a single, hopefully simple and easily configurable, Linux image? And if it might be, why hasn't anyone done that yet?
I will say that I looked at HCL and it looks very nice:
https://github.com/hashicorp/hcl
But somehow it's not as popular as a mess of YAML and Go Templates? That genuinely leaves me scratching my head. I guess it's because people pick platforms and not languages? (BTW, in 2009 I designed and implemented the template language that Go templates are based on, and I find their common application pretty bizarre, e.g. in some Helm charts I looked at from this thread)
Oil is growing a config dialect that looks a lot like HCL (although it's convergent evolution; I've never used it.) I think there is a lot of room for mixing declarative and imperative; as far as I can see HCL is mostly declarative (defining data structures).
Anyway I'd be interested in reading about HashiCorp stuff but for some reason in my neck of the woods I don't hear too much about it. Maybe that's because they're paid services and the open source Kubernetes seems attractive by comparison? Or is it more of a VM vs. container thing?
I’m curious if you’ve taken a look at Nomad and the other HashiCorp tools? They appear focussed and compositional, as you say, and this is why we are probably going to adopt them instead of K8s - they seem to be in a strong position to replace the core of K8s with something simpler.