I actually just deployed k8s on a raspberry pi cluster on my desk (obviously as a toy, not for production) and it took about an hour to get things fully functional minus RBAC.
> What I'm saying is that Nomad is a general purpose workload scheduler
Yeah, Nomad and k8s are not direct replacements at all. Nomad is a great tool for mixed workloads, but if you're purely k8s then there are more specific tools you can use.
> I meant to write 4.4K lines
Just a small difference! Glad no one wrote 44k lines of yaml, that's just a lot of yaml to write...
> close to 12k lines
Our production cluster (not the raspis running on my desk!) runs in about 4k lines, but we have a fairly simple networking and RBAC layer. We also started from just kubernetes and grew organically over time, so I'm sure someone starting today has a lot of advantage to get running more easily.
I mean, don’t get me wrong, it works - now at least. Never liked it until 1.12 tbh, which is when a bunch of things settled.
The article is about “maybe you don’t need...” and as an anecdote I helped build a really successful $$$ e-commerce business with a couple of hundred micro-services on an on-prem LXC/LXD “cloud” using nomad, vault & consul.
You can use these tools independently of each other or have them work together - unixy.
I have anecdotes from my last couple of years on k8s as well, and... it just ends up with a much more narrow scope.
Ah, no, it's not about replacing functionality. It's about opening up for general integrations and ease of use.
If you've set up a fully fledged infrastructure on k8s with all the bells and whistles, there's a whole lot of configuration going on here. Like a whole lot!
I most certainly can't replace all of the above with those two tools, but they make it easier to integrate in any way I see fit. What I'm saying is that Nomad is a general purpose workload scheduler, where k8s is k8s POD's only. Consul is just providing "service discovery", do with it what you want. And so on...
Having worked a couple of years using both these setups I'm a bit torn. K8s brings a lot, no doubt, but I get the feeling that the whole point of it is for google to make sure you _do not ever invest in your own datacenters_. k8s on your own bare metal at least used to be not exactly straight forward.