News for hackers. Stuff that matters.
Re:Kubernetes Failure Stories by ben_jones on Wednesday June 12, 2019 @12:16PM (#20166226)
I feel like their needs to be a sanity check. Realistically what does kubernetes do to "not make software fail"? Health checks? Autorestarting containers that crash? Enforce various timeouts? Relatively quick distributed configuration deployments? These can be hard problems for sure, but do you need all of kubernetes to get these benefits in a production application?
Re:Kubernetes Failure Stories by theflyinghorse on Wednesday June 12, 2019 @12:19PM (#20166266)
Essentially why we made a choice to go with Swarm - less features, although all the features we needed are there (restarting containers, scaling them up or down and across several machines).
Re:Kubernetes Failure Stories by haolez on Wednesday June 12, 2019 @03:09PM (#20168199)
I made this choice as well and I regret it. I've had a nice share of bugs regarding DNS and service discovery.

However, those bugs seem to be fixed after version 19.x, but I'm still bitter about all the problems that I had because of simple features failing on me.

Re:Kubernetes Failure Stories by tepidandroid on Wednesday June 12, 2019 @03:15PM (#20168247)
The same can be said of Hashistack (Consul, Nomad, Vault). Orders of magnitude simpler with the obvious tradeoff of being less featureful.
Re:Kubernetes Failure Stories by chucky_z on Thursday June 13, 2019 @12:42AM (#20171850)
Consul and Vault offer functionality that Kubernetes is completely lacking, though.
Re:Kubernetes Failure Stories by StavrosK on Thursday June 13, 2019 @07:38PM (#20179521)
Is it worth going with those? I'm torn between deploying stuff on Kubernetes (on a managed control plane, of course, which you can get for free nowadays with providers as long as you use their own machines) and rolling my own with the Hashistack.
Re:Kubernetes Failure Stories by tepidandroid on Thursday June 13, 2019 @09:35PM (#20179980)
Depends on your use-case, really. IMO, Hashistack is fine if you have well-defined requirements. You can pick and choose the components you need (including non Hashi stuff) and integrate them with relative ease, keeping resource footprint and operational complexity at a minimum. I’ve found that it’s quite easy to pinpoint problems when they arise.

Kubernetes is more of an everything-and-the-kitchen-sink approach that you can’t really outgrow but because of its monolithic design, debugging can be quite challenging. It does, however, come with a much larger and much more active community that you can go to for help.

Re:Kubernetes Failure Stories by y4mi on Wednesday June 12, 2019 @01:16PM (#20166957)
But swarm only has overlay network iirc?

Swarm is great to get simple orchestration going, but it doesn't really do the same thing.

With k8s you can configure multi-point routing, storage backends and load balancing among other things.

With swarm, you get ... Overlay network (which is shit if you actually need to really scale) and 'scaling', which starts n containers of the same service.

Swarm feels more like an mvp which actually works even for small teams. K8s is more like a behemoth which only dedicated personal can fully tame with a silly amount of features most people don't need.

We've used both at my current job for toy projects (self hosted). Never in production however.

And I'm a personal user of gcp - which works wonderfully .. albeit more expensive than simple root servers

Re:Kubernetes Failure Stories by hinkley on Wednesday June 12, 2019 @01:52PM (#20167387)
Possibly edgy opinion:

Load balancing should be a solved problem already. Swarm and Kubernetes should be using dead simple off-the-shelf software for ingress and load balancing. Any competitor should be able to use the same solutions. To put it another way, this shouldn't be a differentiator.

The problem is that the functionality in tools like nginx are still tied to static network architectures that evolve slowly, and don't take advantage of things like diurnal variability in workloads.

Re:Kubernetes Failure Stories by AaronFriel on Wednesday June 12, 2019 @02:43PM (#20167900)
Kubernetes does use dead simple off-the-shelf software for ingress and load balancing. That software though, unfortunately, has a lot of knobs, and what "Ingress" and "Service" resources do is make sure those knobs are turned to the right settings.

The nginx ingress controller for example, under the hood, just generates and applies a big ol' nginx config! You can extract it and view it yourself, or extend it with a Lua script if you want to be fancy and do some FFI inline with processing the request, etc.

Re:Kubernetes Failure Stories by jhanschoo on Thursday June 13, 2019 @02:08AM (#20172219)
> The nginx ingress controller for example, under the hood, just generates and applies a big ol' nginx config!

I learned the hard way that GKE, in using GCP's load balancers don't support the same syntax for Ingress patterns as when you use an nginx Ingress. Definitely read the documentation thoroughly!

Re:Kubernetes Failure Stories by hinkley on Wednesday June 12, 2019 @06:58PM (#20170213)
> That software though, unfortunately, has a lot of knobs

Lots of people have different definitions of 'easy' which is why I didn't say 'easy'. But how did you get this far off the rails with 'dead simple'?

Re:Kubernetes Failure Stories by cpuguy83 on Wednesday June 12, 2019 @07:44PM (#20170501)
Same with swarm, it's just using IPVS at L4. Does not provided anything at L7, it's up to you to provide such a service on the cluster.
Re:Kubernetes Failure Stories by eeZah7Ux on Wednesday June 12, 2019 @05:44PM (#20169654)
That's the opposite of dead simple and k8s tries hard to make things more complex than needed.
Re:Kubernetes Failure Stories by darkarmani on Wednesday June 12, 2019 @02:50PM (#20167990)
Is it hard to integrate a (HA) hardware load-balancer in front instead?
Re:Kubernetes Failure Stories by elsonrodriguez on Wednesday June 12, 2019 @03:27PM (#20168349)
The easy way to do this is with NodePorts, wherein you configure your LB with all the nodes in your cluster being app servers on a certain port for a certain app. However you will lose some performance as there's some iptables magic under the hood.

Beyond that there's a sea of options for more native integrations that will depend on whether your LB vendor has an K8s integration, how friendly your networking team is, and how much code you're willing to write.

Re:Kubernetes Failure Stories by merb on Wednesday June 12, 2019 @04:49PM (#20169164)
for baremetal you can use metallb or kube-router if you have bgp infra. no need for hardware ha lb
Re:Kubernetes Failure Stories by cpuguy83 on Wednesday June 12, 2019 @07:40PM (#20170476)
Swarm comes with overlay networking, you can install network plugins for whatever you need.
Re:Kubernetes Failure Stories by pepemon on Thursday June 13, 2019 @03:20AM (#20172480)
You can't switch networking for Swarm Mode
Re:Kubernetes Failure Stories by cpuguy83 on Thursday June 13, 2019 @09:09AM (#20174438)
Yes you can. The plugin just needs to be swarm capable.

https://docs.docker.com/engine/extend/plugins_network/

Re:Kubernetes Failure Stories by geezerjay on Wednesday June 12, 2019 @01:37PM (#20167208)
Docker Swarm doesn't support basic features such as shared volumes (between separate hosts), though.
Re:Kubernetes Failure Stories by BretFisher on Thursday June 13, 2019 @01:22AM (#20172018)
sure it does, you just need a proper volume driver like rexray AWS EFS
Re:Kubernetes Failure Stories by geezerjay on Thursday June 13, 2019 @02:10AM (#20172221)
The following link states that rexray does not support shared volumes, and that it was designed so that "it forcefully restricts a single volume to only be available to one host at a time."

https://github.com/rexray/rexray/issues/343