Kubernetes Failure Stories

Re:Kubernetes Failure Stories by dankohn1 on Sunday January 20, 2019 @02:40PM (#18955112)

> The only easy and reliable ways to have a production grade Kubernetes cluster are to lock yourself into either a big player cloud provider, or an enterprise OS (Redhat/Ubuntu), or introduce a new layer on top of Kubernetes [1]

I think you might have misunderstood that page. The standard and universal way to deploy Kubernetes on to either your own bare metal or any cloud provider is to use kubeadm. However, if you would like a simpler and more automated solution and/or one backed by a vendor, you are welcome to pick any of the hosted platforms, distributions, or installers. CNCF has certified 70 conformant solutions: https://www.cncf.io/certification/software-conformance/

> Even if I never moved out of the big 3 (AWS/Azure/GCloud), the migration process could be painful since their Kubernetes may introduce further lock-ins for logging, monitoring, and so on.

If you choose open source solutions for logging and monitoring like Fluentd and Prometheus, then you can avoid locking into anyone's value added services and remain completely portable. If you decide to go with a vendor's solution, you may trade convenience for higher switching costs.

[1]: https://kubernetes.io/docs/setup/pick-right-solution/

Disclosure: I'm executive director of CNCF and run the conformance program.

[ Parent | on HN ]

Re:Kubernetes Failure Stories by pquerna on Sunday January 20, 2019 @05:52PM (#18956128)

The point is not about the minimum conformance, but rather the lock-in provided by the maximum configuration / extensions of each vendor.

Take AWS EKS as an example. Their feature page[1] does mention conformance. Then it mentions 20 other non-conformance focused features that create an effective lock-in.

k8s is becoming like OpenStack in this regards. You need to embrace a vendor version of k8s in order to have a functional cluster without a massive team.

[1] - https://aws.amazon.com/eks/features/

[ Parent | on HN ]

Re:Kubernetes Failure Stories by devthane on Sunday January 20, 2019 @06:51PM (#18956396)

This isn’t my experience at all. I as one person taught myself over the past couple years docker then Kubernetes and am now managing a small 3 node bare metal cluster on my own.

But using rancher 2.0 has helped a bunch to ease me into it. Now I feel comfortable enough to start up a cluster on my own without it.

[ Parent | on HN ]

Re:Kubernetes Failure Stories by Aeolun on Monday January 21, 2019 @12:55AM (#18957981)

I strongly feel that Rancher 2.0 is just not ready for release yet. I run into so many weird edge cases using it, and don’t get me started on persistent storage...

Longhorn is pretty easy to set up, but it seems to have some issues actually working (and slow).

[ Parent | on HN ]

Re:Kubernetes Failure Stories by devthane on Monday January 21, 2019 @09:41AM (#18960633)

I've had issues with longhorn as well, though the problem I was having (volume attachment race condition) is supposedly resolved by upgrading kubernetes.

I'm also trying out rook/ceph at the moment.

[ Parent | on HN ]

Re:Kubernetes Failure Stories by qlk1123 on Sunday January 20, 2019 @09:35PM (#18957178)

Just curious, is your cluster used in production or for any serious purpose?

[ Parent | on HN ]

Re:Kubernetes Failure Stories by devthane on Monday January 21, 2019 @09:48AM (#18960686)

Not for production yet. I'm in the process of moving our infrastructure from VMs that no one was really managing to Kubernetes. We're about to put together the production cluster and move over a couple of the applications that are fully ready and have been tested on the development cluster for a while.

Currently the cluster I'm talking about is being heavily used for QA testing purposes and Beta testing of applications. Kubernetes and a small operator I've written allows us to dynamically create QA servers for issues as developers finish their changes. Beta testing is easily deployed by a helm chart I wrote up.

Next we'll start up the production cluster and use it for internal applications and finally we'll be moving over our customer facing applications.

[ Parent | on HN ]