News for hackers. Stuff that matters.
Re:Setting up an HA Kubernetes cluster with private networki... by jondubois on Wednesday December 07, 2016 @05:41AM (#13121673)
I wrote a similar article a while ago; except using Rancher - Which adds extra features for infrastructure management. https://blog.baasil.io/how-to-install-kubernetes-on-aws-d9fb...
Re:Setting up an HA Kubernetes cluster with private networki... by whalesalad on Wednesday December 07, 2016 @09:38AM (#13122775)
Good lord this is so much easier than it was 6 months ago. Really glad to see the barrier of entry going down to get these clusters built. Kubernetes has been paramount to our engineering growth at FarmLogs.
Re:Setting up an HA Kubernetes cluster with private networki... by chrislovecnm on Wednesday December 07, 2016 @10:33AM (#13123225)
Yah I agree. K8s install has been the bane of the projects existence.
Re:Setting up an HA Kubernetes cluster with private networki... by kris-nova on Wednesday December 07, 2016 @03:44PM (#13126041)
Happy to know we are finally "here" with deploys/installs!
Re:Setting up an HA Kubernetes cluster with private networki... by iddqd on Wednesday December 07, 2016 @04:10AM (#13121378)
kops is pretty great for managing the lifecycle of a k8s cluster. It even supports outputting Terraform code, if that is your Infrastructure-as-Code tool of choice.
Re:Setting up an HA Kubernetes cluster with private networki... by kris-nova on Wednesday December 07, 2016 @04:15AM (#13121393)
One of the best features of kops, is it's ability to be open ended in generating what the user wants! The private networking piece of Terraform codegen is right around the corner, and rumor has it, it will be used in some new API tests
Re:Setting up an HA Kubernetes cluster with private networki... by alexbilbie on Wednesday December 07, 2016 @04:41AM (#13121469)
ECS tasks now support IAM roles; how does this work for K8s equivalent? Do you need to use IAM user credentials for app credentials?
Re:Setting up an HA Kubernetes cluster with private networki... by tazjin on Wednesday December 07, 2016 @06:13AM (#13121762)
IAM credentials for ECS tasks are implemented with a transparent proxy to the metadata API, and there are similar projects for Kubernetes.

The one that's most popular I believe is kube2iam: https://github.com/jtblin/kube2iam

Using IAM roles then boils down to setting an annotation such as `iam.amazonaws.com/role: my-db-access-role` on a pod (assuming the correct trust relationship has been configured).

Re:Setting up an HA Kubernetes cluster with private networki... by sandGorgon on Wednesday December 07, 2016 @06:54AM (#13121921)
kops is nice... but too tied to AWS.

I'm really liking the direction that kargo is taking. It leverages a well known toolset - ansible - to build out its functionality.

kargo is compatible with everything from bare metal to AWS... with the caveat that when using it in AWS, you have to use a different provisioning tool like Terraform.

This is a feature.. not a bug.

Re:Setting up an HA Kubernetes cluster with private networki... by justinsb on Wednesday December 07, 2016 @08:04AM (#13122206)
Kops actually isn't tied to AWS, though we only support AWS right now. We'll likely be adding GCE support next; user requested features on AWS have taken priority so far though...
Re:Setting up an HA Kubernetes cluster with private networki... by sandGorgon on Wednesday December 07, 2016 @08:42AM (#13122417)
@justinsb - oh I didnt know that. For example, we are struggling to use kops on bare metal. However, I quite understand your AWS feature priority.
Re:Setting up an HA Kubernetes cluster with private networki... by justinsb on Wednesday December 07, 2016 @08:49AM (#13122456)
So right from the start I wanted to make sure it wasn't locked to AWS. We used to have GCE support enabled (and continue to have it on a branch, though obviously it has bit-rotted as we've added more AWS feature support) But we built GCE support early so we could be sure that the abstraction wasn't AWS specific. And we have bare metal support on another (more experimental) branch. Bare metal is much more challenging, largely because on clouds we can run etcd in HA mode with automatic instance replacement, which isn't really possible on bare-metal, so it looks we have to settle for HA with operator intervention when an etcd instance fails.

So I actually think kargo is a great choice on bare-metal - and it also does a great job of running on multiple clouds and bare metal. But the design of kops isn't particularly tied to AWS, though it does match better with clouds.

Re:Setting up an HA Kubernetes cluster with private networki... by marcoceppi on Wednesday December 07, 2016 @10:48AM (#13123362)
If you're looking for bare metal, have you tried using MAAS and Juju? https://jujucharms.com/kubernetes-core/
Re:Setting up an HA Kubernetes cluster with private networki... by hackerboos on Wednesday December 07, 2016 @09:04AM (#13122552)
I don't know why projects like this don't use libcloud which allows you to use the many cloud platforms [1] through one API.

[1] https://libcloud.readthedocs.io/en/latest/compute/supported_...

Re:Setting up an HA Kubernetes cluster with private networki... by justinsb on Wednesday December 07, 2016 @09:38AM (#13122780)
Well I can't answer for other projects, but libcloud specifically is in Python, and kops is in golang. But you raise a valid question: why not use a cloud abstraction library like libcloud - for example embedding terraform. It would definitely short-cut things, but the trickier piece IMO is figuring out what cloud resources to create, rather than making the API calls. Embedding terraform might save a little time up front, but would also make it harder where terraform didn't work quite the way we wanted it to (the state store), or even if we hit any bugs. The Kubernetes experience with Salt was not overly positive in this regard (though this was likely more because of how we were using it than Salt itself).

It is possible to build a kops cloud implementation that doesn't do direct-to-cloud-API-calls, but instead just outputs terraform - that would be faster - but you would still have to figure out the right way to run Kubernetes on cloud X. Most of the work is in doing that!

Re:Setting up an HA Kubernetes cluster with private networki... by hackerboos on Wednesday December 07, 2016 @10:48AM (#13123364)
I understand your point - just the vendor lock-in of AWS/GCP can be frustrating if you do not host with them.

There are also golang alternatives to libcloud one of them ->https://github.com/apcera/libretto

Re:Setting up an HA Kubernetes cluster with private networki... by grkvlt on Wednesday December 07, 2016 @04:32PM (#13126417)
This is basically what we do at Cloudsoft. For the Kubernetes cluster deployment in Clocker [1] we use the features of Apache jclouds [2] and Brooklyn to support pretty much any cloud, public or private. At the moment, Clocker can deploy a Kubernetes cluster with an HAProxy load-balanced set of masters and an auto-scalable cluster of workers, using Canal (Flannel and Calico) [3] for networking.

1. https://github.com/brooklyncentral/clocker 2. http://jclouds.apache.org/ 3. https://github.com/tigera/canal

Re:Setting up an HA Kubernetes cluster with private networki... by marcoceppi on Wednesday December 07, 2016 @10:47AM (#13123353)
This is a nice write up, but we honestly needed to stand up kubernetes on-premise and in public/private clouds. This is why we've been using this: https://www.youtube.com/watch?v=B7nMFVaOOi8