News for hackers. Stuff that matters.
Re:Setting up an HA Kubernetes cluster with private networki... by sandGorgon on Wednesday December 07, 2016 @06:54AM (#13121921)
kops is nice... but too tied to AWS.

I'm really liking the direction that kargo is taking. It leverages a well known toolset - ansible - to build out its functionality.

kargo is compatible with everything from bare metal to AWS... with the caveat that when using it in AWS, you have to use a different provisioning tool like Terraform.

This is a feature.. not a bug.

Re:Setting up an HA Kubernetes cluster with private networki... by justinsb on Wednesday December 07, 2016 @08:04AM (#13122206)
Kops actually isn't tied to AWS, though we only support AWS right now. We'll likely be adding GCE support next; user requested features on AWS have taken priority so far though...
Re:Setting up an HA Kubernetes cluster with private networki... by sandGorgon on Wednesday December 07, 2016 @08:42AM (#13122417)
@justinsb - oh I didnt know that. For example, we are struggling to use kops on bare metal. However, I quite understand your AWS feature priority.
Re:Setting up an HA Kubernetes cluster with private networki... by justinsb on Wednesday December 07, 2016 @08:49AM (#13122456)
So right from the start I wanted to make sure it wasn't locked to AWS. We used to have GCE support enabled (and continue to have it on a branch, though obviously it has bit-rotted as we've added more AWS feature support) But we built GCE support early so we could be sure that the abstraction wasn't AWS specific. And we have bare metal support on another (more experimental) branch. Bare metal is much more challenging, largely because on clouds we can run etcd in HA mode with automatic instance replacement, which isn't really possible on bare-metal, so it looks we have to settle for HA with operator intervention when an etcd instance fails.

So I actually think kargo is a great choice on bare-metal - and it also does a great job of running on multiple clouds and bare metal. But the design of kops isn't particularly tied to AWS, though it does match better with clouds.

Re:Setting up an HA Kubernetes cluster with private networki... by marcoceppi on Wednesday December 07, 2016 @10:48AM (#13123362)
If you're looking for bare metal, have you tried using MAAS and Juju? https://jujucharms.com/kubernetes-core/
Re:Setting up an HA Kubernetes cluster with private networki... by hackerboos on Wednesday December 07, 2016 @09:04AM (#13122552)
I don't know why projects like this don't use libcloud which allows you to use the many cloud platforms [1] through one API.

[1] https://libcloud.readthedocs.io/en/latest/compute/supported_...

Re:Setting up an HA Kubernetes cluster with private networki... by justinsb on Wednesday December 07, 2016 @09:38AM (#13122780)
Well I can't answer for other projects, but libcloud specifically is in Python, and kops is in golang. But you raise a valid question: why not use a cloud abstraction library like libcloud - for example embedding terraform. It would definitely short-cut things, but the trickier piece IMO is figuring out what cloud resources to create, rather than making the API calls. Embedding terraform might save a little time up front, but would also make it harder where terraform didn't work quite the way we wanted it to (the state store), or even if we hit any bugs. The Kubernetes experience with Salt was not overly positive in this regard (though this was likely more because of how we were using it than Salt itself).

It is possible to build a kops cloud implementation that doesn't do direct-to-cloud-API-calls, but instead just outputs terraform - that would be faster - but you would still have to figure out the right way to run Kubernetes on cloud X. Most of the work is in doing that!

Re:Setting up an HA Kubernetes cluster with private networki... by hackerboos on Wednesday December 07, 2016 @10:48AM (#13123364)
I understand your point - just the vendor lock-in of AWS/GCP can be frustrating if you do not host with them.

There are also golang alternatives to libcloud one of them ->https://github.com/apcera/libretto

Re:Setting up an HA Kubernetes cluster with private networki... by grkvlt on Wednesday December 07, 2016 @04:32PM (#13126417)
This is basically what we do at Cloudsoft. For the Kubernetes cluster deployment in Clocker [1] we use the features of Apache jclouds [2] and Brooklyn to support pretty much any cloud, public or private. At the moment, Clocker can deploy a Kubernetes cluster with an HAProxy load-balanced set of masters and an auto-scalable cluster of workers, using Canal (Flannel and Calico) [3] for networking.

1. https://github.com/brooklyncentral/clocker 2. http://jclouds.apache.org/ 3. https://github.com/tigera/canal