I think you might have misunderstood that page. The standard and universal way to deploy Kubernetes on to either your own bare metal or any cloud provider is to use kubeadm. However, if you would like a simpler and more automated solution and/or one backed by a vendor, you are welcome to pick any of the hosted platforms, distributions, or installers. CNCF has certified 70 conformant solutions: https://www.cncf.io/certification/software-conformance/
> Even if I never moved out of the big 3 (AWS/Azure/GCloud), the migration process could be painful since their Kubernetes may introduce further lock-ins for logging, monitoring, and so on.
If you choose open source solutions for logging and monitoring like Fluentd and Prometheus, then you can avoid locking into anyone's value added services and remain completely portable. If you decide to go with a vendor's solution, you may trade convenience for higher switching costs.
[1]: https://kubernetes.io/docs/setup/pick-right-solution/
Disclosure: I'm executive director of CNCF and run the conformance program.
Take AWS EKS as an example. Their feature page[1] does mention conformance. Then it mentions 20 other non-conformance focused features that create an effective lock-in.
k8s is becoming like OpenStack in this regards. You need to embrace a vendor version of k8s in order to have a functional cluster without a massive team.
But using rancher 2.0 has helped a bunch to ease me into it. Now I feel comfortable enough to start up a cluster on my own without it.
Longhorn is pretty easy to set up, but it seems to have some issues actually working (and slow).
I'm also trying out rook/ceph at the moment.
Currently the cluster I'm talking about is being heavily used for QA testing purposes and Beta testing of applications. Kubernetes and a small operator I've written allows us to dynamically create QA servers for issues as developers finish their changes. Beta testing is easily deployed by a helm chart I wrote up.
Next we'll start up the production cluster and use it for internal applications and finally we'll be moving over our customer facing applications.
If someone could point me to an article explaining why k8s is so much better than swarm, I'd really appreciate it. Are the big advantages only at 100-node scales?
I've recently started using Kompose to autogenerate Helm charts from docker compose files and I've found that pretty satisfactory.
That said, I was at a conference, talking with a CIO of a K8s related company, back during the initial days of that startup ecosystem. I was told off the record that Google and Redhat were offering an "marketing budget" for companies that would come on board the Kubernetes ecosystem. The person rightly stated that K8s was going to roll everyone else because of it.
Can't validate that - it's hearsay, but I definitely think that it was a big factor
It's quite simple for a 20 year SA to stand up a highly integrated environment with modular monitoring, directory service, virtualization and hybrid cloud options for all services in a week. Why don't you hire one of these for the job instead of recipe/containering yourself into 'doesn't work, I dunno' posts.
Because every.single.one of these "integrated environments" I've ever come across was an objective mess, poorly documented and littered with tech-debt.
It was clear the "20-year SA" had forced 20-year old administration abstractions and ideas on top of modern infrastructure and application concerns. It was cheaper/better/easier to throw it out and rebuild on something like k8s than to make any attempting at "scaling" the existing solution.
You're simply trading the "in-fashion" disease for the "I'm a 20-year Linuxbeard I know best and no one tells me different" disease.
For your information I find the new breed and k8s FUD to be more of a reaction rather than an educated, seamless and practical systems approach.
Why is it better to spend money to rebuild a fraction of K8S with a patchwork of infrastructure put together by a single person?
K8S can replace quite a lot of sysadmin responsibilities and is currently doing so at thousands of companies.
If you have ever had to talk down execs bitten by the k8s hype in a scientific compute or HPC environment you would know that the marketing and sales talk is both prevalent and damaging.
The kubernetes folks describe a tentative solution to cloud lock-in here: https://kubernetes.io/docs/concepts/cluster-administration/f... OP isn't the only one with those concerns.
It would be nice when you can switch your cluster load from any of the cloud providers, or your own on-prem setup as you go. For instance, I could see people wanting to have a default small cluster on their on-prem setup, and be ready to scale on cloud when needed.
There are two points of lock-in: beneath Kubernetes or on top of Kubernetes.
You still need to install and manage your cluster, so the low level lock-in of a distro is hard to avoid. What you may be looking for is kubeadm. https://kubernetes.io/docs/reference/setup-tools/kubeadm/kub...
It is the closest primitive to help standardize “here’s how to install and run a K8s cluster in a certified way”. It’s unfortunately incomplete.
As for what’s on top of Kubernetes, that will be an area ripe for completion and lock-in.
It used to be really clunky, but these days all you need is a simple bash script or ansible play (or whatever you’re comfortable with) to get going.
But yeah, no unix philosphy vibes from k8s as a whole...
Actually, I barely saw any Kubernetes cloud provider provides meaningful service which can lock myself in, they are basically managed Kubernetes clusters with their cloud services as plugins. You can verify this by comparing GKE/AKE/EKS, you'll find they are almost same thing.
Red Hat OpenShift on RHEL, Pivotal Container Service on Ubuntu, Red Hat’s nextgen CoreOS based Kubernetes, Canonical’s Charmed Kubernetes Distribution on Ubuntu, etc. all have different config management , install, upgrade, patching mechanisms that vary from Ansible, to Terraform, to BOSH, to Juju. Some handle PXE bare metal, some don’t. Etc.
There usually are free / no pay versions of the above that you can use self-supported, but then you’ll also need to coordinate your own upgrades and use community forums for q&a rather than being able to contractually have someone looking out for you and answering your questions.
If you’d prefer to avoid lock-in, All of that plumbing would otherwise have to be configured and scripted yourself with your chosen toolchain plus the newer “k8s small tools” like Kubeadm, Kops, Kube-spray, etc.
As the old saying goes, open source is only free (as in beer) if your time has no value.
Yes but in the case of RedHat specifically those goals are not achieved.
I am a pretty basic user, I have started using k8s on this project as a learning and 100$ was too much for the learning price, but now on DO I get a similar cluster for less than half of GKE price and I feel like it is worth it, considering all the simplicity and observability of deployments. Also, DO allows me to select regions without any price difference, so I was able to select Amsterdam to get 10 times better latency from where I live. My setup is quite basic, my app with aroud 8-10 pods, + additional stuff such as cert-manager and prometheus.
YMMV, but so far I am really happy with DO's offering, both in terms of performance, simplicity and performance. I am not a power user and definitely operate at no scale, but using DO in general is much simpler than using GCP with GKE.
First, the biggest problem I see is the huge vendor lock-in you accept with the PaaS offerings such as AWS EBS or GCP App Engine. When you commit to one of these platforms, it is really hard to get out of it; it requires engineering effort to move to another provider and feature parity between the providers for your application to be supported. Plus, you get to learn platform-specific stuff which has no standards across providers. Plus, it is usually slow and bloated; have you ever tried deploying something to EBS? It takes at least five minutes without any meaningful information about what is going on or if your deployment succeeded.
Second, the tooling you get is usually very small compared to what Kubernetes ecosystem has. Each and every platform ask you to use their own tools, but there is a high possibility that the tools don't fit your usecase, or you may need to modify your workflow. With a solution like k8s, you only need to support the standard, which is k8s itself roughly, and you are free to use whatever tooling you want.
Third, done right, Kubernetes allows you to move to another provider very easily without changing a single line of code in your Kubernetes definitions or your application. You define the desired state of your cluster, you check in all these stuff into your VCS, and since k8s forces you to do these stuff from the beginning, at the end you usually have a nice, reproducible system that is more or less cloud agnostic. You have logging, horizontal scalability, isolation, easy deployments, easy rollbacks and all that stuff. I have migrated from GKE to DO's Kubernetes offering without changing a single line in my Kubernetes definitions or my application. Of course, my usecase is very very very small compared to most of people around here, but that was my experience.
FWIW, I think Kubernetes is still a good learning to understand about current state of infrastructure, deployments and the ideal state we all try to achieve. Whether or not a business should depend on it is a whole another topic.
There's no strong vendor lock-in either, buildpacks and backend services are much of a muchness across Dokku, Herokuish, Flynn, Cloud Foundry etc. If your app is 12-factor with externalised state, you're plain sailing with most PaaS and simple docker setups, or at least I don't get what K8s brings to the table in terms of operational simplicity.
When you say dashboard, which dashboard do you mean? Happy to look at it. Thanks again for the feedback.
The first disappointment is setting up a local development environment. I failed to get minikube running on a Macbook Air 2013 and a Ubuntu Thinkpad. Both have VTx enabled and Docker and VirtualBox running flawlessly. Their online interactive tutorial was good though, enough for the learning purpose.
Production setup is a bigger disappointment. The only easy and reliable ways to have a production grade Kubernetes cluster are to lock yourself into either a big player cloud provider, or an enterprise OS (Redhat/Ubuntu), or introduce a new layer on top of Kubernetes [1]. Locking myself into enterprise Ubuntu/Redhad is expensive, and I'm not comfortable with adding a new, moving, unreliable layer on top of Kubernestes which is built on top of Docker. One thing I like about the Docker movement is that they commoditize infrastructure and reduce lock-ins. I can design my infrastructure so it can utilize an open source based cloud product first and easily move to others or self-host if needed. With Kubernetes, things are going the other way. Even if I never moved out of the big 3 (AWS/Azure/GCloud), the migration process could be painful since their Kubernetes may introduce further lock-ins for logging, monitoring, and so on.
[1]: https://kubernetes.io/docs/setup/pick-right-solution/