Re:Kubernetes 1.11 includes In-Cluster Service Load Balancin...
by bryanlarsen on Thursday June 28, 2018 @07:55AM
(#17416097)
- It's certainly possible to run Kubernetes on bare metal. We do it. I recommend kubespray to set things up, there are a bunch of other ways.
The two big things you get on the cloud that you need plugins for on bare metal are networking and storage. Networking plugins are basically fire&forget these days. Storage is definitely not at that stage yet. You can use a plugin that provides redundancy & failover like Rook/cephfs or Heketi/gluster or you can use local storage and handle redundancy/failover similar to how you'd do it without Kubernetes.
- Kubernetes is like any other abstraction. It complicates things and adds another layer to manage and understand, but it lets you manage disparate server apps similarly. So it really only makes sense if you have a good number of disparate server apps to manage.
- If you're looking for 5 nines or something like that, as a small operator you're more likely to screw things up through a PBKAC or misconfiguration than you are via a hardware failure. So you probably want two clusters with manual failover between them. That way you can switch to one cluster while you operate on the other. The second cluster could be a GKE cluster scaled down, but that's still a bunch of complexity and some expense. You're talking about zonal failures, so you'd need two clusters to protect against that anyways, so...
- you probably need several layers of backups. An etcd backup, ark, and your persistent storage backups. Ark handles persistent storage backups on the cloud, but if you're on bare metal, you need to do that yourself the old fashioned way.
The two big things you get on the cloud that you need plugins for on bare metal are networking and storage. Networking plugins are basically fire&forget these days. Storage is definitely not at that stage yet. You can use a plugin that provides redundancy & failover like Rook/cephfs or Heketi/gluster or you can use local storage and handle redundancy/failover similar to how you'd do it without Kubernetes.
- Kubernetes is like any other abstraction. It complicates things and adds another layer to manage and understand, but it lets you manage disparate server apps similarly. So it really only makes sense if you have a good number of disparate server apps to manage.
- If you're looking for 5 nines or something like that, as a small operator you're more likely to screw things up through a PBKAC or misconfiguration than you are via a hardware failure. So you probably want two clusters with manual failover between them. That way you can switch to one cluster while you operate on the other. The second cluster could be a GKE cluster scaled down, but that's still a bunch of complexity and some expense. You're talking about zonal failures, so you'd need two clusters to protect against that anyways, so...
- you probably need several layers of backups. An etcd backup, ark, and your persistent storage backups. Ark handles persistent storage backups on the cloud, but if you're on bare metal, you need to do that yourself the old fashioned way.