As an example, let's say you can deploy your app as containers orchestrated with SystemD on an Amazon EC2 instance. Switching providers will require you to rewrite your whole deployment procedure but not the containerized application code. Your application is cloud agnostic, but not your deployment pipeline.

Now, let's say instead, you have an Helm chart to deploy to k8s, you can move from Google GKE, to Azure AKS, to Amazon EKS, to baremetal k8s, without modifying your Helm chart. Your package is cloud agnostic, but not your infrastructure.

If your application requires a PostgreSQL database, you can provide it with Amazon RDS, or by self-hosting it with KubeDB in a k8s cluster, or any other solutions. To your application, it would be changing the `DATABASE_URL` environment variable and running a migration script from the old DB to the new one. There is no true lock-in in that specific case.

In other words:

  - yes, using Kubernetes can facilitate the deployment of your app on your infra in a "cloud agnostic" way
  - no, using Kubernetes does not mean your whole infra is "cloud agnostic"
  - it's ok if parts of your infra is not "cloud agnostic", that does not mean you're "vendor locked"

https://cluster-api.sigs.k8s.io/user/quick-start.html#initia...

K8s is the closest you are going to get to putting your entire application on a USB stick and plugging it into any computer to run. It is currently the de facto app for modern applications. It’s missing a lot of stuff, but it’s a good way of packaging things and with CRDs you can bolt on the other things you need and make them portable as well.

Whenever the majority of the industry agrees on an abstraction level we are able to make lots short term improvements. I say short term because obviously platform hegemony is often a bad thing, but it frees us up to focus on one area of the stack. So if you can build a great app with k8s and move it between clouds, you have sufficient lock-in protection for a provider. When someone improves on k8s they will create a mostly clean interface so you don’t have to do a ton of things to port your app.

Look at Docker as an example. Everyone thought that was the greatest thing in thr world, but once they have everyone a clean abstraction layer they setup a ton of other container runtimes to slide/shim in and now the runtime layer is being well commoditized. The value for most people and businesses is at the top of the stack.

Here's what makes sense to me: 1. Design your system as components that communicate via interfaces you can implement easily anywhere. In other words, don't design your system to use a proprietary system to communicate. If you can't easily reimplement dynamo or s3, don't make that an inextricable part of your system between your services.

2. Use the cloud services that make implementation easier inside your services. The idea is to reduce the implementation effort/investment as much as you can. Sure, porting to something else could be costly later, but you are saving real effort now. When and if you do need to reimplement on another provider, attempt to do the same thing.

3. If your business doesn't really need or want cloud, don't do cloud. Doing cloud half-way is worse than a well thought out alternative.

Kubernetes is a waste of computer resources if we are speaking about production deployments. What is your container distro? Alpine? Just run your app on alpine! It’s not that complicated. Networking will actually function properly and safely and you don’t need Kubernetes to have declarative infrastructure as code. Addicted to yaml? Ansible uses YAML, can reproduce your build steps and doesn’t leave detritus like an agent nor waste any target resources.

Avoid containers unless you have a specific reason for using them. It’s a silly trend, IMO

Yes, because orchestrating storage volumes, network policies, redundancy, secrets, and containers on a cluster with potentially heterogeneous nodes (some might have a GPU, some might not), is a waste of resource

> you don’t need Kubernetes to have declarative infrastructure as code

True.

> Addicted to yaml? Ansible uses YAML

Yes, because rollbacks in case of failure are so easy to do with Ansible. And who cares about an API that is becoming wide-spread (to not say standard) so you don't have to relearn everything when jumping from project to project?

> your build steps and doesn’t leave detritus like an agent nor waste any target resources

Only if you wrote your playbooks properly. Which is hard.

> Avoid containers unless you have a specific reason for using them. It’s a silly trend, IMO

Yes, because who cares about isolations, reproducibility, and avoiding the "it works on my machine" when integrating your app on different OS with various configurations?

> I fail to see what anti-lock-in benefits either containers or Kubernetes provide.

This sums up your whole unhelpful comment.

Kubernetes will not get you either. You still have cloud specific to stand it up and more.

The short of it is that you cannot be cloud agnostic without significantly more code and complexity. Best to have a few configuration points. Even with Kubernetes, you will still have cloud specific pieces. Without, you can still get many of the benefits with Packer + Terraform.

A good question is what is the purported benefit of going cloud agnostic. If it's to switch clouds, that is not a good one. You won't do it often enough to get an ROI. What is the cost to develop and maintain this posture? (significant)

Run k8s on bare metal with metallb, and something like ranchers longhorn. Then, when you've got that working, move it into the cloud. Run it all on VMs and not cloud-managed k8s. Then you probably still aren't quite there, because IAM and S3 and so on.

In short being cloud agnostic is dumb. Just know that you will have some vendor lock-in.

Pick how much you can tolerate, and live with the rest.

IMHO, it's a stupid constraint. Just pick a provider and go all in.

If not, get the definition from management and skate as close to it as you can.

That's why I'm thinking "cloud agnostic" has to mean something different. I don't see there being a problem with needing to re-work an application a little bit in order to deploy it to another cloud platform. The question is what kind of work would a reasonable person expect "re-work" to entail?

Otherwise I would rather be cloud agnostic when it makes sense. Because the cloud provider you pick will have many bad / low quality / expensive products that you will be forced to use because using another cloud provider is too much efforts. Being cloud agnostic allows you to use the best on the market.

Our level of abstraction is containers, we don't use services that cloud providers sell, for example, Lambda, as they are very cloud provider specific. We prefer to use technologies like RabbitMQ which can be deployed on any cloud or on prems easily.

https://docs.ansible.com/ansible/latest/scenario_guides/kube...

ansible-collections/kubernetes.core: https://github.com/ansible-collections/kubernetes.core

geerlingguy/ansible-for-kubernetes book: https://github.com/geerlingguy/ansible-for-kubernetes

openshift/openshift-ansible: https://github.com/openshift/openshift-ansible

> [OKD] Features: A fully automated distribution of Kubernetes on all major clouds and bare metal, OpenStack, and other virtualization providers;

Single-node OpenShift 4 requires at least 16GB of RAM and 8 vCPU : https://www.redhat.com/en/blog/single-node-openshift-manufac...

> Single node OpenShift offers both control and worker node capabilities in a single server and provides users with a consistent experience across the sites where OpenShift is deployed, regardless of the size of the deployment.

MicroShift requires 2Gb of RAM on e.g. an ARM64 Raspberry Pi 4 or similar SBC: https://microshift.io/docs/getting-started/

Are you clustering the rabbit?

This is probably the minimum. What else does your solution include? Does it include hosting, infrastructure as code is it a managed service, something that clients run themselves or a SaaS?

Generally if you keep deployment simple you don't have to use docker and kubernetes. For example if the app code is delivered as a monolithic .jar file or native executable and you don't have a large number of 3rd party services that need to be deployed.

Docker and Kubernetes do help be cloud agnostic for a particular type of solution architecture - many microservices and potentially many other requirements like queues, caches, 3rd party including open source applications. If you had to create this distributed system on different providers it would be a lot of duplication of effort. In k8s there is still some - you most likely have to configure docker repositories, storage, load balancers, ingresses for different providers but configuration is better than reimplementation. And you still have to manage the cluster. So depending on your architecture and application requirements you may find it much simpler to use Docker and Kubernetes or them to be a needless complication.

There are also certain best practices that help with portability - the 12 factor is a good guide.

For things like Lex, yeah, that's never going to be cleanly cross-platform. Your application architecture can isolate that technology from the rest of your application so you can cleanly port it if you have to.

Appreciate the nod to 12 factor. Thanks!

If you're looking for alternatives, or something lighter weight than Kubernetes, I've used Nomad (plus Terraform and Ansible) and some shell scripts to get repeatable clusters deployed and migrated between cloud providers: https://www.nomadproject.io/

OpenStack is somewhat EC2 API compatible, but the reverse is not true.

What's the reasoning for that requirement, though?

I understand the fear of vendor lock-in, but if AWS decide to put their prices up tomorrow, it's non-trivial to just move everything to Azure (for example). Sure, you have some container images which are portable in theory.. but it's still a large undertaking.

I'd be interested to know if anybody has built with 'cloud agnostic' in mind, and actually needed to migrate to another provider.

Of course, in this case, (unless you install a private cloud), you forgo all the convenience and advantages of a cloud infrastructure (so one can argue that this wouldn't really count as 'cloud agnostic'), but since such on-premise deployments should only be required by big enterprise clients, they should have deep enough pockets to pay for it.

Unless you expect to migrate between cloud providers relatively often, and/or have built your application to be "cloud native" up-front (implying low cost to move to the cloud), it may be cheaper and swifter simply to perform a one-time rewrite and redeployment once you've chosen a desired deployment platform.

However, it is one of the only technologies that pretty much is available on every major cloud provider in the world as a managed service with guarantees of compatibility

https://www.cncf.io/certification/software-conformance/

- Terraform

- Ansible

- Systemd (sometimes Docker)

This way the only thing that changes between providers (e.g., moving my whole infra from digital ocean to linode) are the terraform files.

Do I need a load balancer? That’s nginx on a droplet (terraform apply + ansible-playbook). A database? That’s mysql on a droplet (same commands to deploy and provisioning)

Multiple droplets and a keepalived/haproxy on each one if you don't want a Single Point Of Failure in your infra, being the load balancer.

> A database? That’s mysql on a droplet

And another one for backups, making sure your first droplet has 2 network interfaces, one used by your application, the other one used by your backup procedure so that it does not create too much latency.

Also, potentially multiple droplets if you want your database to be highly available so that you avoid yet another SPOF in your infra.

What you propose is fine for small needs, but it does not scale well unfortunately. I'd still recommend to keep things simple and avoid premature optimization though.

In my case, my production infrastructure runs on a k8s provided by DigitalOcean with one of their load balancer (provided by a k8s Service of type LoadBalancer). I have a droplet running a single-node k8s provided by k0s[0] for testing purposes.

I have a terraform to create the cluster and configure the DNS, then I developed klifter[1] (based on ansible) and klander[2] to provision my cluster via Github Actions.

  [0] - https://k0sproject.io/
  [1] - https://klifter.datapio.co
  [2] - https://klander.datapio.co

https://spinnaker.io/docs/setup/install/providers/