So a lot of our effort in moving to kubernetes has required us to try and duplicate all of our previous installation work in ways that don't require root access while the containers are running.
To help, we've chosen to use Helm to manage versioning/upgrading our application in the cluster: https://helm.sh/
Journey to Cloud Native: Breaking the monolith and scaling towards tomorrow - https://docs.google.com/presentation/d/1fsgvSuGpn-MnMqKaTOoi...
Our biggest blockers have been how to separate file system dependency for our various parts. For Git content, we've implemented Gitaly (https://gitlab.com/gitlab-org/gitaly/). For various other parts, we've been implementing support for object storage across the board. Doing this while keeping up with the rapid growth of Kubernetes and Helm has been a challenge, but entirely worth the effort.
We are also bringing all of these changes to GitLab CE, as a part of our Stewardship directive (https://about.gitlab.com/stewardship/, https://about.gitlab.com/2016/01/11/being-a-good-open-source...). We don't feel that our efforts to allow for greater scalability, resiliency, and platform agnosticism belong only in the Enterprise offering, so we're actively ensuring that everyone can benefit, just as they can contribute.
We have a lot of improvements coming as well: https://about.gitlab.com/direction/#ci--cd
While it's not a single button, GitLab is getting closer to that goal with the addition of our cluster integration feature: https://docs.gitlab.com/ee/user/project/clusters/#adding-and...
If you create a trial account on GCP, you can then use GitLab to make a new GKE cluster and it will be automatically linked. You can then click a single button to deploy the Helm Tiller, a GitLab Runner, and then run your CI/CD jobs on your brand new cluster!
The GCP trial is pretty nice, you get $300 in credits and you won't be automatically billed at the end: https://cloud.google.com/free/
What made you eventually go ahead with K8s other than the main obvious reason (the community) itself?
One big reason we've chosen to implement with Kubernetes is community adoption. You can now run Kubernetes on AWS, GCP, Azure, IBM cloud, as well as a slew of other platforms both on-premise and as a PaaS. With the expanding ecosystem, our decision has only been further confirmed, as the list of available solutions and providers continues to grow rapidly (https://kubernetes.io/docs/setup/pick-right-solution/)
Another big reason is far simpler: ease of use and configuration. A permanent fixture in our work for Cloud Native GitLab has been that this method must be as easy, if not easier to provide a scalable, resilient, highly-available deployment of GitLab at scale.
We can’t say, “This is our new suggested method. By the way, it’s harder.”
What we have found is that many other solutions require a much larger initial investment of time to understand and configure GitLab as a whole solution, as compared to the combination of Kubernetes with Helm. Helm provides us templating, default value handling, component enable/disable logic, and many other extremely useful features. These allow us to provide our users with a practical, streamlined method of installation and configuration, without the need to spend countless hours reding documentation, deciding on the architecture, and making edits to YAML.
We've been looking at these technologies for two years now, with our focus being Kubernetes for the last one.
We also try to use the same deployment tools for GitLab.com that we provide to customers, and this lets us offer a scalable production-grade deployment method that can run nearly anywhere.
When its done we expect to be able to scale the different parts of our application independently. This also makes it easier to detect problems (why did Gitaly suddenly autoscale to twice the number of containers).