Re:What we learned after a year on Kubernetes
by marinj on Thursday September 17, 2020 @06:46AM
(#24503038)
> Wrong. Actual analysis by the engineers of one of their migrated jobs (urgent-other sidekiq shard) says their cost went from $290/month when running in VMs to $700/month when running in Kubernetes.
This is only one of the shards of one service that we migrated, and we migrated many more.
if you are curious. I am happy to have a conversation with anyone about our experiences in more detail, not to convince anyone that k8s is the "one-true-way" (because I am still not convinced myself), but of the benefits this type of change can bring.
> They tried to use auto-scaling but failed completely and ended up disabling it
That issue is just one of many though. We disabled it at the time, but we have it enabled for a number of other services.
Regardless of how I personally feel about K8s, I have to say that the migration we are doing for GitLab.com is generating set of benefits that goes far beyond just moving to a new platform.
One of the largest benefits I've seen so far is that it was a great forcing function to resolve some long running architectural challenges, and is making us think more about how the application can run at a very large scale, without being at the very large scale. Things that we could get away before, like the issue you referenced there, we can't anymore.
Disclaimer: I am one of the people involved in this migration.
This is only one of the shards of one service that we migrated, and we migrated many more.
I did a high level writeup of this and many other things we've observed in https://about.gitlab.com/handbook/engineering/infrastructure...
if you are curious. I am happy to have a conversation with anyone about our experiences in more detail, not to convince anyone that k8s is the "one-true-way" (because I am still not convinced myself), but of the benefits this type of change can bring.
> They tried to use auto-scaling but failed completely and ended up disabling it
That issue is just one of many though. We disabled it at the time, but we have it enabled for a number of other services.
Regardless of how I personally feel about K8s, I have to say that the migration we are doing for GitLab.com is generating set of benefits that goes far beyond just moving to a new platform.
One of the largest benefits I've seen so far is that it was a great forcing function to resolve some long running architectural challenges, and is making us think more about how the application can run at a very large scale, without being at the very large scale. Things that we could get away before, like the issue you referenced there, we can't anymore.
Disclaimer: I am one of the people involved in this migration.