Re:Container technologies at Coinbase: Why Kubernetes is not...
by nihil75 on Monday June 08, 2020 @07:26PM
(#23461903)
Odin is just Step Function flow that deploys Auto Scaling groups. It's not that complicated, and not so different from your setup.. main difference is they deploy docker images to ec2 rather than run ansible and configure a running machine.
Re:Container technologies at Coinbase: Why Kubernetes is not...
by kryptk on Monday June 08, 2020 @09:32PM
(#23462720)
What about resource allocation? With Kubernetes I can specify both requests and limits of cpu and memory and it will fill my nodes to match. A node autoscaler gets me more nodes if needed. I can define vertical pod autoscaler that can dynamically modify those requests and get me an emptier or bigger node if needed. I can define a horizontal pod autoscaler to keep aggregate cpu at a set target by auto-spawning more containers for me, and k8s handles load balancing via dns. Do typical production setups not require some/most of these features? Mine do.
- container orchestration platform is Odin + AWS ASGs (auto-scaling groups).
- Codeflow (our internal UI for deployments)
- Odin kicks off a step function and begins to deploy your application. New VMs are stood up in AWS and loaded into a new ASG, your software is fetched from various internal locations, a load balancer starts health-checking these new instances, and eventually traffic is cut over in a Blue/Green manner to the new hosts in the new ASG behind the load balancer.
- To handle secrets and configuration management we have built a dynamic configuration service that provides libraries to all internal customers with a p95 of 6m
- re-scheduling/moving of your containers if your VM dies/becomes unhealthy in your ASG
So if your company has all of that, I agree no reason to use Kubernetes. But what if your company doesn't have any of above system? Kubernetes.
I have used bare ec2 vm, deploy with Ansible, Capistrano, Chef, Docker..all of them. I even roll my own autoscaling with Consul, SQS(for termination notice) and I have more downtime than when using Kubernetes.
With Kubernetes, the learning curve is very high, but once you get it. It's painless to bring in new service, with AWS ACM to terminate TLS, and an ingress controller, pretty much as long as you have a Dockerfile, you can run it.