How does autoscaling work if terraform/ansible are involved before kubernetes can take over?
Firstly, kubernetes is able to create multiple instances of your app as load increases and scale them out over all the nodes.
Secondly, the nodes your kubernetes cluster is running on can also autoscale. With terraform you can set up an AWS autoscaling group for example to automatically increase the size of your cluster as load increases.
With a dockerfile is there any point to using Ansible/Puppet/Chef any more? I have used those tools to keep 'how machines are installed' in code in the past, but with dockerfiles these days I don't get the point anymore. Seems like the future will be having a few dockerfiles (staging, prod, dev, db, etc) then just finding a place to run them (digital ocean, docker swarm, aws container service - or your macbook).
Bash is perfectly fine if the machine can be installed with less than 100 lines of code and immutable infrastructure is being used (so no need for idempotent operations). Just make sure to add ShellCheck to the CI pipeline.
Nope, Ansible just needs SSH access.
It is always tough to predict the future; something < 100 lines can easily grow over time. It is nice to start with a tool like Chef/Ansible/Puppet which can more easily grow as the project grows vs Bash.
1. Build an AMI with Packer that contains Ubuntu + Docker + aws-cli.
2. Build and publish docker images for all the services that will run on the instances. (one image per service, not per environment like you suggested)
3. Configure the EC2 instance or AutoScaling Groups with userdata. The userdata usually just contains a `docker run ...` instruction.
I used to have Chef, run test kitchen, bake AMIs, cycle all the machines. Now I have a single AMI and do most of the testing on the docker side and then some on staging. In both cases I am using terraform to manage the overall infrastructure (and userdata scripts).
The only issue is how to manage secrets as userdata are readable with the DescribeInstance ACL. The best solution that I found is to upload the secrets on S3+KMS and use IAM Role on the machine to retrieve them. Or use Hashicorp Vault.
The point of using Docker containers is that you get to deploy to production your dev environment as is.
Cook a base VM image that has the docker engine and anything else you need. Do this once and spin up VMs you need after that. This is what images are for, and they're far easier, faster, more reliable and more secure than dealing with all this other junk.
If anything changes, update the startup script, then run this "make new base image" script and you're all set. Update VMs as needed. Really doesn't get much more complicated than that and using tools like ansible just to setup your VMs so you can run docker containers later is unnecessary.
Unnecessary = not needed to function. Replacing functionality is just preference, which is fine, but specifically not necessary.
My point is that's a loaded question. You could just as easily ask why do you need bash in every VM when you can just have Ansible in a single machine.
I'm curious why you recommend this? I'd argue the opposite. With the declarative nature of Terraform, you can know exactly which part of your infrastructure is going to change before you actually apply it. With Ansible, this information is a bit more opaque. You basically just run the playbook and pray it goes as planned and things haven't drifted too much.
It should go a bit like this:
- Use Terraform to provision VMs, networking resources and storage from DigitalOcean. Basically, write the scripts that make your whole infrastructure available with a single command.
- Then use Ansible for anything you might want installed on those machines - Kubernetes, security packages, SSH keys for your team.
- Use Kubernetes to then deploy your application on top of your secure and replicable infrastructure.
Each of the steps above should be roughly one shell command. If you are disciplined enough to always provision machines, install packages and deploy your app via config files, this should be very much achievable.