While Immutable Infrastructure is also in our opinion (and I've written about this extensively) the way to go in the future updating systems in place is still the primary way to deploy systems and will be for a while. By providing a centralized systems to upload new released and manage the deployment (how many instances get the new deployment in which timeframe) you can take away some of the security problems of opening up ports for access and potential deployment errors where the SSH connection dies.
Especially when deploying into a large infrastructure connecting into each instance for update becomes painful. That's where an agent based services like CodeDeploy is really powerful and removes the single point of failure that is the machine/network that you deploy from.
With ElasticBeanstalk, Opsworks and Cloudformation they now really start to surround all the deployment workflows.
Definitely a great service that will in my opinion become very important to many many teams. You can also read more about our specific integration in our blog: http://blog.codeship.com/aws-codedeploy-codeship/
In the failure case however, even with a fleet of only 20 instances, a rolling update that has issues after the 10th instance puts you in a world of pain.
We use librato for monitoring our build server infrastructure and mostly only look at max/min values for metrics that could mean trouble. Generally we're able to separate data of different instances by their instance id so we could look into them individually.
We use NewRelic for our Rails application on Heroku and pump Heroku data into Librato as well (we love data and metrics)
And of course you can always send me an email to [email protected] with questions.
It's fair to say if you're already doing things in the way you describe, this service isn't for you.
At my org, we currently use a very similar method of deployment that AWS CodeDeploy seems to provide, except that we wrote it in house and use some python fabric scripts to do it. It sounds like AWS CodeDeploy would help us to "AWSify" this method of deployment and also have some nice benefits of health checks/rollbacks, UI based management, and a deployment history log. However, we would essentially still be maintaining/writing bash/python scripts to do the heavy lifting.
However, we are in the process of moving to a method of deployment which uses SaltStack to prebake an AMI with everything that is required to run in production and then use Netflix's Asgard to manage the deployment of the prebuilt AMIs. We are very excited about this method of managing deployments to take advantage of AutoScaling groups and using a well defined/tested stack of tools.
Interested to hear thoughts from others using a deployment approach that they think would benefit from AWS CodeDeploy.
Deploy and rollback doesn't happen on live instances. (Group of) Instances are taken off service for rolling deployment and you can configure what percentage of your fleet should be deployed to at once.
The CodeDeploy service seems to operate by you manually launching base ec2 instance with a code deploy agent and then this agent will checkout your git code on the live instance, run any provisioning steps and then if things break somehow rollback all that work, still on the live instance.
I'm sure this is still a big improvement to companies who are manually sshing into servers and running deployments by hand, but as someone who pre-bakes ami's and does rolling upgrades with autoscaling groups this service seems like a step backwards.