> (Note - here EBS = elastic beanstalk, not elastic block storage)
I, really, really don't recommend using EBS as an abbreviation for Elastic Beanstalk, since Amazon uses the actual abbreviation canonically to refer to Elastic Block Storage. At worst, just call it Beanstalk, when the AWS context is clear.
The AWS abbreviation for Elastic Beanstalk is EB.
Re:Migrating from Heroku to AWS using Docker
by philip1209 on Saturday April 04, 2015 @12:44AM
(#9319935)
We use beanstalkd queue too (SQS gets expensive at scale!), so "beanstalk" is unfortunately also equivocal in our infrastructure. I think it's just bad naming for AWS Elastic Beanstalk overall.
Re:Migrating from Heroku to AWS using Docker
by coenhyde on Saturday April 04, 2015 @12:38AM
(#9319925)
You should do a blog post. I had no idea Elastic Beanstalk allows multiple docker containers per instance. This changes EVERYTHING! I would love to hear how you got this all tied together.
Re:Migrating from Heroku to AWS using Docker
by philip1209 on Saturday April 04, 2015 @12:41AM
(#9319931)
Here's the announcement - it's a little over a week old:
https://aws.amazon.com/about-aws/whats-new/2015/03/aws-elast...
We're using EBS at StaffJoy and in the process of switching to multiple containers per EBS host. This means that a staging environment can have cheap redundancy (e.g. for zero-downtime deploys) by utilizing the same EC2 hosts, but also that queue workers (sporadic, processor-intensive scheduling operations) can split a processor with the more memory-intensive web app.
Getting EBS to work with the private docker registry was a bit of a pain because documentation was lacking - if you're having trouble with this, feel free to ping me. I'm considering a blog post on our deployment system, including easy rollbacks.
The problems we haven't figured out yet on EBS are streaming logs from hosts to another service (e.g. loggly/logentries/papertrail for realtime production error monitoring) and monitoring deploys (specifically to send a Slack notification that changes are fully deployed and to flush the Cloudflare cache).
(Note - here EBS = elastic beanstalk, not elastic block storage)
Edit: A final note - we ended up doing a Jenkins install instead of using CircleCI for build promotion purposes. We build a container, push to the docker registry, then deploy to a stage. We have a separate job that lets us deploy to prod. However, there's no rebuilding - it's just a button that takes a build number and tells EBS to deploy that existing build to the master cluster. For rollbacks - you just enter a prior build number and a prior build immediately starts rolling back We couldn't replicate this stage->master promotion/rollback in CircleCI.
Edit2: We chose to build the image ourself and register with the private registry instead of having AWS build it because we run our unit tests within the built container, so we can guarantee that the image that passes tests is the exact same as the one deployed to production. There was some concern that the image hash could differ - e.g. due to a pip download failure - if AWS built the image separately.