I don't see benefits to running Docker in AWS. In my opinion, AWS implemented its Docker-based Container Service very poorly. I advise my customers against using AWS when want to use Docker. There are many bare metal as a service providers out in the marketplace.
>the argument about running the same container in dev/test/prod
Is this issue really caused by Docker because you said that you had consistent environments when built by chef?
Instead of making life easy, it just added unnecessary burden of learning Docker for the future project members. Documentation is poor, had to hunt for hours for solutions to simple queries.
On AWS, I would suggest you stick with the basic Linux flavour that you know. Use their Docker build only IF you know docker very well.
We looked into using it earlier this year.
The web UI was flat-out busted in several ways - they only listed the first 100 security groups, and we have a magnitude more than that. The command line interface was poorly documented, and was missing some of the functionality.
It was a total waste of a week; I wonder if they've fixed any of that.
Yes, you can achieve something similar using multiple layers in OpsWorks or different deployment schemes with normal Chef, but IMO containers make provisioning and deploying combinations of components easier than most other provisioning and deployment solutions. There's less opportunity for unexpected version collisions and because the network infrastructure is virtual, you can move containers between underlying VMs, allowing capacity planning without substantial reconfiguration.
I think this is the way Container-as-a-Service to go, instead of the current form of AWS ECS.
I also don't get the argument about running the same container in dev/test/prod. For example my company is working on going Docker and one of the problem with these environments is that app running there has different configuration. So the idea to solve it is to create three different versions of the same container. Genius! But now are you really running the same thing in dev/test/prod? How is it different to what we did in the past? Especially that before Docker through our continuous delivery we actually were using exact same artifact on machines set up with chef that were configured the same way as in prod, while with Docker now we plan to use three different containers.