News for hackers. Stuff that matters.
Re:AWS Elastic Beanstalk for Docker by dmourati on Wednesday April 23, 2014 @06:38PM (#7637385)
Help me understand why docker adds any value in prod? When I tried it, I got quickly frustrated with the "expose a single port" to the local OS. Sure, I could go ham on it and move ports around but where is the scale in that? If we are limited to 1 docker/VM seems like we waste more than we gain. Same problem in physical servers.

Not trolling here, seriously interested in where people see value in the real world.

Re:AWS Elastic Beanstalk for Docker by nickstinemates on Wednesday April 23, 2014 @06:49PM (#7637431)
I'd love to spend some time with you if you're seriously interested.

You can expose more than a single port to the local OS, so it sounds like your experience is frought with misunderstanding, which can only be characterized as our fault :)

Reach out - [email protected]

Re:AWS Elastic Beanstalk for Docker by mikebabineau on Wednesday April 23, 2014 @07:52PM (#7637694)
The "one docker container per VM" is a Beanstalk-imposed constraint, not a Docker constraint.

I think it's helpful to frame Docker in terms of encapsulation.

Imagine a basic Java app with multiple dependencies. Rather than build a lean app.jar and rely on the host providing dependency1.jar and dependency2.jar, many folks package it all into a single app_with_dependencies.jar.

But what if your app is a web app with static content? Rather than depending on the host system to serve the static content separately, many folks package it all into a single app_with_content.war.

But what if your app needs a specific version of Java? Rather than depending on the host system to have the exact Java you need, folks are starting to package the app into a Docker image that has their desired Java version.

Each layer of encapsulation simplifies server provisioning and thereby improves app portability. Taken to an extreme, you can treat your hosts as homogenous and run any Docker container from any server.

Re:AWS Elastic Beanstalk for Docker by tszming on Thursday April 24, 2014 @12:30AM (#7638516)
IMO, docker is less useful when you are on a cloud platform like AWS since you already can pack and ship your application as AMI, they even have a marketplace for this: https://aws.amazon.com/marketplace. One of the exceptions I can think of why you really need to run docker inside ec2 is you want to build a PaaS or something like memcached as a service, since it is expensive to give every customer to own a dedicated instance.

Honestly, I would prefer to run as many small/medium instances rather than a powerful instance but to host many dockers inside it, because if you put all the eggs in one basket and when then host machine is down, you are having a bigger problem; if you host a single docker on an ec2 instance, and as I said before, you can already solve this via AMI without the overheads/abstraction.

Of course, it is still very useful to run docker on a physical machine when you don't want to setup KVM or mess up with OpenStack.

Re:AWS Elastic Beanstalk for Docker by benjaminwootton on Thursday April 24, 2014 @01:29AM (#7638643)
Docker is massively more lightweight than shipping AMIs.

- We build hundreds if not thousands of docker images a day after each CI commit. Thats not viable with virtual machines or AMIs.

- Docker has the union file system, so pushes and pulls of Docker images after each CI build are tiny;

- Docker is more platform independent whereas AMIs are very EC2 specific. No other cloud provider have anything as sophisiticated as AMI;

- Rollbacks are instant with Docker;

Docker can also be used in conjunction with autoscaling and resilience. We have 10+ containers on a box, but still replicate this 5 or 6 times and will possibly add EC2 AutoScaling in future. Cloud and Docker are complementary not an alternative in my opinion.

Re:AWS Elastic Beanstalk for Docker by dmourati on Thursday April 24, 2014 @12:43AM (#7638549)
Cool. This analysis is similar to my thoughts. Still trying to get educated and reached out to the docker fellow below for more info.

Big fan of packer.io + AMI / Virtualbox / GCE for this.

See generally: http://www.infoq.com/articles/virtual-panel-immutable-infras...

Re:AWS Elastic Beanstalk for Docker by robszumski on Thursday April 24, 2014 @01:27AM (#7638639)
Sounds like you might be a perfect candidate for CoreOS. fleet allows for single-purpose containers to move around a cluster. It supports all of the platforms that you mentioned (AWS, Openstack, KVM and bare metal).
Re:AWS Elastic Beanstalk for Docker by benjaminwootton on Thursday April 24, 2014 @01:24AM (#7638634)
One docker / VM is not a Docker restriction - more a restriction of this AWS offering.

Was one port exposed a legacy restriction on Docker?

It's not the case now - you can expose many ports to the OS, and then map them as you wish.

This is actually a massive benefit. We might deploy 10 containers to one box all exposing SSH port on 22 and HTTP port on 80, but then at runtime choose which port the OS exposes and map 10 different ports onto 22/80 in the containers.

The real value I find of Docker in production is the deployments - fast, repeatable, easy to rollback. Once they're on a box, it's much of a muchness once you've mapped the ports.

Re:AWS Elastic Beanstalk for Docker by dmourati on Thursday April 24, 2014 @01:48AM (#7638684)
Please explain the port mapping in more detail. For example, let's say I want to run N containers on one host OS. Each of the N containers exposes port 80. How do you do the mapping?
Re:AWS Elastic Beanstalk for Docker by benjaminwootton on Thursday April 24, 2014 @02:06AM (#7638724)
So you have one image exposing port 80.

You create say 3 instances of that image (i.e. 3 containers) with different port mappings 5001:80, 5002:80, 5003:80. These are just specified at the command line (docker run -p5001:80)

Against the same containers you might also do 6001:22, 6002:22, 6003:22 to expose SSHD to different ports on the host.

This allows you to defer the decision of which containers will be listening where to deployment or runtime, giving you lots of flexibility.

Re:AWS Elastic Beanstalk for Docker by dmourati on Thursday April 24, 2014 @02:17AM (#7638749)
Thank you.

OK, how does a client outside of my docker host know to connect to 5001 to get to container1 port 80 and 6001 to get to container1 port 22?

BTW, this was the scenario I mentioned initially as "I could go ham on it and move ports around."

Thanks again for the reply, getting somewhere.

Re:AWS Elastic Beanstalk for Docker by benjaminwootton on Thursday April 24, 2014 @02:44AM (#7638816)
The less mature piece of the puzzle for that is service discovery.

So instead of your apps having hard coded endpoints, they go to some config file, zookeper, etcd to find the services. Your config management piece can register endpoints with the same directories as they come online.

An alternative is just to register with a load balancer dynamically.

This bit does take work but it gives your system a very nice dynamic fungible property.

Happy to discuss more offline with anyone as I'm a massive advocate of docker now! Same username at Twitter.

Re:AWS Elastic Beanstalk for Docker by shykes on Thursday April 24, 2014 @04:34AM (#7639007)
Thanks for the solid explanations Benjamin.

Docker will soon offer a better solution to service discovery. It will allow taking advantage of the various SD tools out there (zookeeper, etcd, synapse/haproxy, skynet, serf) while keeping your containers compatible with all of them. This will help keep all docker containers compatible with each other, instead of fragmenting them among mutually incompatible service discovery protocols.

Happy to chat about it more on irc - #docker and #docker-dev on Freenode.

Re:AWS Elastic Beanstalk for Docker by mikebabineau on Thursday April 24, 2014 @10:43AM (#7640557)
"an alternative is to register with a load balancer dynamically"

To expand on this, a common pattern is to have a local proxy installed on each host machine. To route traffic to a new container, one simply updates the config for the local nginx/haproxy/hipache/other proxy.

A second layer of load balancing exposes the singular, top-level endpoint. All Docker hosts are registered with this LB, but those without live containers for a given service are simply "out of rotation" for that endpoint (health check failure).

This simplifies the service discovery aspect for architectures not already using a more sophisticated mechanism.

Re:AWS Elastic Beanstalk for Docker by sendob on Thursday April 24, 2014 @02:58AM (#7638841)
http://12factor.net/port-binding

"In deployment, a routing layer handles routing requests from a public-facing hostname to the port-bound web processes."

Re:AWS Elastic Beanstalk for Docker by steinnes on Wednesday April 23, 2014 @11:03PM (#7638300)
The "expose a single port" default suggestions always annoyed me a bit, the fact is that each docker gets an IP on a virtual ethernet interface and with the right routes in place can be accessed by anyone on the same network -- so obviously more ports can be exposed :-)
Re:AWS Elastic Beanstalk for Docker by dmourati on Thursday April 24, 2014 @02:21AM (#7638760)
Aha!

This is called advanced networking, yes?

http://docs.docker.io/use/networking/

They didn't have this when I looked last summer.

Re:AWS Elastic Beanstalk for Docker by steinnes on Thursday April 24, 2014 @09:24PM (#7644196)
It's been a few months since I managed to dig into Docker for some test projects at work, but docker0 was the name of the virtual bridge interface so probably the documentation you linked is what we're going on about :-)

I only started playing with Docker last December, and I think this was supported from that point in time, so definitely it could be a recent addition.

Currently I'm envisioning Docker playing a nice role in our CI and development infrastructure at work, where we can choose to start a new docker for any given project, and get a locally accessible IP back in a matter of seconds ... all from our soon-to-be-created CI+deployment web ui ;-)