One thing about this getting started guide is that it recommends the Phusion base image which boots init. That seems to go against the best practices outlined in a recent article by Michael Crosby - http://crosbymichael.com/dockerfile-best-practices-take-2.ht...
Phusion's baseimage does not go against Michael Crosby's best practices. His best practices states not to boot init, and with that he means the normal init. He states that not because it's the init process itself that's a bad idea, but because the normal init performs all kinds of work that is either unnecessary or harmful inside a Docker container. In fact, this is exactly what the Baseimage-docker documentation also states: don't use the normal init.
The Phusion baseimage does not contain a normal init, but a special init that is specifically designed for use in Docker.
pg_fukd_mydog
Pointed out that FreeBSD jails do this right.
Look at "fleet" from CoreOS, and especially their "sidekick" example that uses systemd dependencies to trigger etcd updates: https://coreos.com/docs/launching-containers/launching/launc... though you can certainly do this without fleet too.
Then on the haproxy/varnish box (or put them in a container), put something that does "etcdctl exec-watch /services/website -- updateconfig.sh", where updateconfig.sh would be a script to watch for changes and regenerate the config / reload.
I don't see how your config will get "bloated" any more than it would otherwise - presumably your number of domains won't increase.
Right now, if you use SkyDNS[1] as your DNS server, and attach Skydock to the Docker host, this all Just Works. Most people want to use etcd instead of Skydock, though, so support for that is coming soon[1] too.
[1] https://github.com/skynetservices/skydns
[2] https://groups.google.com/forum/#!topic/coreos-dev/iklEYHh5J...
Am I mistaken?
Better to have a healthcheck service doing healthchecking, and modify SkyDNS along with whatever else happens when a service goes down.
It might be a stupid question but I wonder what's considered a leaky abstraction in this case.
By the way, I'm not sure I fully understand your concerns over reverse proxy routing, but I recall that Ambassador pattern linking[0] is a suggested way of tying Docker containers over network. Also, these slides by dotCloud[1] may be helpful as well (I'm not sure if approaches described are up-to-date, though).
[0] http://docs.docker.io/en/latest/use/ambassador_pattern_linki...
[1] http://www.slideshare.net/dotCloud/deploying-containers-and-...
I consider poking a hole a leaky abstraction because you are exposing the internals of your stack. The consumer should not know or care that you are using docker containers to serve the application. From a security perspective directly exposing a container may lead to potential exploits of docker itself.
Exposing a container is a hell of a lot safer than exposing a service on the host OS.
Also not sure I follow you how a consumer would know you are using docker or not.
I have a box sitting somewhere which, like virtually any dedicated machine, is wildly overprovisioned for it's current usage patterns.
I would like to virtualize my services so that I can one day, when my needs outgrow my box, scale out without having to rewrite any code.
My box has limited IPs available, so I'll need the network between services to be private/internal.
How do I set that up with Docker?
I think it won't be until you can truly easily answer that question that Docker will really take off.
Could you not run multiple docker containers/services behind a single nginx or apache container on a production server? Then the nginx container basically gets one of your public IP addresses, and you use linking to that container to provide it with knowledge of the other running processes' IP addresses (each within their own container, of course). In that way, you have one public facing container which has knowledge of the other containers and can use the information provided through -link to configure the nginx server to route requests appropriately. This requires a bit of bash script / sed command line hackery to update your nginx configuration to accommodate the changing IP addresses of the other containers on restart (unless you can set them by hand now using Docker, we still don't), but once you get it setup you never have to think about it again.
Like I said, maybe I'm just showing my ignorance, but something like the above scenario is how we get around hosting multiple services with limited public IP addresses available.
CoreOS' "fleet" ( https://coreos.com/docs/launching-containers/launching/launc... ) gives a cluster-wide solution.
But even without using fleet, the overall mechanism is fairly easy to adapt: Either use systemd dependencies like in their example, or have a script that queries docker on each host to spot changes in running containers, and update an etcd instance (or whichever your preferred config server is).
Your services need to read something to get the IP, which ultimately comes from an ENV variable. In the linked container scenario Docker sets that variable. Otherwise you set it manually. That's the only extra complexity.
I was worried about this too, so I tried it out[1]. In this case I have a YAML config file, which can be overridden by ENV variables (which may come from Docker).
This isn't as automatic as CoreOS (eg, no failover etc), but it is a lot less complex.
[1] https://github.com/nlothian/Acuitra/blob/master/services/que...
Most (all?) of the available reverse proxies will stop sending traffic to a server that is offline, but not discover them. There are solutions such as etcd which you can hook into, or you can write a toy application to use UDP-broadcasts to advertise "Hey I'm http://dev.local.com/ on port 4444", but there isn't a lot beyond that.
Templating configuration files and running "haproxy reload" is a common enough middle-ground, but I've seen it fail often. (Specifically keepalived not reloading correctly and still sending traffic to old nodes.)
ObRelated: Varnish is a beast that few people can configure easily. I'd love to work on a caching reverse proxy that was simple, extensible, and fast.
It doesn't have as many caching-specific bells and whistles as Varnish, but nginx is an excellent reverse proxy with some caching abilities (and simple configuration).
That means you can't expire the cached content by URL.
It seems odd for nginx to try to commercialise such basic parts of the stack where 3rd parties can easily write such functionality.
If you are hosting data for several users on the same nginx cache and you want to purge only one of them, your only options are to scan the full cache on disk and delete the files that have a key with your prefix, or fork >$1K/year per nginx box for the commercial license.
Varnish at least can route using DNS [0] - You do need a nameserver or two to handle the internal domain of course, but they're reasonably easy to set up using powerdns for example.
[0] https://www.varnish-cache.org/docs/3.0/reference/vcl.html#th...
http://jasonwilder.com/blog/2014/03/25/automated-nginx-rever...
Definitely an interesting issue. Have you seen etcd from CoreOS? Useful for service discovery.
Once you have the reverse proxy setup the next problem that arises is routing to containers based on the domain. Now your HAProxy or Varnish config is going to get bloated and every time you deploy a container the config needs to be modified and reloaded. By this time you might be looking at chef or puppet for automating the config generation.
Chef and puppet are not simple to learn. They have their own set of quirks (like unreliable tooling support on Windows). I'm in the process of conquering this, but I hope one day there will be a simpler way.