It is actually built from source - by a separate open-source project called "stackbrew" which is basically the "homebrew of official Docker images". It was setup by the Docker maintainers before Hub introduced auto-builds, and then it just stuck. The idea is to allow anyone to participate in the maintenance effort, contribute new images etc. by simply making a pull request (again, just like Homebrew).
Here's the source URL from which that Postgres image is built: https://github.com/dotcloud/stackbrew/blob/master/library/po...
My solution to this is to avoid combining stuff in the same image.
E.g. my home server has an image for haproxy that splits incoming requests to my various test web apps by hostname. Each of those web apps run in their own docker container that bind-mount their appropriate directory from my home dir.
Then the container(s) I do my development in likewise bind-mount the appropriate directory. For the most part I ssh in to a single screen session, but for anything that has particular dependencies, I have separate containers (e.g. I keep a Ruby 1.8 container available to test code that should retain compatibility).
So far, all of my Docker containers are extremely small for this reason: I compose by orchestrating multiple Docker containers, not by stuffing multiple things into each container.
This for the most part means that dependencies I need to manually add rarely is more than installing an extra package with apt, or installing another gem.
I agree with you that it's hard to find decent images to rely on. But by keeping the containers small and single purpose I find that it's not that often it matters - the generic part of the config is often a matter of a couple of lines, while the Dockerfile and associated config files etc. tend to be dominated by my personal requirements.
This does seem like a pretty major limitation, is this a fundamental constraint or is it something that might be fixed in the future?
So you can branch out from a common base image, but you can't somehow co-mingle multiple parent images into one child. If you think about it like the filesystem giving you a snapshot at each line of the Dockerfile, you can see how inheriting from a single snapshot is trivial, but somehow inheriting from multiple snapshots into one would require much more awareness in order to merge them.
The practical effect is that either you build a more one-size-fits-all "base" image which you reference for all your services, or you have multiple base images with different more tailored sets of dependencies built up, or you have a very thin base image (e.g. not much more than ubuntu-latest) and just add what you need inside each individual Dockerfile.
I ran into the same issue at first when laying out the Dockerfile hierarchy for all my services/roles, but I think in this case it actually falls under "less magic is better" / KISS. Dockerfiles are so easy to read and write once you get going, you won't mind the simplistic approach.
But you can't use composition to combine the actual downloaded images themselves.
Using Chef within Docker to build up the services is documented on the Docker site[1], and of course puppet or whatever is essentially the same.
Very true. I've yet to find a suitable image for a simple Nginx + Unicorn + Postgres + Rails setup.
You lose a lot of the flexibility by integrating everything in one big pre-configured blackbox. In some cases that's nice, like when you actually want to ship a blackbox to customers, but when managing infrastructure? Thanks but no thanks. When you have proper configuration management in place, spinning up and configuring an extra postgres, apache, nginx, jboss, redis, ... whatever container should be dead-easy, otherwise you're doing something wrong.
Wouldn't it be just crazy if someone maintained a list of software they built to be modularly installed along with other optional dependencies compiled on a common architecture and operating system for a common base image?
We could make a tool to download them from a list of approved sources and verify that the files were cryptographically signed by a particular builder. We could also have the dependent images downloaded automatically. Then we could extend scripting support to run custom-built tools to tailor the install to what was on the system at the time of install or uninstall. Heck, considering all that they could make it cross-platform or even multi-platform.
Maybe they could call it the Really Professional-docker-image Manager, or the Docker Professional Knowledge Gem, or the Yodeler's Ultimate Maintainer package.
That would be crazy. Somebody should create a startup for this ASAP. [note to NSA crawlers: this has been an example of sarcasm]
I would add to the list that it is currently hard to even find decent images of popular services that you would trust deploying to production (e.g. Postgres). I see with the launch of Docker Hub that they have some flagged as "official" now, but for example the Postgres one is a black box (no Dockerfile source available - not a "trusted build") so I can't trust it.[1] I've had to spend time porting my Ansible playbooks over to Dockerfiles due to this.
I think part of the problem is that composition of images is strict "subclass"-type inheritance, so they don't compose easily. So it's hard to maintain a "vanilla" image rather than a monolithic one that has everything you need to run your particular service - so people just keep their images to themselves. For example, if I want a Ruby image that has the latest nginx and git installed, I can't just blend three images together. I have to pick a base image, and then manually add the rest myself via Dockerfile.
Also, although Vagrant 1.6 added Docker syntax, you really have to understand how Docker works in order to use it. If you're learning Docker I'd stick with the vanilla `docker` syntax first in building your images, maybe using something like fig[2] to start with.
At the end of the day it's another abstraction to manage. It does bring great benefits in my opinion, but the learning curve and time investment aren't cheap, so if you already have a suite of provisioning scripts it may not be worth it to make the leap yet.
[1]: https://registry.hub.docker.com/_/postgres/
[2]: http://orchardup.github.io/fig/