News for hackers. Stuff that matters.
Re:Docker 0.11 Released by j_s on Wednesday May 07, 2014 @04:01PM (#7712332)
The first thing I wanted to do with docker was cleanup the mess I made walking through the examples. Turns out there is no built-in command to do that, and it looks like it is going to stay that way.

https://github.com/dotcloud/docker/issues/928

I was surprised to see that nothing had been done about this; a comment there mentioned that someone was able to clean up 19GB of space once he figured out what was going on. The solution is easily google-able since so many have run into it, but cleanup of unused resources should be a first-class operation in every virtualization solution.

http://stackoverflow.com/questions/17236796/how-to-remove-ol...

Re:Docker 0.11 Released by shykes on Wednesday May 07, 2014 @04:28PM (#7712464)
Hey, I'm a Docker maintainer.

We don't plan on "keeping it that way". Disk clutter after lots of builds is a problem which we're very much addressing. It's specifically a top-level docker clean command that I'm not a fan of, because it doesn't capture a clear intent - what are you cleaning, and who determines what should be kept and what shouldn't? In general I like to avoid Docker having to guess what you want - the proverbial "magic".

A few notes:

* The reason Docker can clutter your disk is because we made an explicit tradeoff: it's better to keep too much data than to remove too much. That is why we're able to say "yes, you can run a database on Docker. Neither the container nor its data will be removed, at any point, unless you explicitly request it". Everytime I hear someone complain "there's too much stuff on my disk", I rejoice that I am not instead hearing "Docker ate my data".

* We have already introduced a few improvements. For example Docker used to keep the intermediary container for each build step (for later inspection of the build output, transient state etc). Now, by default, we don't.

* The most elegant fix, I believe, is to use naming and tagging as an indication of interest. If you are interested in keeping an image, give it a name (for example with 'docker build -t'). When an image no longer has a name referencing it, instead of keeping it around as an anonymous image like it does today, Docker can remove garbage collect it. That should take care of the problem elegantly and without requiring a new, clunky top-level command.

TLDR: the "disk clutter" problem is the result of a conscious tradeoff which we are glad we made. We have started fixing it, and there are more fixes to come.

Re:Docker 0.11 Released by CSDude on Wednesday May 07, 2014 @04:46PM (#7712560)
When you make a dangerous `apt-get remove xxxx` (that will cause critical packages to be removed), it warns you and wants you to write "Yes, do as I say!" https://fbcdn-sphotos-a-a.akamaihd.net/hphotos-ak-ash3/t1.0-...
Re:Docker 0.11 Released by olouv on Wednesday May 07, 2014 @07:28PM (#7713171)
It's a real PITA that the first thing you do when you start experimenting with docker is to have to find out about this missing clean command and to design a custom alias to fit your needs. In my case, I was expecting it (eg. `bower cache clean`, `npm cache clean`). Regarding the unclear intent, that's what cli help is for! `docker clean` could output it by default.
Re:Docker 0.11 Released by shykes on Wednesday May 07, 2014 @07:58PM (#7713275)
What would you expect it to do exactly? Remove all untagged images?
Re:Docker 0.11 Released by blueyed on Thursday May 08, 2014 @07:10AM (#7714688)
That's a good starting point.

I have a docker-cleanup script, which removes untagged images and containers: https://github.com/blueyed/dotfiles/blob/master/usr/bin/dock...

Re:Docker 0.11 Released by j_s on Wednesday May 07, 2014 @04:33PM (#7712485)

  > We don't plan on "keeping it that way".
Cool! Can you point me to where the discussion on this is occurring? I was confused because the issue I referenced above was closed without a reference to any issue open on how this problem will be addressed.
Re:Docker 0.11 Released by shykes on Wednesday May 07, 2014 @04:55PM (#7712605)
Most of the discussion has been on irc: #docker and #docker-dev on Freenode. There are several contributions underway under the general "theme" I described, but if it helps I could write a summary of how the experience will change for the user (normally there should be almost no UI change).
Re:Docker 0.11 Released by threeseed on Wednesday May 07, 2014 @11:19PM (#7713773)
Are you serious ? If you want to capture a clear intent then simply ask the user to provide parameters.

docker clean remove-untagged

docker clean remove-old 4w

It's bizarre that the Docker team is so oblivious to the issues running Docker in a real world environment.

Re:Docker 0.11 Released by darkarmani on Wednesday May 07, 2014 @04:37PM (#7712522)
Thank you for your feedback! I'm not a Docker contributor but I'm a big fan, and everyone that responded to you, forgot to thank you for your feedback. Your feedback is immensely valuable to making it better. Obviously if you think nothing has been done about it and they are saying there are fixes, it needs to be communicated better.
Re:Docker 0.11 Released by shykes on Wednesday May 07, 2014 @04:56PM (#7712612)
+1 on that, anyone who takes the time to tell us about their issues with Docker is automatically making a valuable contribution to the project. And it's appreciated, so yes, thank you :)
Re:Docker 0.11 Released by CSDude on Wednesday May 07, 2014 @04:48PM (#7712569)
There is another problem of other than space. I grade programming assigments with Docker and let the students grade too, and when containers get too much, docker tries to read each of the metadata and it hangs, and I could not find the issue and could not remote reboot my server, so I walked to server room at 4 am. For a 8 core system, load avg was 33. So, be careful with your containers, remove them when not needed.
Re:Docker 0.11 Released by danielbln on Wednesday May 07, 2014 @04:12PM (#7712377)
I actually have a cron job running that cleans up old images once in a while. Not pretty, but yeah, otherwise disk usage grows and grows and grows.
Re:Docker 0.11 Released by koffiezet on Thursday May 08, 2014 @11:27AM (#7715783)
There is a difference between "messing around", and actually running docker in production. I do NOT want a 'cleanup' command to be available on my servers. Cleaning up on commandline is rather easy with a few docker commands and xargs. I wrote small shellscripts to do this for me when building images, but these will never ever be installed on a production server.
Re:Docker 0.11 Released by nickstinemates on Wednesday May 07, 2014 @04:13PM (#7712380)
What would you recommend be done? Good solutions are always welcome.
Re:Docker 0.11 Released by j_s on Wednesday May 07, 2014 @04:35PM (#7712505)
I'm still learning my way around, but based on my understanding of containers as instantiations of images, there should be a command to delete unreferenced images. I understand the need to manually specify containers to delete but not the difficulty in cleaning up unreferenced dependencies.
Re:Docker 0.11 Released by danudey on Wednesday May 07, 2014 @07:15PM (#7713137)
The problem with that design is that in a lot of use cases, images don't necessarily need to be 'referenced'. For example, I'm using docker for a build/test system; we spin up a container from an image, run some stuff in it for a few seconds, and then shut it down. 90% of the time, the images are 'unreferenced'.

The other suggestion, put forward by a docker dev in another comment, is to have a clean command for unnamed/untagged images. Our build system has 'buildbot' (python package compilation) and 'software' (test environment) images, so this sort of automated cleaning would get rid of all of that extra crap that we have lying around.

The big issue for me is stacks of changes. Like git, anything you commit, remove something from, then commit again still takes up that space forever until you consolidate down. Which you can't do automatically.