The concept is great - and in fact, many companies have built great tooling around Linux cgroups. It lets you efficiently binpack applications on a single server - which is why 'containers' were created in the first place.
The side benefit of letting you define your OS libraries, and other things, is a nice bonus, and way overblown in my opinion.
Docker and its tooling is just plain bad. It's so bloody unstable, unless you pick some esoteric combination of versions and storage backends (old docker + old aufs + old ubuntu seems to do the trick) - and even then, you'll run into problems. Documentation won't help you here - it's trial and error.
The docker group seems focused on feature, feature, feature, while the basic stability and performance remains poor. It's quite amazing how they get it so wrong. Browse through the github issues for performance and hanging issues. The surrounding tooling is poor as well - registry v2 doesn't even support deletes (because they need to maintain compatibility with the 100000 different storage backends for it). There is no LTS release, bug fixes only happen in latest version, so you're in a constant state of brokenness. And so on.
I remember one of the engineers giving a talk, saying that the problem they had was growing too big too quickly - they didn't have time to properly work out the base architecture in the early days.
They have effectively no incentive on getting rid of bugs in the core product or to test features they do add exhaustively.
Sooner or later someone actually using containers will produce a docker replacement that will take over unless Docker focusses on what actually matters.
They do however seem to be cutting so many corners technically, that the risk is they get undercut on their core offering.
Did anybody have better experience with e.g. Rkt?
Or mesos where everything needs to be in a tarball, which is extracted and sole program run.
If stuff is statically linked and related files (config, assets, ...) are part of bundle that can be chrooted, what is the value add of a container?
* http://jdebp.eu./Softwares/nosh/guide/move-to-control-group....
That's why nobody can get it right - because we already did.
Now as far as which distros put processes in a namespace and cgroup by default, I know at least CentOS 7 and Ubuntu 15 do this. And those two distros on their own would qualify for "most".
To check if your distro does it, one way of checking is just doing a `cat /proc/1/cgroup`. This will show you what cgroups process 1 is in. By default you will be in the "root" cgroup.
To check your namespaces, `ls -l /proc/1/ns/`. You'll see the process is in some randomly generated namespace ID per item.
I'm sure you could recompile your kernel to disable this behavior, but the default reality of modern Linux is that everything is already running in a "container".
Now the question is whether or not people want to take advantage of that reality, and separate out processes in isolation, or keep running everything on a system in such a way that any single process can impact the whole system.
The whole process is much simpler, I think, than that of creating a Linux container (that's the whole reason Docker exists: to simplify & abstract something which isn't really inherently complex, but is accidentally complex).
Plan 9 certainly wasn't perfect, but it had some really high-quality ideas we still haven't assimilated in mainstream platforms.
If you could just pass `-static` to gcc and it actually worked like you expect this would never have happened.
Fortunately that seems to be changing somewhat. Go is totally static, and Rust can easily be made totally static using muscl. You can even do totally static C/C++ apps fairly easily with muscl.
(repeat for network namespacing: it's really nice not to need to play games to have your CI server start 3 running jobs which all think they're listening on port 80)
None of that is impossible – in the case of chroot there's many years of precedent - but if you do it regularly, there's a strong appeal to automating a common pattern.
This is especially true when your goal is supporting development teams: with something like Docker, normal users don't need root just to start a daemon on a privileged port or write to a couple of files. If you work in a large or security-conscious environment, that's a fairly big draw.
(Not saying that Docker is perfect or necessarily the long-term winner in this space, only that there's a usability gap which a lot of people fall into).
On the contrary, any user that can run arbitrary containers (such as rootplease[0], for example) has root-level privileges on the host system.
It's true that Docker doesn't protect against compromised or malicious users with privileged access. That's a very hard problem in general which can only partially be addressed at this level — especially since many of the most damaging attacks don't need it (“The bad news: they exfiltrated our customer database. Good news: they didn't get root on the EC2 instance”). I think most of the answers for this problem are going to continue to rely on existing practices like code review, auditing, getting finely-grained SELinux / seccomp rules into the development mainstream, etc.
Why would you, though? A developer would at the very most require the application's privileges, not the super user's. And that's only really necessary when doing live troubleshooting.
You can reduce the number of things which hit that friction in a number of ways: having easily repeatable builds so a developer can test on their own VM; using a cloud service so test VMs are disposable and never shared; setting up a platform / microservices approach so a wider range of things either don't need to be touched or can be deployed without privileged or direct access, etc.
Containers (really namespaces) one way to hit that last goal: as a classic example, if you have a web app running on port 80 everyone who deploys will need to periodically restart Apache/nginx/Varnish/etc. and that may include elevated access to debug processes running as a different user. This is by far the simplest problem in this class and there are various ways (proxies, site users, firewall NAT rules, moving config into .htacccess or its equivalent, etc.) to reduce the number of times you have to care about it but but it still requires work to maintain and adapt code (especially with third-party apps).
Some people quite reasonably prefer to solve this entire class of problems by tossing everything into a container so it can run as if it's the only thing on the box.
This is absolutely fine if you intend to remain a shit sysadmin. Go nuts with Docker. The 21st century will be waiting for you when you're done.
So will 19 fucking 60 because nothing's fucking changed.
> I had a huge amount of headaches to get it actually working.
Moving from running one container to running multiple containers is probably one of the most confusing parts of getting started with Docker
There are a large array of orchestration options and tools - each with their own pros and cons: Swarm, Kubernetes, Mesos, Marathon, Mesosphere, Centurion, Rancher, etc.
Docker 1.12 now having built-in orchestration with Swarm should make this easier[1].
[0] https://github.com/containous/traefik
[1] https://blog.docker.com/2016/06/docker-1-12-built-in-orchest...
http://www.nextplatform.com/2016/03/22/decade-container-cont...
Docker may be flawed, but containers aren't. If you need some enterprise leader to tell you this instead, here are some Gartner posts showing this is the way:
VMs may be well established and "magic quadrant", but they are also on decline, and containers make better use of hardware/resources:
http://www.informationweek.com/cloud/infrastructure-as-a-ser...
It's important what you use to roll out containers and requires forethought:
https://www.gartner.com/doc/3267118/containers-change-data-c...
Google Container Engine:
https://cloud.google.com/container-engine/
Open-source MapReduce implementation:
https://gigaom.com/2015/02/18/google-open-sources-a-mapreduc...
MongoDB? Yes, don't worry about data design or normalization! Just throw that stuff in there, call it anything you want! Finally programmers are free from the tyranny of decent databases. I can't think of any downsides to that arrangement.
Docker? Yeah man, you can just say "docker pull redis" and then you don't even have to bother with Ubuntu getting in your grill with all of its apt-get shenanigans. It's awesome! Now how do we make that port accessible...
I think the best approach for your cert issue is to abstract that into a separate service (nginx is an option, but I'd recommend the Rancher approach below). Yes, that means you have to add another container, but that's just another block in your docker-compose.yml file. Embrace the approach of separating your components into containers and organizing them as a stack. You can easily link containers together, share data volumes, and start/stop individual containers or the stack as a whole.
The problems that you're having are pretty easy to fix with some tooling. Rancher (http://rancher.com/) greatly simplifies the cert issue by allowing you to import certs and provide them to the Rancher loadbalancer service (which you can add to any stack). There's also a LetsEncrypt community catalog template that automatically retrieves and imports certificates to Rancher. There are other open source orchestrators like DCOS, but Rancher is probably the simplest to use, and it's the only one I'm very familiar with. There are SaaS options that you can look into, but I don't have experience with them.
As for the container crashes, it's trivial to automatically restart them. Just pass the --restart=always flag to the Docker run command. You can also add the flag to a docker-compose.yml file.
wait. aren't these things supposed to give us less pieces to arrange and manage?
> The problems that you're having are pretty easy to fix with some tooling
yes, of course the solution is more tools. what exactly was the problem again?
Docker compose lets you group these containers together, and that is what ultimately makes it feel like you have fewer pieces to manage. With that, you can start/stop a stack of containers (e.g. django, nginx, postgres, redis) with a single command, but still inspect and manage each component separately. This is something you might normally do with bash scripts, but with Docker you can take that same app to an orchestration platform and run it on any host. Run it on your laptop, run it on a linux server, run it on a SaaS provider like Docker Cloud, run it on a private cloud with an orchestration platform.
> yes, of course the solution is more tools. what exactly was the problem again? Docker is just the foundation. Nothing more. I'm fine with learning more tools because I feel that the foundation is solid. The problem is being able to ship and manage your apps. That is much, much easier for me now and I'm very glad I invested the time.
This mentality is WHY Linux/docker/containers/Linux worlds NIH syndrome, is such a tire fire.
Windows containers are now available, if you can't stand linux: https://msdn.microsoft.com/en-us/virtualization/windowsconta...
Anyway, don't use containers if you don't want to. I'm glad I invested the time since I understand them very well and use them to my advantage. But I did have to learn a lot and experiment with a bunch of tools, and maybe that's not worth it to you.
That said, my reply was to someone running an IRC server, presumably on a single server, so don't stretch my advice to a production app handling millions of transactions. Obviously the core issue is that the app crashes, and it's still up to him to fix that. This is almost certainly a problem with his app/config, not Docker itself (though it ain't perfect). If it's something that happens every 6 months, then auto restarting will probably save him a lot of problems. If your transactions are so precious, don't pass the flag- it's up to you.
Also good to know about automatic restarts. I don't know how I missed that, I had to read a lot of docs to get where I am.
Still, the amount of tooling that exists and the knowledge needed to pick the right ones for a given situation goes to show that this isn't as simple as packing a shipping container and letting someone ship it...
You don't need to use Rancher if you're just running one app. If that's all you need to do, then it could be as simple as running docker-compose on a linux server and mounting the certs into the nginx container as a host volume (https://github.com/jwilder/nginx-proxy). This is a fine approach until you want to split your containers across several hosts (redundancy or scaling) and you have several apps to worry about.
What's the easiest way to get a .yml on a cloud server somewhere and let it assemble it assemble the containers for you?
Also, do you know where containers set environment variables? The official Postgres makes available something like PG_PORT_3542 and u can just refer to it from another container. Where I can't seem to do the same with the Redis one...
If you want to use your own Linux host, then the simplest way would probably just be to SSH into the box, git pull, and run "docker-compose build && docker-compose up".
Setting environment variables is pretty easy, but I don't think you need them in this case. If you're trying to make a redis or postgres container available to another container (your app), then you can do so easily with links in docker-compose. Something like:
myapp:
image: myimage:0.0.1
command: cmd to run
links:
- redis:redis
ports:
- 80:8000
environment:
- hardcoded_var=my_env_value
- var_from_host=${host_var}
redis:
image: redis:latest
If you need to define environment variables, you can do so with an environment dict as shown above. There are a few other ways to define env vars:
https://docs.docker.com/compose/compose-file/#variable-subst...
https://docs.docker.com/compose/environment-variables/#/the-...
Running out of memory is a bit like running out of space in the container.
If I dropship a container on your yard, you're going to have a container but your options in what to do with it are limited.
Now if your container has a lifecycle and a scheduler because it needs to be in China on Wednesday, you suddenly have a lot more complexity.
Docker itself being a buggy piece of crap is neither here now there in the grand scheme of things.
Unfortunately, when my cattle arrived in the US, they were all dead. I was told containers "would just work".
After some experimentation, we managed to get our cattle in a container by building a system that correctly managed food and waste. Then we found a partner who had successfully packaged sheep and so we just used that. Unfortunately, they appeared to have a leak in their waste management system, and the container overflowed and all the sheep died. We did not have this problem when the sheep were able to fill the entire hold with their output.
Clearly the idea that shipping containers just work is a gross oversimplification. In fact containers seem to be complex thingamabobs with multiple points of failure.
It is refreshing, however, being able to write, in a couple of hours, a setup script from scratch that debootstraps an install, chroots in there to configure it and launches the container via systemd-nspawn. Very few moving parts, excellent for development environments. Not yet something I trust in production, though.
Base lxc may be a bit of a pain. Look at lxd - I believe that makes things much much simpler.
Another thing about lxc is the unpriviledged container - which I think is great for security (not sure if docker provides this).
However, I do see it as positive, because its hype, regardless if good or not, has created the traction for Go and OCaml on the data center, thus eventually leading to less C code for such use cases.
So hype or not, maybe we do get some security improvements on the overall stack.
So anyone that wants to improve Docker or adapt it to their distribution of choice needs to eventually use them.
For example, Microsoft did several contributions in Go for making Docker run on Windows.
The TCP/IP stack used by Docker on OS X is taken from MirageOS, written in OCaml.
It's popular to hate on C (and C++) because the languages are so ubiquitous and long-used that a large body of terrible, unsecure, and poorly written code exists using them. Other languages haven't had the same success as these two yet, so haven't had their warts exposed enough to be dumped in the "automatically hated" category. Java comes close, but it also is typically lumped in the "automatically hate it" bucket and for similar reasons.
Dennis M. Ritchie himself on the history of the language[0]
"To encourage people to pay more attention to the official language rules, to detect legal but suspicious constructions, and to help find interface mismatches undetectable with simple mechanisms for separate compilation, Steve Johnson adapted his pcc compiler to produce lint [Johnson 79b], which scanned a set of files and remarked on dubious constructions."
Lint which is still mostly ignored by the masses to this day. At CppCon 2015, about 1% of the audience acknowledge using static analyzers.
Per Brinch Hansen letter to C.A.R. Hoare in 1993a [1]
"The 1980s will probably be remembered as the decade in which programmers took a gigantic step backwards by switching from secure Pascal-like languages to insecure C-like languages. I have no rational explanation for this trend. But it seems to me that if computer programmers cannot even agree that security is an essential requirement of any programming language, then we have not yet established a discipline of computing based on commonly accepted principles."
There are many other sources of similar statements since C exists, so the hate isn't something new.
Regarding C++, yes unfortunately it inherits C flaws, but at least the community tends to embrace language features to improve the language safety and push for type based programming.
[0] https://www.bell-labs.com/usr/dmr/www/chist.html
[1] brinch-hansen.net/papers/1999b.pdf
Whenever I'm forced to use a C++ program it's buggier than the C equivalent.
https://github.com/isocpp/CppCoreGuidelines/blob/master/CppC...
http://erdani.com/index.php/books/modern-c-design/
https://msdn.microsoft.com/en-us/library/hh279654.aspx
http://stroustrup.com/Tour.html
http://elementsofprogramming.com/book.html
Most of the C++ bugs I found happened to be written by former C developers that disregard using C++ stronger type safety, RAII, the standard library containers and usually use naked pointers alongside malloc() and free().
And that is the biggest problem with C++, its copy-paste compatibility with C, which allowed its adoption by C compiler vendors, but made it as safe as C when developers disregard best practices.
I'm using jwilder's nginx rproxy container, the let's encrypt helper container for that plus half a dozen Web apps on a VPS. Among those Weechat, two instances of vanilla nginx, rutorrent, dovecot, mattermost, nextcloud and a test environment for my Python tinkering. Works like a charm. Upon bringing up a new container, I supply the desired subdomain and my letsencrypt user data, the container comes up and uses SSL plus automatically renewed certificates.
My Docker experience so far - in a private, limited, very much not production environment - has been "virtualisation light" all the way.
I think of Docker as a somewhat extended chroot, to wrap my mind around it.
Containers are a symptom of our still immature ways of building applications.
The problems you bring up would still be there without containers, but you might not notice them, until you e.g. want to bring your setup over to another machine, for example.
Or want to duplicate it somewhere else. Or want to upgrade something else on you machine that just happens to interact badly with your setup.
I wrapped my IRC client (weechat + glowing-bear) in a Docker container. Oh, not a container though, because I also needed https, which meant I needed either a mechanism to build and update letsencrypt certs in the weird format that weechat expects, or to run an nginx instance in the front (and also somehow get the certs, but that's easier with nginx). So two containers.
And even though there was a ready-made nginx+letsencrypt https reverse proxy container (actually several), I had a huge amount of headaches to get it actually working. Even with the system set up, I occasionally have the container crash with exit status 137 (IIRC), which I've assumed might be because weechat leaks/consumes memory and eventually the host server kernel kills the process. Maybe.
So in my limited experience, comparing Docker containers to shipping containers is a gross simplification. Shipping containers are simple constructions requiring well-defined simple maintenance, while Docker containers seem to be complex thingamabobs that have multiple points of failure.