Without Docker everything new becomes a project on its own where you need to call in "ops" for the smallest of things. Not anymore. Ofcourse this has some other downsides, but in general it's a massive step forward.
I hope that one day we will stop echoing that or framing it as a bad thing.
It's always good to involve people. Writing, build and running software is a shared responsibility.
I truly believe that if you be nice with your ops, explain what you want to achieve, they will help you. It is a two-way street.
It is not a tool that will "fix" how humans interact.
Also, put yourself in their shoes, it could be that:
"With Docker everything new becomes a snowflake on its own where you need to call in "dev" for the smallest of things." -- or uncompress the image and look around how/what a specific image does things when it start, place app/config/data files, etc...
All too often, they're responsible to handle things breaking, but not responsible for getting features out the door.
Hence, they become a huge gate to get anything out; they have no incentive to help you.
Meanwhile, you get dev owning some of the operational burden, and it works pretty well. Until you have compliance needs to separate them.
Probably the same people who said Dropbox brings nothing new over rsync. Making something easier and bringing it to more people (two halves of the same thing) is a BIG value add. Often the majority of the value.
Not quite!
I wrote my own rsync-like-toolset before rsync itself was released (far too late, when you think of it!), and I immediately loved DropBox.
Maybe because I am dev, AND I am ops, or maybe because (unlike DropBox) I am not sharing folders with my mother, but more likely, because I'm an idiot (everyone tells me "Catcher in the Rye" is great - and after having read it every 3rd year for nigh on 30 years, I still don't get it), but I don't really understand the appeal of docker.
Sure, I can use it, because you kind of have to these days, and I don't really find it objectionable, but it seems like there is a whole lot of "something there" that I'm missing. At this point, I'll probably never understand the benefit of. It won't be the last tech ritual I will have to endure before I die...
I’m going to keep trying, probably, because so far it’s one of the only big classics I haven’t liked. Anyway, maybe we’re both idiots!
I started in my mid-teens. I was told: "It changed my life!".
I've read it every three years or so (even for the decade-plus that I've lived out of the US, once in Japanese!), and ... I just don't get it.
I probably don't get Moby Dick either, but I find it to be remarkably funny, and not a whiney teenager, so I'm going to give it a pass.
Thanks for the comment. I have a bit of a hang-up about how much I don't get "The Great American Novel" (as a born-and-bred rural American, I should get those, you know? :-) )
Starting with the outright lies about what Docker could or could not do (Docker does X, Y, and Z - but Y and Z were actually still in development), to the outright refusal to play nice with the open source community (remember the Docker vs. CoreOS fights for standardization of container formats?). If Docker didn't invent it, or think of it first, it was a terrible idea and had to die.
They even went so far as to start a convention before they had a production product.
Then Kubernetes, Rocket and all the other container platforms and tooling came along and ate their lunch. Why did people adopt them so quickly? Because they played nice with each other and made it easy to build your system as you wanted it... not just as Docker wanted it.
The entire thing was a money grab. And when things went sideways, they made some terrible decisions (the entire Moby fiasco). It's not surprising Docker finally fell...
Because Google of course.
We don't need one. Red Hat is working on podman.
And, no. Podman doesn't offer any advantages in the Kubernetes ecosystem by mapping better into k8s "models".
It's one reason I'm really not a fan of the Silicon Valley model .... pick out the top 10 most important technology advances in the last 20 years and about half of them were uncommercialisable in the Silicon Valley sense.
https://en.wikipedia.org/wiki/List_of_Java_virtual_machines
Google is the only one that worked around the licensing, and now we have our phones stuck on a mix of Java 6 - 8 subset.
Something like Docker may eventually hold this title, but Docker itself? Certainly not. It was simply too poorly executed, both strategically but especially technically.
Running production workloads in Docker containers was, is, and will be remembered as nothing short of professional negligence. Often expedient, sometimes worth the risk, but always a liability.
A bunch of disorganized unpinned dependencies get thrown into a container that will work in a reproducible manner only until one of the upstream deps changes.
This approach also allows people to get away with deploying an app without truly understanding it. That's always been a good recipe for success.
Another poster claimed that it's mainstream practise, so what? So was running your web-server on 90s era IIS. Not that that was good idea either.
Even if you install from repos in the dockerfile, they should have version numbers.
I guess I'm old school but I install the stuff on my linux instance or server and work out the dependencies so they don't clash.
I haven't found it that difficult.
Just cos the devs want to throw some ill conceived container over the fence onto my systems, doesn't mean I let them. ( I mean I guess they can, but only if they get the support calls instead of me.)
“Adopt whatever buzzword shit developers are into” is about the worst career advice you can actually give sysadmins that work in non-cutting edge companies.
This might have been true in your experience but it hasn’t been in mine (.com, .edu, .gov). There’s common term (shadow IT) for what inevitably happens: someone hires their own staff, gets their own servers, arranges their contractors to host it, architects their application requirements so it can’t be run by central IT, etc. The reason it happens is that the IT department is not listening to its users and thus giving them a strong incentive to find alternatives to accomplish their jobs — and they’ll almost always win unless the CIO is politically untouchable and has an iron hand on the budget.
This is especially true in these cases: using the cloud or containers is what senior management are hearing from all of their consultants, and likely many peers. If you’re trying to hold that back without a really good reason, it is unlikely to end well for you unless you’re related to the CEO, especially since it’s easy to find people with equivalent or greater experience who can point to good results from having adopted what are now relatively mature technologies with plenty of case studies.
> “Adopt whatever buzzword shit developers are into” is about the worst career advice you can actually give sysadmins that work in non-cutting edge companies.
If you think that’s true, I would suggest you reread my comment and ask whether that’s a fair reading of my suggestion about finding a way to balance concerns. There is a huge amount of space between “no” and “you can run whatever you want”.
I've built sophisticated automation to achieve this, so it's all reproducible.
I just don't use Docker that's all. It doesn't bring anything to the able that I can't achieve more simply some other way. And it has it's own problems.
I've been a sysadmin for 25+ years and I've administered production DNS server, mailservers, web server, app servers, routers, switches ...
I think stuff like docker and k8 appeals to people today because it promises that you don't have t understand any of that old stuff.
But to me it just makes my life harder because I don't need the underlying tech abstracted away. If anything it makes my life harder to add an unfamiliar layer on top.
I'm not worried, there will always be market for guys like me. When the fancy new stuff stops working, someone has to fix it.
The reason why containers are popular is that you get a simple way to manage all of that and it's completely standard on every project. App A doesn't need to be modified to coexist with App B because they're running in an isolated environment, the sysadmin doesn't need to learn how the container was built just to manage services or deploy new instances, builds are much faster because of the layering model, you can deploy a single tested image with no chance of differences creeping into your automated deployments in testing & production, etc. All of that is something which you can achieve with other tools but there's a huge win to not having to reinvent each wheel when you encounter it and being able to reuse other people's work easily.
Note also that I said “containers” instead of Docker — some people might prefer, say, podman / buildah but the key concepts of having a clear support boundary with an isolated-by-default filesystem model are really more important than the implementation.
I disagree on that point.
Just taking a docker image from a developer and deploying that to production does not work to well in my experience, and guess who gets the support calls when things break?
If a sysadmin gets a say in how the container images are configured, it usually works out much better.
Also partitioning the filesystem to avoid dependency clashes used to be more important when I used rack mounted servers. Now it's easy enough to deploy separate cloud instances to partition things. And it's a real VM with real tools on it so I can diagnose problems if they arise.
In my experience, that's been about the same for any mode of deployment by a developer without ops experience. The main thing Docker added was standardizing the mechanics and avoiding certain classes of errors like not having reproducible builds or deploying local artifacts inconsistently.
Because your automation is made by you, and for you, no one else can take it over if you leave. And even if they can take it over, they will have way higher salary - compared to the container based system.
Source: consulted for a company that got bitten by allowing containers without a patch management process for vulnerabilities.
They tried it their way for a couple of years, and there were performance problems and outages.
Definitely not.
Docker is a lovely developer tool, but it’s not, in my experience, ever been (even with swarm when that was a thing) suitable for production workloads.
Containers: yes.
Docker: that’s a container runtime, and definitely not mainstream for production workloads.
Docker runs as the Container Runtime under most Kubernetes clusters (in fact every Kubernetes cluster I've seen). The only other CRIs that are commonly used are CRI-O and ContainerD (which Docker already uses), and last I saw CRI-O was < 10% of the Container runtime market.
Care to explain what makes that docker tool/runtime unsuitable for production ?
No one is seriously confused about "using docker" meaning, oh hey, I run my containers on kubernetes. I have literally never met someone who was confused about this. I suppose some people just stop at "I use docker" and don't know what that means when it hits production... but I'm skeptical they actually think a copy of docker is running out there in prod.
...but to be fair, the parent comment did say that running "docker containers" was mainstream; and fair enough.
My point was running containers using docker most certainly is not.
So containers are most certainly run in prod using docker (to be specific docker daemon), are they not?
If you still install a full ecosystem every time you want to try a piece of code in any language in Github instead of doing docker run I think you are missing something.
With that said I too use it in my dev machine and don’t really see the downsides for the kind of projects I work on, so I’m curious if you can explain your opinion.
Code should therefore be organized so that programmers have to interact with the fewest possible artifacts (repos, compilation units, deployment artifacts, processes) to deliver that value, ideally 1. (This is symmetric with Conway's Law.)
Delivering value on that single artifact should be possible in isolation, meaning (after cloning) without internet access, or any other service running on the development environment. Concretely, all services should have a "development mode" that allows them to exercise their business logic against mock or stub dependent services, in-process/in-mem databases, etc.
So,
> If you still install a full ecosystem every time you want to try a piece of code in any language in Github instead of doing docker run I think you are missing something.
If I'm working at a company, I expect that delivering business value means checking out, compiling, running, and testing a single process with no external dependencies. If this isn't the case, I will make it the case, before starting any value-delivering work.
If delivering business value requires me to spin up the entire universe of services on my laptop, creating a miniature staging or production environment, that's a problem that needs to be fixed. This means that docker-compose, in fact Docker at all (unless the business value involves e.g. Linux-specific features), Kubernetes, etc. in my build/test/run cycle are all red flags.
Exceptions exist, but they are and should be rare.
The previous company I worked in and others I’ve heard of were dysfunctional because organized like what you describe compared to their size. Backend engineers working on their API in isolation, without a good contact with end users, their business, and the UI, is the huge red flag in my case.
> Concretely, all services should have a "development mode" that allows them to exercise their business logic against mock or stub dependent services, in-process/in-mem databases, etc.
This brings only complexity and additional code in our case without any clear advantage (except code that’s easy to test but that’s circular reasoning). If your code is in the chain between a user action and the database, then the sooner you test the whole chain for side effects the better, without complex integration testing processes (with subs that will never be as accurate as the real thing). Bonus for being in the user’s shoes.
> If I'm working at a company, I expect that delivering business value means checking out, compiling, running, and testing a single process with no external dependencies.
Given the complexity and the number of dependencies of an OS running a browser and your editor/IDE, I don’t really see the point here.
> If this isn't the case, I will make it the case, before starting any value-delivering work.
That’s exactly the kind of issues I experienced at previous companies: engineers endlessly reworking the architecture until it’s perfect in their view with no rationale regarding real value (« it’s best practices » is not).
> spin up the entire universe of services on my laptop, creating a miniature staging or production environment, that's a problem that needs to be fixed.
This nicely summarize our different point of view: that’s a killer feature for me.
Again it really depends on the context and I’m not disagreeing with your experience, I disagree with how you state this like it’s universal rules of good software development without taking the context into consideration.
One should not miss the importance of running Docker under Windows or OSX natively, with a VM running Linux tucked behind the scenes. It makes developers' experience so much nicer, and compatible across platforms. This helps adoptions a lot.
Time and again, path to mass adoption is not technical excellence, but making an important thing stupidly easy, when one has no excuses not to try.
What Docker did provide that other packaging tools didn't was free image hosting for anyone, no vetting, no strings attached. Of course that was a major hit with developers. Free hosting often is, especially when well integrated with development tooling.
The thing is that it sets you up for a nice acquisition. But for some reason Docker, Inc. felt they wanted to go the VMware Way instead and sell enterprise tools to enterprise customers. Those are two different strategies that will never be reconciled. It makes sense for them to shed that part and double down on Docker Hub and the Mac/Windows tooling.
It does. You can do crazy shit like this with Docker containers:
thinking about the time we created docker at work in 2009:
nobody could remake our centos4 build machine for our C++ code
so we shut it down, imaged the file system into a .tar.gz, and
made our makefiles transparently extract it & build via chroot.
so absolutely fucking cursed -- @stdlib
It's essentially still exactly what docker allows you to do:
tar-up your whole f'ing disk, and run it in a chroot somewhere else.
Of course it's horrible. But now, you can get away with it.
What has Docker to do with this?
It, well, I wrote a bunch of perl scripts to "streamline all life cycle commands across all services" back in 2002, where all we had were FreeBSD jails.
So, again, what's so special about Docker?
".. in 2009: nobody could remake our centos4 build machine for our C++ code ... so absolutely fucking cursed"
What lead a computer running a supported operating system[1] in a state like this? Distill that and you have docker :)
1. https://en.wikipedia.org/wiki/CentOS#End-of-support_schedule
Just a guess...
Problem is, you still need to update your dependencies, so sooner or later you’re bootstrapping modern GCC and Stdc++, openssl etc all from a Centos 5 base just to get a forwards compatible rpm
A that point you start tarring up your chroots so you know how to reconstruct this Frankenstein build enviroment
Thank Docker for giving an easily portable environment that’s almost as easily accepted by customers as a RPM
Building software on top of frankenstein build environments is how you end with broken software, and should not be encouraged :)
Let alone development environment.
Infect is a too strong word, because normally the cross compiler will coexist well with your system. But you are right about the chroot, throw it away and start again, repeat. That is great!
I normally install gcc-arm-linux-gnueabihf + qemu-user-static and enable binfmt. It works well for building armhf, but I can imagine for things like ESP32 where you don't have the toolchain sources and etc.
Software has to be read or at least installed in a safe place (before installing on a live server) so then one can be sure it does nothing silly.
Packaging is not hard, but there are some rules, including placing binaries in /usr/bin, configuration files in /etc and /etc/default (deb) or /etc/sysconfig (rpm), general package files in /usr/share. If you place files in the standard locations probably the package manager will detect changes in config files.
Docker (the software, not the company) and Stripe are two of the best recent examples of that.
What Docker did is make a great application packaging tool, and the fact that it uses cgroups/namespaces underneath is irrelevant for 99% of users, because they're deploying containers in VMs.
Docker is still an over-engineered solution to this problem, but its popularity is evidence that there is in fact a problem.
This created the need for piles of configuration management or machine build scripts to deploy the app.
Docker became popular because it provides an abstraction that simply (and elegantly) eliminates the problem of distributing and running server applications.
You're being purposefully obtuse for snark points.
In general, I always try to give the contrarian opinion as it causes an interesting discussion between the community members.
Also, there are some young members here, that might not realize that what we see as "innovation" might happened before (multiple times).
The truth is actually in the middle.
I could care less about the points.
Docker had the right mix of existing tech, good packaging, great user interface, and good marketing. And together that made a product that made developers do new things in practice, and it changed the industry.
I think it is important to remind people that in order to disrupt an industry with a new thing, you need the whole package. Not just a good tech.
This is why I have beef with docker, it was marketed to solve multiple things, but ultimately it becomes over-glorified zip file. It absolutely fails in reproducibility and I constantly see scenarios where Dockerfile that worked one day broke another day, or it worked on one machine and broke on another and so on.
Yes, you can put extra effort and make your dockerfile reproducible, but similarly you can make a reproducible bash script.
Even for AWS Lambda, I use docker to build linux lambda packages on non-linux environments (docker for windows/mac). The easiest way to get a service up in Elastic Beanstalk is with a Dockerfile.
Even if Docker (organization) does die, Docker (open source tooling) will live on for a very long time. And although I do like where podman and others are coming from, I don't think that it will displace docker in the long run.
One of the daemonless tools out there now or some interface specifically for Kube will exist as a dependency
This was eliminated way before Docker. Amazon had this problem solved around 2009 when it was not a new thing in the company. LXC also predates Docker. I think you are talking about immutable images that can run on any OS. This is what Docker introduced. This is not as ground breaking as people try to make it, this is why commercial success avoided them at the first place.
Not anymore, over the past couple of years Kubernetes slowly stripped Docker most of its power to the point that Docker can be totally removed from k8s clusters. And this will be widely be the norm in a couple of years or so
> which for most users today still relies on Docker
Was that inaccurate? Are most users manually running Kubernetes clusters with an alternative runtime these days?
It's just a rootfs compressed tarball. Running on top of Linux technology.
Docker was always a brittle tool with lots of limitations and huge compromisses.
Want to ${FOO_PORT:-1234}? Forget about it. Running random stuff as root? Why not?
It also made very easy to consume and produce junk/snowflakes.
Every docker image place its files/configuration/data in a special place. FHS? Who need that?
People have been running, packaging and distributing software more efficiently and more secure way before docker even existed.
Another tool that made existing technology more popular is Kubernetes.
But it did a better job in this "container" space.
You replied by talking about your perceptions of Docker having a bad technical design.
These two points are entirely orthagonal, which is why I'd guess your comment got downvoted.
It is frequently the case that the most popular technology is not the one with the "best" technical design.
For example the popularity of JavaScript or PHP. Neither is one that is regarded generally to have good technical design, however they have had a huge impact.
Thanks for your comment, I appreciate that! Both tools you mentioned are great and kinda relate to this experiment too.
The existence of docker and how it was developed and absorved by people brought nice things too. The kernel had to workout some areas.
The tool itseft is ok. I use something like this:
.
|-- Dockerfile
|-- README.md
`-- rootfs
|-- etc
| `-- sample
| `-- sample.conf
`-- usr
`-- bin
`-- sample
But I'm not talking about that.
For me is just interesting, you know, the tool itself doesn't matter, one can do that with any tool.
This experiment was to measure how humans entering an existing field will react... just like language has been shaped.
It was a bit sad that nobody was curious enough to talk about ${FOO_PORT:-1234} and why we should deprecate (links) or about "defaults". It was a success, though.
It's unfortunate they didn't figure out a way to make money off the best thing to happen to building and deploying software in 20 years.
A lot of other people did, from the startups now selling value-adds to Kubernetes like Kong and Tigera and TwistLock (since acquired) and others, to the public clouds which all offer Docker-based build services, image registries, and PaaS deployment tooling, to Kubernetes itself which for most users today still relies on Docker.