At the moment, Kubelet needs to control the Docker daemon (a somewhat brittle relationship), and a bunch of concepts — from networking to volume management — could be simplified by making the container runtime more K8s-specific and less general-purpose. A tighter coupling would also mean you'd always be running K8s against the exact container logic it was made for, as opposed to now, where the loose coupling means you'll be subject to bugs (and there are so many) in new Docker versions.
That and the fact that Docker is increasingly building in competing concepts (services, etc.) via swarm mode, which I don't care for.
Docker still has a place for things like image building, of course.
Personally I think the long-term solution is for kubernetes to adopt containerd - it's the small, stable, rock-solid container runtime that it needs.
Whether the Kubernetes community continues to build primarily on Docker, or switches to containerd - either way we will support them and help in every way we can. Behind the superficial internet drama, there is a solid engineering collaboration between both communities, particularly between Docker and Google. We may disagree on a lot, but at the end of the day we have a lot of users who rely on us, and we have a responsibility to work together to solve their problems.
That's quite inaccurate. It's is like saying that OpenStack started out as an extension to KVM. Both had very different (original) scopes.
> Whether the Kubernetes community continues to build primarily on Docker, or switches to containerd
Kubernetes has 3 container runtimes at the moment: Docker, Rocket, and the new remote runtime which allows you to plug in a separate server using a protobuf interface. There is a lot of focus on the remote runtime now. The CRI-O project for example is implementing a remote runtime that allows any OCI conformant runtime to be plugged into Kubernetes.
The focus on diversifying away from Docker is very recent, and is the direct result of the competitive tension I talk about earlier in the thread. It's completely understandable.
EDIT: A perfect example of that competitive dynamic is actually this very thread! the GP (`sysexit`) works at a competing vendor. You can tell in his post that de-emphasizing the dependency on Docker is important to him. In fact that talking point is so recognizable that I knew where he worked before even looking him up. There is nothing wrong with that, every vendor has an "official party line", and Docker is no exception. The point is, this is a delicate issue which has more to do with inter-personal and business dynamics than with code. And you should definitely verify the credentials and agenda of everyone speaking of authority on the topic of containers platforms. We vendors are everywhere, and we are biased as hell.
I think you miss a reason at least equally as important, if not more: There are a lot of people who want to do non-container runtimes but still leverage the rest of Kubernetes. I don't want to say no to them (who am I to decide what tech is "good"?) but I don't want their code in my tree (too big already, too hard to maintain, too much email, etc).
Ergo: plugins.
But you're right, kub itself is a neatly generic orchestration component which can be used with non-container things. That is an excellent reason for decoupling the underlying container runtime.
As a Pivot, I can confirm: I am biased as hell.
The experience of the Garden API (a containerising system that ultimately predates Docker) is that tighter coupling is actually a mistake.
If Garden had been tightly bound to its implementation or orchestrator, it'd still be using raw Linux system calls and running under a 2nd-generation orchestrator.
Instead it's now using runC, OS X or Windows APIs to create containers and runs under multiple consumers -- most heavily under Diego, a 3rd-generation orchestrator and Concourse, a project automation tool.
Disclosure: I work for Pivotal. Many, but not all, of the engineers who've worked on Garden work for Pivotal also.
Great concept, poor execution.
Mad props.
Great! There is one question I would like to have answered, by you and you alone:
if I'm using SmartOS, and therefore zones, and all my applications and configurations are packaged into OS packages, and I use imgadm(1M), vmadm(1M), and Smart DataCenter (SDC), what do I need Docker for?
Fully isolated, virtualized UNIX servers. Servers are cattle, not pets. Deployments in up to 25 seconds.
That was the challenge. And it was fun to solve.
Have fun maintaining OS packages.
Oh I am! It's incredibly satisfying to be able to run systems where no human has ever logged into, and no file has ever been manually modified, and all the files and all the changes are accounted for by the operating system.
See also, not quite the same, but very much related and relevant to your comment:
http://www.perkin.org.uk/posts/building-packages-at-scale.ht...
https://blog.nimbleci.com/2016/10/11/how-to-deploy-jenkins-c...
https://news.ycombinator.com/item?id=12750685
Configuration management at scale, using an operating system's software management subsystem is the old, new-new thing. It is the foundation and the future of even more massive cloud computing. Now you'll be able to say someday in the future: "ha! I read that on Hacker News years ago!", and point to this post.
Thank you for saying this! Building OS packages and OS images and provisioning servers was done automatically 10 to 12 years ago with much less work.
It's still the preferred choice for many large tech companies and nobody talks about it because it just works and it's nothing new. Fast deployments, immutable infrastructure, modularity.
Anyway, someday in the future I might look back on this and think "ha! someone put all that effort in even though NixOS already existed".
All I did was ask a question.
Is critical thinking, even just asking questions, undesirable now, unless someone is being praised to high heavens? Is that what we're down to now?
"ha! someone put all that effort in even though NixOS already existed"
SmartOS virtualization is based on Solaris zones, which existed long before people on GNU/Linux even realized that containers were the way to go. The groundwork, and my own work was laid down long before any of this became hype. Project Kevlar, nee zones was started around the same time as NixOS, and another participant already mentioned that large companies did large scale deployments and configuration management via OS packages, which is both true and correct.
Docker is just a simple wrapper that reduces the amount of knowledge needed to run a containerized process. It did not become hype because of any technological advantage, it became hype because this old technology became accessible not just for the kind of people/companies that don't (can't) run enterprise class Solaris deployments, but also the kind that don't have 20 years of experience in making robust well managed deployments.
Your process might seem easy and straightforward to you. But for the average developer and Linux enthousiast, there's not much that can the simplicity of the Dockerfile. Systemd or a more specialized system like Kubernetes to run the containers, and a simple script to provision your machine is all you need. You can pick all of this tech up in a few days and steadily grow into more advanced subjects like monitoring, logging, security.
Anyway, you're not helping anyone by just saying "SmartOS is better than Docker+(list of redundant and unrelated other tech)", maybe instead show us a one-page tutorial on how to set up a SmartOS system with a webapp stack (i.e. LAMP, MEAN or RoR) that can be executed by a newbie without a creditcard, registration, trial or phone call.
Yes I did.
You asked a question leading into a lecture on the technology you prefer.
Only after you left your Good luck with packaging everything, which I have good reason to believe was cynical sarcasm.
It did not become hype because of any technological advantage, it became hype because this old technology became accessible not just for the kind of people/companies that don't (can't) run enterprise class Solaris deployments,
Performing configuration management at scale with OS packaging has nothing to do with Solaris - it works on any operating system which has a Bourne-family shell, AWK, and sed.
but also the kind that don't have 20 years of experience in making robust well managed deployments.
Then it would be a good idea for such people to actively seek mentors.
But for the average developer and Linux enthousiast, there's not much that can the simplicity of the Dockerfile.
That's what distinguishes the average developer. A true enthusiast will go out of their way to become a master of the art, so the assertion doesn't apply to them.
It's easy to hack several hundred thousand files together and deliver them via Docker. But, what happens when it's time for upgrades? Go through the entire hackfest again? Every time?
Anyway, you're not helping anyone
that depends on what your mentality is: if you're a true enthusiast, you're probably reading a lot, and when you stumble upon crumb trails like mine, the light bulb might go off reading what I wrote, and it might just be the clue you need (that's the idea). And you wouldn't be the first or the last person that happened to.
As for me, I'd gladly show how to do what interests you, only I have no blog, and no time to set one up, fight spam, hack attempts, and so on. I can afford one short answer like this, but I cannot afford months of engineering implementing the infrastructure for a blog and hardening it.
And by the way, nice stab at Joyent on your part, but SmartOS is free, open source software, and the entire ecosystem + the 14,000 packages never cost a penny. Even their cloud on a laptop ("COAL") preconfigured image is gratis.
As for packaging being hard... there is such an abundance of packaging documentation for any popular operating system... all you have to do is sit down and "warm up the chair" by reading it. Do you think I learned it some other way?
which I have good reason to believe was cynical sarcasm.
Then it would be a good idea for such people to actively seek mentors.
Do you think I learned it some other way?
Gladly accepted.
I have done some packaging for Debian, CentOS and recently Alpine and whilst the systems are smartly designed and effective, using them is neither easy nor enjoyable.
I cannot write anything of Alpine Linux, as I have never used it, but I have done Debian / Ubuntu packaging and configuration management with packages, and that, compared to RHEL / SLES / SmartOS / Solaris / HP-UX / IRIX, was the hardest to get right.
One of the reasons why Debian and Ubuntu are so hard to get right is that the Debian packaging guidelines are insane: for example, it is not allowed to have packages named in any way other than lowercase; there is no /opt, as the Debian people do not understand the filesystem hierarchy standard and consider it a hindrance, for reasons completely unknown to me.
So now imagine you had to build, say, your own PHP with OCI8 support: you cannot name the package php, nor can you deliver it into /usr, because the next upgrade from Debian could break your production. And there is no /opt, so neither vendors nor other 3rd parties have a place to safely deliver their software (/usr/local is not it, and is against standards).
What I'm trying to say is, you picked one of the worst operating systems, ever. Even RHEL, as bad as it is, is nowhere nearly as poorly designed as Debian GNU/Linux.
Pick a better OS. pkgsrc format is pretty straightforward, and AT&T SVR4 packaging[1] is the absolute best when it comes to doing configuration management. Unbeatable.
This might not be your cup of tea, obviously it wastes system resources, but it does simplify the ordeal.
If delivering software which JustWorks(SM) is an ordeal, then you need to step back and seriously re-think what needs to be corrected. Delivering software should be a breeze, and it should happen automatically, meaning that the software management subsystem of the OS should worry about it for you. But that also requires you to seriously revise how you put software together, because if it's an ordeal for you, something is majorly wrong. (Since I don't know your situation, I cannot comment what.)
Or they could use Docker and focus their energy on building value for their company so at some point it can afford to employ proper system administrators. Who then may or may not decide to migrate the systems to SmartOS.
This line of thinking, is, in my experience, completely, utterly incorrect on several levels:
developers should not act as if system administration is beneath them. One can never, ever become a master developer without becoming a top-notch system administrator: how can anyone develop software which is correctly integrated and runs on an operating system, if they do not understand how to administer that system? I have seen that over and over and over again in my decades of experience, and I have seen no developer who was able to code quality software which the system administrators didn't hate, because it was all hacked up and required lots of hacking to get working.
I just think that if knowledge-intensive engineering can be avoided it is worthwhile. Especially when an operation is young.
If you do that, it will come back to haunt you. With a vengeance. And the reason it will come back to haunt the company is because it's taking on a huge technical debt.
Are you familiar with the concept of taking on technical debt?
> What is the motivation and benefit for running containers without docker?
I regularly hear from people who want to run containers without Docker. There are several motivations, all of which are perfectly valid:
1. Learning. It's fun to build things from scratch to understand how they work under the hood.
2. Bad experience. Early versions of Docker were quite buggy, and we initially struggled to keep up with the colossal growth in usage and feature requests. As a result, many of the people who tried Docker in production too early were badly disappointed. Some of them decided Docker wasn't for them, and started looking for alternatives.
3. Extremely custom use case. If your deployment is larger, or more complex, or more specialized than 99.99% of deployments out there, then "mainstream" platforms like Docker might not be the right fit for you. Of course we try to make Docker as customizable as possible, to support more a wider spectrum of use cases with plugins. But realistically, no single platform can cover all use cases, and I don't think any platform ever will. Docker is no exception.
4. Philosophy disagreement. Different people have different opinions on how applications should be developed and deployed. Docker tries very hard to be agnostic - to accommodate as many opinions as possible. But we can't please everybody. If Docker does not fit your philosophy of development and deployment, then the natural response is to look for an alternative.
5. Competition. Many Docker competitors started out as extensions or modifications of Docker, and over time are looking to reduce their dependency on Docker.
I'm probably missing other reasons, but these are the ones I've been most exposed to. These are all reasonable reasons to not want to use Docker.
Our approach is that, if you want to run containers without Docker, we should make it as easy as possible. In practice that means spinning out as many of the underlying components as possible (what we call the "plumbing") so that you can assemble it yourself without being stuck using the entire Docker platform. For example:
- containerd [https://github.com/docker/containerd] is our low-level container runtime.
- runc [https://runc.io] is a standardized "container executor", which we donated to the Linux Foundation as the reference foundation for the OCI spec.
- libnetwork [https://github.com/docker/libnetwork] is the low-level networking implementation (including overlay networking which is a very useful primitive for container clustering)
- swarmkit [https://github.com/docker/swarmkit] is a clustering/orchestration implementation
- notary [https://github.com/docker/notary] is a cryptographic content verification tool, which you can use to sign and verify container images.
- infrakit [https://github.com/docker/infrakit] automates the provisioning of infrastructure capable of running containers.
Our opinion is that, even if you don't use Docker, by using these components for your own purposes, you are indirectly contributing to making Docker better. Splitting out these components has also forced us to refactor Docker into a more modular, more robust design.
We even send Docker employees to explain how to run containers without Docker :) For example here's a talk we gave at Linuxcon: https://linuxconcontainerconeurope2016.sched.org/event/7oHM/...