News for hackers. Stuff that matters.
Re:Talk of a Split from Docker by csears on Tuesday August 30, 2016 @11:02AM (#12390748)
I doubt they would ever consider it, but I think Docker Inc's best move would be to push reset for Docker 2.0:

- Fully embrace Kubernetes for orchestration

- Drop Swarm

- Roll Docker Engine back to its pre-1.12 scope

- Get on board with standardizing the image format, now

- Stop fighting Google and instead let them help you succeed

A Docker distro of Kubernetes would do very well in enterprise on-prem or private cloud environments. They already have a great developer experience. Companies will pay for support on both.

Continuing to oppose Kubernetes risks damaging the significant brand equity they've accrued as containers in production become mainstream.

Re:Talk of a Split from Docker by kozikow on Tuesday August 30, 2016 @11:43AM (#12391149)
Sadly, the current situation seems to be optimal for Docker inc, even if it is not optimal for users of Docker.

K8s makes docker commoditized. For example, I am running GKE k8s cluster and I only use docker directly to take Dockerfile and produce an image. I push images to google replacement of docker hub - google container registry. I run things locally via minikube. If GKE would be using something custom to run images I would not even notice. I am close to the point that switching from docker to rkt would be barely noticeable.

Re:Talk of a Split from Docker by mstade on Tuesday August 30, 2016 @11:55AM (#12391235)
OT: any chance you have or will do a write up on your set up and process? It sounds interesting!
Re:Talk of a Split from Docker by kozikow on Tuesday August 30, 2016 @12:04PM (#12391318)
This is pretty much what you would end up with when following official google cloud tutorials. Some time ago I started with https://cloud.google.com/python/django/container-engine. I still run django based on this tutorial in addition to a few more things.

Some things like minikube are somewhat bleeding edge, so maybe they are worth describing.

Re:Talk of a Split from Docker by felixgallo on Tuesday August 30, 2016 @11:22AM (#12390952)
Unfortunately there is no way to justify a unicorn valuation if all that you have as an asset is an open source file format.

Consider Vagrant -- like Docker, it's essentially a wrapper around other people's facilities and libraries. Mitchell and co appear to be trying to escape the trap by moving into nearby areas of real value, but there's no meaningful way to monetize Vagrant no matter how popular it is.

Re:Talk of a Split from Docker by kordless on Tuesday August 30, 2016 @11:35AM (#12391064)
> Unfortunately there is no way to justify a unicorn valuation

Then it is indeed unfortunate that Docker is building infrastructure code which is tainted by the game theory surrounding the need for a unicorn valuation. Maybe that's the problem, instead of Docker itself.

Re:Talk of a Split from Docker by sjellis on Tuesday August 30, 2016 @12:26PM (#12391523)
That is exactly the problem. Linux containerisation is done by building tooling that uses kernel-level features. You don't need a start-up with millions in VC funding to implement the basic feature-set. At the other end of the complexity scale, both flatpak and systemd-nspawn are more specialist containerisation tools, which are each largely the product of just one person.
Re:Talk of a Split from Docker by cookiecaper on Tuesday August 30, 2016 @05:30PM (#12394076)
Docker itself is part of that problem, not separate to it. They willingly accepted almost $200 million in venture capital. We all know what the VC game is. If they can't turn that $200M into a $5B+ exit, they lost.
Re:Talk of a Split from Docker by dragonwriter on Wednesday August 31, 2016 @05:29PM (#12401645)
> If they can't turn that $200M into a $5B+ exit, they lost.

Well, the VCs lost. Presumably, the people at Docker -- including the people who would do fantastically well with an exit that also made VCs happy -- are drawing paychecks, in some cases substantial ones, and will have in many cases done tolerably well (rather than have "lost") if no VC-friendly exit ever occurs.

Re:Talk of a Split from Docker by djsumdog on Tuesday August 30, 2016 @01:50PM (#12392248)
Do you remember when open source meant people just working on stuff for fun and trying to provide free as in speech alternatives to pay crap?

I feel like this is just another step into the corporate OSS world which is a far departure of what a lot of the original OSS architects envisioned.

Trying to monetize OSS often leads to rushed features and bad decisions like in most traditional non-OSS/enterprise products. If you strip off the polish and the fancy website, you should still have a usable, well documented thing that people need. That may also be a product you can sell support for or host, but it doesn't have to be.

Unfortunately, there aren't as many grants and not as much developer free time to go around anymore.

Re:Talk of a Split from Docker by bluejekyll on Tuesday August 30, 2016 @02:33PM (#12392657)
I'm not sure OSS ever meant that:

Many people believe that the spirit of the GNU Project is that you should not charge money for distributing copies of software, or that you should charge as little as possible—just enough to cover the cost. This is a misunderstanding.

Actually, we encourage people who redistribute free software to charge as much as they wish or can. If a license does not permit users to make copies and sell them, it is a nonfree license. If this seems surprising to you, please read on.

from:

https://www.gnu.org/philosophy/selling.en.html

Re:Talk of a Split from Docker by dragonwriter on Tuesday August 30, 2016 @02:43PM (#12392736)
> Actually, we encourage people who redistribute free software to charge as much as they wish or can.

What they don't mention is that, when every recipient of the software is free to redistribute it and compete against you, as much as you can rapidly becomes zero (or, at most, just enough to cover the cost of the most-efficient distributor), since free competition drives prices down to the marginal production cost.

Re:Talk of a Split from Docker by cyphar on Tuesday August 30, 2016 @05:56PM (#12394271)
Except of course, that the business model of free software is generally support-license based. Yes, someone can take your software and provide support instead of you but they usually are not as experienced as you with what you created. If they are, you should hire them.

Companies make money charging for free software all the time, and I really wish we would stop having to go through this discussion every time that free software shows up as a point. Companies have been making money from free software for more than 20-30 years at this point, and none of it required taking away the freedom of their end-users.

Re:Talk of a Split from Docker by dragonwriter on Tuesday August 30, 2016 @06:18PM (#12394420)
> Except of course, that the business model of free software is generally support-license based

Its charging for support (not support-license since the support contract isn't a license when the software is actually delivered on a Free license) because that's an alternative to charging for software, because, practically, you can't charge for Free software, for the reason discussed in GP.

> Companies make money charging for free software all the time

No, they don't. They often make money charging for ancillary services related to Free software, which their involvement in contributing to the Free software may have positioned them to provide at an advantage to competitors, but not many make money charging for Free software.

Re:Talk of a Split from Docker by cyphar on Tuesday August 30, 2016 @07:02PM (#12394669)
> Its charging for support (not support-license since the support contract isn't a license when the software is actually delivered on a Free license) because that's an alternative to charging for software, because, practically, you can't charge for Free software, for the reason discussed in GP.

The industry calls them support licenses (since generally you only support X machines running your software -- though of course you can have alternative models), so that's what I'm going to call them.

As for not being able to charge for Free Software, it is true that unless you have some value-add (preinstalling or burning to physical media) then yes, youre going to have trouble selling the bits that make up a piece of software. But then again, why does is that model taken as being the "right model" with the free software model being the "alternative". In fact, many proprietary companies have the same model (Oracle will charge you for support too). How is it a better model that you buy a set of bits and that's all you get (no promise of updates, no support if something breaks, nothing other than the 1s and 0s that make up version X.Y of the binaries you built). In fact, I'm having trouble of thinking what companies have such a model, because it's so user-hostile (even for proprietary so software).

Re:Talk of a Split from Docker by molecule on Tuesday August 30, 2016 @03:52PM (#12393308)
OSS is a corporate-friendly fork of Free Software like GNU

https://www.gnu.org/philosophy/open-source-misses-the-point....

Re:Talk of a Split from Docker by user5994461 on Tuesday August 30, 2016 @02:40PM (#12392720)
> Do you remember when open source meant people just working on stuff for fun and trying to provide free as in speech alternatives to pay crap?

OSS is a path bound to failure.

On the one hand, going OSS means the project will never have the resources because it ain't charging money. On the other hand, it will never be able to charge money because people don't want to pay for something that's advertised as free. That's a vicious circle.

The average non-crap software is taking astronomical effort to make and it's getting harder and more complex over time. A bunch of guys working for free in their spare time just don't have the resources to keep up.

P.S. When in doubt, remember that the difference between the great pyramid of Kheops and Microsoft Windows, is that Windows took more effort to make.

Re:Talk of a Split from Docker by cyphar on Tuesday August 30, 2016 @05:58PM (#12394284)
> because it ain't charging money.

Free software refers to freedom, not price[1]. Companies have been making money selling free software for more than 20-30 years at this point (even free software that is developed in the open).

[1]: https://www.gnu.org/philosophy/free-sw.html

Re:Talk of a Split from Docker by user5994461 on Saturday September 03, 2016 @06:41AM (#12418673)
> Free software refers to freedom,

Philosophically speaking, maybe. In practise though, free software are considered because of pricing and not freeness.

Bring a PAID open-source software for a comparison in front of your directors and/or fellow engineers. I did that many times, from $1000/year to $3M/year subscription.

Even the most open-source aficionados will quickly reveal that he doesn't care about open-source. All he thinks about is pricing.

Re:Talk of a Split from Docker by cyphar on Saturday September 03, 2016 @08:55AM (#12419012)
I gladly pay for free software, and donate to several groups of developers developing free software. Maybe I'm in the minority, but at least I stick to my principles -- free software for me is about freedom (I contribute to many projects that I use every day, which I wouldn't be able to do without free software). Price is irrelevant.

In addition, I work for a free software company -- everything I work on is done in the open and all of our customers have freedom when they use our software. So I really do practice what I preach, and encourage others to follow suit.

Re:Talk of a Split from Docker by cyphar on Tuesday August 30, 2016 @05:53PM (#12394249)
> open source meant people just working on stuff for fun and trying to provide free as in speech alternatives to pay crap?

You're mixing quite a few things. "Open source" never meant that, and was always geared towards corporations. Free software does mean providing "free as in speech" alternatives to things, but it never meant stopping people from charging users (you having to "pay crap").

Re:Talk of a Split from Docker by wstrange on Tuesday August 30, 2016 @11:43AM (#12391147)
I wonder if the Docker Hub could be their monetization vehicle. They become the "github" of the container world.

It may not justify the valuation they want.. which is another issue.

Re:Talk of a Split from Docker by cyphar on Tuesday August 30, 2016 @12:00PM (#12391287)
I believe that their business model is providing a proprietary fork of the Docker engine (along with a bunch of other proprietary tooling) to enterprises as well as support for it. I won't lie, it really bothers me that code I've helped write is being used in proprietary software.
Re:Talk of a Split from Docker by pavlov on Tuesday August 30, 2016 @12:18PM (#12391451)
Then don't contribute to projects whose license you don't agree with?
Re:Talk of a Split from Docker by cyphar on Tuesday August 30, 2016 @05:38PM (#12394142)
Personally I make all of the projects I make GPLv3-or-later. However, certain projects are of signficiant importance to the free software community that I consider the benefits to free software to outweigh the potential drawbacks by proprietary forks. Unfortunately, it still makes me feel unhappy about people taking my work and forking it under a proprietary license.
Re:Talk of a Split from Docker by majewsky on Tuesday August 30, 2016 @02:22PM (#12392552)
> I won't lie, it really bothers me that code I've helped write is being used in proprietary software.

Then you better don't contribute to anything that's not exactly GPL.

Re:Talk of a Split from Docker by cyphar on Tuesday August 30, 2016 @05:50PM (#12394226)
Or AGPL. This is a fair point, and I make all of the code I write GPL. Unfortunately, there is only one containerisation project that is under the GPL and only the core of it is under the GPL (LXC is [L]GPL but LXD is not). It's quite worrying that so few Golang projects (because for some reason every containerisation project is written in Golang seemingly in anger) are under the GPL.
Re:Talk of a Split from Docker by cookiecaper on Tuesday August 30, 2016 @05:32PM (#12394097)
Make it AGPL. The GPL can be circumvented by slapping an HTTP interface on the logic, which is how Google has gotten away with keeping so much private.
Re:Talk of a Split from Docker by djsumdog on Tuesday August 30, 2016 @01:51PM (#12392258)
I've written about this before:

http://penguindreams.org/blog/the-philosophy-of-open-source-...

TL;DR is that we have a lot of corporate open source now for infrastructure, but the golden age OSS end products people hoped for in the 90's/00's never really happened.

Re:Talk of a Split from Docker by cyphar on Tuesday August 30, 2016 @05:42PM (#12394168)
Note: "Open Source" does not have a philosophy[1]. The free software movement does (it has a philosophy based on software freedom), but the "open source" movement does not discuss any of the ethical reasons why users should have freedom (it only discusses practical issues). Personally I think that the "open source" split is one of the reasons that the state of free software is so worrying today.

[1]: https://www.gnu.org/philosophy/open-source-misses-the-point....

Re:Talk of a Split from Docker by hosh on Tuesday August 30, 2016 @12:08PM (#12391355)
That's a great point. The thing is, I don't see them as getting serious about it. For example, quay.io provides better functionality around registry. Docker Hub is "good enough" and gets traction from the sheer fact that Docker runs it.
Re:Talk of a Split from Docker by hosh on Tuesday August 30, 2016 @11:19AM (#12390920)
As the proverb goes, "Choose, or someone will choose for you."

rkt is getting into Kubernetes as an alternative runtime engine:

http://thenewstack.io/kubernetes-chief-we-back-docker-althou... (2015)

http://blog.kubernetes.io/2016/07/rktnetes-brings-rkt-contai... (2016)

Back then it was, "It's sad to see so much drama; Docker engine has much more traction." Remember all of that?

The story is now, "hey, rkt lets us test out our framework for supporting alternative runtime engine".

There's a shifting attitude there. Goodwill from the community is eroding. I don't know how long this window will remain open, but it won't remain open indefinitely.

Re:Talk of a Split from Docker by amouat on Tuesday August 30, 2016 @11:26AM (#12390992)
I think there's also momentum to add runc as an alternative runtime engine, which makes sense.

Also, Mesos have started shipping their own container runtime.

This feels like an inflection point in the history of containers.

Re:Talk of a Split from Docker by cyphar on Tuesday August 30, 2016 @11:50AM (#12391198)
> I think there's also momentum to add runc as an alternative runtime engine, which makes sense.

Actually, we're working on getting generic OCI runtimes supported (runC being an example of such a runtime). In principle, you should be able to swap out the OCI runtime and still get Kubernetes to work with that runtime. It's an exciting time. :D

Re:Talk of a Split from Docker by jondubois on Tuesday August 30, 2016 @04:43PM (#12393724)
Yes, I think the 'Not invented here' syndrome has reached epidemic proportions - Especially within open source communities where a lot of people create stuff from scratch just for the fun of it.

When I read the article announcing Docker v1.12, it was riddled with enthusiasm - That's a red flag - This rarely happens in our industry.

Working on open source is not like working in a startup; it's not like riding a roller coaster; it's more like riding the metro - You spend most of your time underground and when you finally arrive at your destination, the slow ascent up the escalator is rarely that exciting.

Re:Talk of a Split from Docker by ktamura on Tuesday August 30, 2016 @12:32PM (#12391573)
>A Docker distro of Kubernetes would do very well in enterprise on-prem or private cloud environments. They already have a great developer experience. Companies will pay for support on both.

Some early data here: https://twitter.com/dberkholz/status/720686568671940608

Re:Talk of a Split from Docker by dmourati on Tuesday August 30, 2016 @01:05PM (#12391858)
Interesting thing about that survey is that it covers PaaS solutions (Cloud Foundry, Kubernetes, etc) on top of OpenStack. Other folks seem to be going in the other direction using PaaS to deploy OpenStack. https://tectonic.com/openstack/ http://www.tcpcloud.eu/en/blog/2016/06/27/making-openstack-p...
Re:Talk of a Split from Docker by SEJeff on Wednesday August 31, 2016 @06:54PM (#12402086)
One word for you: Investors

They could never do that and expect to survive as a company as it would be "giving away" money on the table so to speak.

[dead]
Re:Talk of a Split from Docker by cyphar on Tuesday August 30, 2016 @11:49AM (#12391188)
Except of course considering the fact that Google Engineers probably have some experience with scalability. Not to mention that the current Kubernetes community has an incredibly open design process, and is very community-oriented. So it's very far from its origins as an ex-Googler project.
Re:Talk of a Split from Docker by cmcluck on Tuesday August 30, 2016 @12:55PM (#12391767)
Disclosure: I am one of the Google people who founded the k8s project. Product guy, not engineer though.

We are really concerned about 'writing letters from the future' to the development community and trying to sell people on a set of solutions that were designed for internet scale problems and that don't necessarily apply to the real problems that engineers have.

I spent a lot of time early on trying to figure out whether we wanted to compete with Docker, or embrace and support Docker. It was a pretty easy decision to make. We knew that Docker solved problems that we didn't have in the development domain, and Docker provided a neat set of experiences. We had 'better' container technology in the market (anyone remember LMCTFY?) but the magic was in how it was exposed to engineers and Docker put lightening in a bottle. The big things were creating a composable file system, and delivering a really strong tool chain are the obvious two, but there were others. I remember saying 'we will end up with a Betamax to Docker's VHS' and I believe that remains true.

Having said that there are a number of things that weren't obvious to people who weren't running containers in product and at scale. It is not obvious how containers judiciously scheduled can drive your resource utilization substantially north of 50% for mixed workloads. It isn't obvious how much trouble you can get into with port mapping solutions at scale if you make the wrong decisions around network design. It isn't obvious that labels based solutions are the only practical way to add semantic meaning to running systems. It isn't obvious that you need to decentralize the control model (ala replication managers) to accomplish better scale and robustness. It isn't obvious that containers are most powerful when you add co-deployment with sidecar modules and use them as resource isolation frameworks (pods vs discrete containers), etc, etc, etc. The community would have got there eventually, we just figured we could help them get there quicker given our work and having been burned by missing this first time around with Borg.

Our ambition was to find a way to connect a decade of experience to the community, and work in the open with the community to build something that solved our problems as much as outside developers problems. Omega (Borg's successor) captured some great ideas, but it certainly wasn't going the way we all hoped. Kind of classic second system problem.

Please consider K8s a legitimate attempt to find a better way to build both internal Google systems and the next wave of cloud products in the open with the community. We are aware that we don't know everything and learned a lot by working with people like Clayton Coleman from Red Hat (and hundreds of other engineers) by building something in the open. I think k8s is far better than any system we could have built by ourselves. And in the end we only wrote a little over 50% of the system. Google has contributed, but I just don't see it as a Google system at this point.

Re:Talk of a Split from Docker by brokebook on Tuesday August 30, 2016 @05:30PM (#12394079)
Thank you, a very insightful comment. FWIW I think you made a smart choice by leveraging docker and it's ecosystem (not just the tools, also mind share, the vast amount of knowledge spread via blog posts and SO, has to be considered). While docker's technical merits over competing solutions were indeed arguable then, it's UX was a game changer in this domain, everyone got the concept of a Dockerfile and "docker -d" and many embraced it quickly.

Kubernetes came around exactly at the time when people tried to start using it for something serious and things like overlay networking, service discovery, orchestration, configuration became issues. For us, k8s, already in it's early (public) days, was a godsend.

Also, i can confirm: the k8s development process is incredibly open. for a project of this scale, being able to discuss even low-level technical details at length and seeing your wishes being reflected in the code is pretty extraordinary. It will be interesting to learn how well this model will scale in terms of developer participation eventually, i figure the amount of mgmt overhead (issue triaging, code reviews) is pretty staggering already.

Re:Talk of a Split from Docker by cyphar on Tuesday August 30, 2016 @05:46PM (#12394193)
Thanks for the insightful response. I didn't mean to say that the entire project came from some shot of inspiration by some Google engineer, nor do I think of Kubernetes as a "Google system" -- I'm well aware of that separation.

The main thing that I think makes Kubernetes the perfect example of a modern free software community is the open design process and how the entire architecture of Kubernetes is open to modification. You don't see that with Docker, and you don't see it with many other large projects that have many contributiors.

Re:Talk of a Split from Docker by harlowja on Wednesday August 31, 2016 @01:55AM (#12396284)
So is there ongoing work to replace borg with K8s (afaik the adoption of omega pretty much stalled and/or died)?

I myself would feel more confidence in K8s if there was usage of K8s outside of GKE and the dogfood being created was eaten by the rest of google as well (or at least there was a publicized plan to make that happen), because otherwise it feels more like K8s is a experiment that google folks are waiting to see what happens before the rest of google adopts it.

Thoughts?

Re:Talk of a Split from Docker by mentat on Tuesday August 30, 2016 @12:27PM (#12391526)
Well, they're about a decade ahead of everyone else, at least. It's not about if you feel "tired", it just is. If you don't like it try to learn from them and do better.
Re:Talk of a Split from Docker by bertomartin on Tuesday August 30, 2016 @12:17PM (#12391442)
Agreed! If the issue is Google and other tech giants feel their orchestration tools is being threatened by swarm, too bad. I fully agree with Docker Inc here. Innovation involves risks, and if you don't appreciate that, then don't use it. However, I do agree that more attention and more thorough testing should be performed before something is stated as being "production ready". Keep innovating Docker!