News for hackers. Stuff that matters.
Re:A Decade of Container Control at Google by jamesblonde on Sunday March 27, 2016 @07:23PM (#11371956)
I tend to agree. It's another case of - here is an API to access our products so you can play around with it. The scalable, powerful scheduler, however is not included. There are no other batteries that fit it, apart from GCE. Kind of like Blaze and TensorFlow - nice and shiny, but hollow (missing the good distributed filling).
Re:A Decade of Container Control at Google by SEJeff on Sunday April 10, 2016 @06:13PM (#11468432)
FWIW, you can plug mesos in as a Kubernetes scheduler. From the OSS world, that is about as heavy duty as you're going to get and proven on quite large (although not google large) 10k+ node clusters.
Re:A Decade of Container Control at Google by skj on Sunday March 27, 2016 @08:46PM (#11372134)
The internal google scheduling system is simply not appropriate for anyone other than Google to use. A huge number of machines, all alike, with a huge number of trusted binaries that can be multiplexed onto these machines without fear that they're going to break the jail and cause havoc (since there is a solid trail from source to running artifact). It's just not the reality that other companies exist in.
Re:A Decade of Container Control at Google by justicezyx on Sunday March 27, 2016 @11:32PM (#11372576)
Scheduler is actually one relatively simple piece of the whole picture. If scheduling is a pain, then Kubernetes would just addressed. The fact that Kubernetes did not choose scheduler, means that it is actually not a big problem, at least not the biggest one.
Re:A Decade of Container Control at Google by stonogo on Monday March 28, 2016 @11:09AM (#11374688)
That is not an argument against releasing the code. Why would Google assign itself as gatekeeper? I personally could use this code on supercomputers now. I don't work for a supercomputing company; my use case is academic work and computational science. I absolutely have thousands of huge machines that I multiplex trusted binaries to -- and scheduling is not a trivial problem.

So what's the real reason nobody gets to see this code?

Re:A Decade of Container Control at Google by thockingoog on Monday March 28, 2016 @12:15PM (#11375122)
The code is literally millions of LoC, all of which needs to be audited for stuff we can't release for whatever reasons. All that code is built upon layer after layer of Google internal stuff. Open-sourcing Borg means open-sourcing Chubby, internal form of gRPC (older), hundreds of libs, monitoring infrastructure, etc. Net result is O(50M) LoC. And when someone sends us a patch - then what? The cost of doing it is simply prohibitive. I'd love to do it, it's just not practical and has no RoI.
Re:A Decade of Container Control at Google by stonogo on Tuesday March 29, 2016 @12:30AM (#11379498)
That's a much more sensible reason! If the code is truly Google specific, then I agre. It sounded to me like the code was not released because nobody else has a lot of computers, which I found odd.

Thank you for the details!

Re:A Decade of Container Control at Google by mike_hearn on Monday March 28, 2016 @08:46AM (#11373908)
Borg jobs are not trusted! The system sandboxes them, prevents them spying on other jobs data files, and assumes they might abuse system resources in arbitrary ways. The days when all Google machines trusted all Google employees is long in the past.
Re:A Decade of Container Control at Google by thockingoog on Monday March 28, 2016 @12:17AM (#11372702)
As for "no other batteries that fit it" - I am confused. We do run on AWS, OpenStack, and other cloud providers, as well as on bare metal. It's not like nobody is using this thing. In fact, just a finger in the wind, I'd guess the number of people using it outside of Google Cloud is several times more than people using it on Google Cloud.
Re:A Decade of Container Control at Google by thockingoog on Monday March 28, 2016 @12:13AM (#11372694)
"Powerful scheduling" is such a tiny piece of what Kubernetes does, it's funny. Yeah, Borg's scheduler is faster and more scalable and has more features. It also has 12 years of optimizations under its belt. I have 100% confidence that, should Kubernetes be around 10 years from now,this will be a non-issue.
Re:A Decade of Container Control at Google by x0rg on Monday March 28, 2016 @05:06AM (#11373320)
Sure, even though having multiple schedulers support (for different type of workloads) would be great to increase the cluster utilization, which is one selling point of such systems. I understand that Kubernetes is developed in the open and with the community but the heavy marketing as the "solution you can use now, directly from the creators of Omega" makes some people think it's ready, perfect and will fix all of their problems, but that's simply not true.
Re:A Decade of Container Control at Google by thockingoog on Monday March 28, 2016 @12:16PM (#11375133)
We have multi-scheduler support in v1.2 :)
Re:A Decade of Container Control at Google by x0rg on Monday March 28, 2016 @02:19PM (#11376146)
Great, the docs are a bit bad for that, I'll have a look at the code :-)