News for hackers. Stuff that matters.
Re:Google embraces, extends, and extinguishes by chatmasta on Friday May 04, 2018 @08:53AM (#16994174)
Hey, I was just talking about this in a comment the other day! I totally agree with the author here. I was surprised not to see any mention of cloud facing open source products. Kubernetes seems like a prime example. Google lost its containerization first mover advantage to Docker (remember google was famous early for its internal use of containers). So they targeted the next level up the stack of container orchestration. Kubernetes is a great project, but its clear priority is compatibility with the cloud computing business model. It didn’t take long for google to offer GKE on top of kubernetes. At least for now, there is feature parity between open source and hosted versions. But I don’t expect that to last long. I would be wary of the kubernetes project introducing features that only work well in google’s cloud.

The main externality of this open source EEE strategy is developer opportunity cost. When google swept into the chat market, they used their size and first mover advantage to push an open standard, XMPP. So developers who might have worked on other projects/standards naturally gravitated toward XMPP and contributed their time there. If those developers had known the future of XMPP, they likely would not have worked on it. They could have invested time in a competing, more open project instead.

This to me is the real danger. It’s bad enough that google gets to benefit from thousands of free contributors to its “open source” code. It’s even worse that they are taking those developers away from other projects, so that when google moves to the third phase of EEE, there are no viable alternatives.

Re:Google embraces, extends, and extinguishes by bryanlarsen on Friday May 04, 2018 @09:20AM (#16994356)
IMO Kubernetes is safe. It has way too many outside contributors for Google to pull it back in. And it's going in the opposite direction as you describe: networking, storage and ingress used to be considerably more clunky on non-GKE based installations, but now we're at the stage where people are starting to use third-party ingress and storage plugins on GKE because those have features that the Google implementation doesn't have.

And Google is probably ecstatic. It's disrupting the AWS monopoly that was developing, tons of outside developers are improving GKE for them, and it's running a textbook "commoditize your complements" play.

Compare that with Android, which Google is successfully doing the EEE strategy on. Despite widespread usage, it has few outside contributors, so Google is bearing all of the costs of it being open source without reaping few of the benefits.

But that's Google's fault. It's hard to build and contribute to Android. Development is done privately and thrown over the wall.

OTOH, it's easy to contribute to Kubernetes, and it's done in the open.

Re:Google embraces, extends, and extinguishes by scrollaway on Friday May 04, 2018 @09:24AM (#16994391)
Posted further down, but I'll move it here since it's more relevant to android's development:

So a few years ago I created an open source community in gaming (which coincidentally, the author of this blog post was a part of early on). We had a Github organization, an IRC channel, etc. It was fun.

At some point I grew part of the community into a company, which today is growing and thriving. We're rooted in open source. I'm rooted in open source (Nearly all my work has been public and on Github for the past decade).

Today we still have nearly all our work up and public on Github but get next to zero external contributions. Because contributing as someone external to the company is hard.

Unless your actual goal as the company is to produce free & open source software, it's going to be really, really hard to make it possible for external actors to contribute to the core products as smoothly as for the company's employees. There's loads of things that pile up, like reusing internal CI systems for testing, requiring API keys and passwords for access to resources that are required or highly useful for development (eg. testing a redshift environment), and so on. All those things are fixable, but take a huge amount of time to fix, and why would you fix it if nearly nobody external to the company is contributing anyway? It's not like making the experience easy is a guaranteed win either.

Things like product design, project management, etc... it's not obvious how to make those open processes. It's not like you can do part time project management contributions to any random open source project out there. And when the processes aren't open, the barrier of entry to contribution is that much higher.

And quite frankly, certain people (who will recognize themselves) make it extremely unappealing to spend a lot of efforts on keeping the entire project open. The kind of people who will complain about every little detail of how you're trying to open up the project, until its name is prefixed with GNU/. "All or nothing" is an atrocious approach to open source and actively harms the ecosystem.

Point is, don't blame Google for making Android and Chromium hard to contribute to. It's not like it's on purpose, it just takes a huge investment for it to be possible. I mean fuck, Firefox isn't exactly easy to contribute to, and Mozilla is one of the most open companies out there, and in many ways a role model of how an open source company should be run.

Re:Google embraces, extends, and extinguishes by jcranmer on Friday May 04, 2018 @11:19AM (#16995218)
There's a simpler way to put this:

In open source, there's invariably an "in-track" and an "out-track." The in-track consists of people who will get their concerns pretty quickly addressed and can make their patches known and reviewed fairly quickly. The out-track is those who can't do that.

The strengths and weaknesses of open-source models is in how easy it is to make it into the in-track, and how stark the in/out dichotomy is. My experience with Mozilla is that they do a fairly good job at speeding people from out-track to in-track (although I suppose it depends on what areas you work in--Thunderbird was ridiculously easily, but I can imagine Gecko or SpiderMonkey is going to be much harder), but the in/out dichotomy remains somewhat stark. I've seen other places, like LLVM, where the in-track is much harder to break into, but the in/out dichotomy is not so stark.

Re:Google embraces, extends, and extinguishes by bryanlarsen on Friday May 04, 2018 @10:09AM (#16994706)
Kubernetes is relatively easy to contribute to, but that almost seems incidental. IMO The reason why Kubernetes sees lots of outside contribution is because Red Hat, Microsoft & IBM et al recognize that it's in their best interest to contribute to Kubernetes and to apply technical and business pressure to ensure that Kubernetes doesn't become a silo.

I have no idea why this didn't happen for Android. Samsung & LG have spent a fortune on Tizen & WebOS, and would have been a lot better off with a truly open Android. Whether that's their fault or Google's is hard to tell, but I suspect that answer is "both".

Re:Google embraces, extends, and extinguishes by nojvek on Friday May 04, 2018 @01:02PM (#16996142)
Chromium has a ton of external contributors. Have you read the AUTHORS list? Sure, it’s not all on GitHub but a number of its parts are.

It’s a pretty hackable project.

Re:Google embraces, extends, and extinguishes by pgeorgi on Friday May 04, 2018 @10:09AM (#16994704)
> certain people (who will recognize themselves)

They probably won't.

Re:Google embraces, extends, and extinguishes by jchw on Friday May 04, 2018 @09:17AM (#16994336)
Call me naive but I disagree with the points about Kubernetes. While they did make Kubernetes specifically for their GKE offering, it was before Docker Swarm even existed, so it didn't directly compete with Docker. I doubt Swarm was doomed to failure; more, it felt like the priorities were off (it was simpler than Kubernetes at the cost of being less useful.)

Further, Google has been taking the Google bits out of Kubernetes. They declined to call their own offering "Google Kubernetes Engine" officially until the playing field was level and they had a certification program. Amazon, Azure, DigitalOcean and others are implementing their own managed Kubernetes solutions.

Google doesn't have to cheat here. Speaking as someone dealing with Google Cloud directly, they are mainly hitting on the point that Google Cloud offers better network and disk performance, in a more flexible manner, for significantly less money. And honest to God they're right. The CPU performance is worse in some cases, but we're saving money on workloads so far.

On the flip side, as long time AWS users, the Kubernetes experience has been pretty good since Kubernetes 1.3. New features typically support AWS pretty early in, with the main exception being Ingress controllers (for good reason: ALBs weren't even released when that became a thing.)

Google is one of those companies big enough to make it easy to feel like they have evil ulterior motives, but I'll be honest: in my dealings with them, I've felt that the opposite is true and generally the intentions are good.

Re:Google embraces, extends, and extinguishes by scarface74 on Friday May 04, 2018 @09:22AM (#16994372)
I wouldn't trust my business to Google Cloud Platform because they have a habit of abandoning services that aren't related to their core business - search and ads.

I have more confidence in Amazon and Microsoft.

Re:Google embraces, extends, and extinguishes by jacksmith21006 on Friday May 04, 2018 @09:24AM (#16994393)
Really? Do not think they would walk away from over $4B of business a year.

Google sticks with things that are core. Google reader was obviously not core.

Glasses moved to enterprise and continue to be used in that setting.

Re:Google embraces, extends, and extinguishes by scarface74 on Friday May 04, 2018 @10:15AM (#16994738)
It's not about abandonment as much as it is about focusing on it. Take Apple as an example, the Mac is an $6 billion+ A quarter business for Apple but only 11% of their business. No one would say that Apple focuses on the Mac.
Re:Google embraces, extends, and extinguishes by jacksmith21006 on Friday May 04, 2018 @02:08PM (#16996785)
Google is really focused on GCP and suspect they will continue to be. Constantly adding new capabilities.
Re:Google embraces, extends, and extinguishes by jchw on Friday May 04, 2018 @09:26AM (#16994418)
I'm sorry you feel that way, because Google Cloud is really nice and if nothing else, it provokes a lot of thought. It's worth evaluating in my opinion.

I'm not saying it's better than other offerings - It's not all around - but the building blocks are really nice.

Re:Google embraces, extends, and extinguishes by scarface74 on Friday May 04, 2018 @09:42AM (#16994529)
It's not about being "better". It's about trusting Google to have the same focus as Amazon and Microsoft. Besides, no one ever got fired for choosing AWS....
Re:Google embraces, extends, and extinguishes by jchw on Friday May 04, 2018 @09:52AM (#16994596)
What I'm saying isn't "migrate a critical part of your workload to Google" - just suggesting that it's worth evaluating Google Cloud, merely for your own reference, even if you are sure you aren't going to use it. They offer a pretty good free trial to do so.
Re:Google embraces, extends, and extinguishes by scarface74 on Friday May 04, 2018 @10:10AM (#16994710)
If I were starting a business in 2018 or consulting with someone who was doing an infrastructure migration, I couldn't in good conscience tell them to go to GCP. The support infrastructure isn't as good - ie people who have experience.

There is a lot to be said about choosing the default. I've been a .Net developer for over a decade, but when I started looking into Lambda for instance and saw that it looks like most people were using Node and to a lesser extent Python, I begrudgingly started learning Python. In hindsight, I'm glad I did. I'm actually starting to like Python and I usually hate scripting languages.

Besides, for my own selfish reasons, why would I spend time on the cloud platform that is the least marketable? Companies are asking for AWS and to a lesser extent Azure experience. Very few are asking for GCP experience.

Re:Google embraces, extends, and extinguishes by 0xEFF on Friday May 04, 2018 @11:40AM (#16995407)
Fifteen years ago this was the same with Linux. Everyone was asking for MCSE certified professionals. And yet, my Linux experience has been invaluable over the years and supply and demand for highly specialized skills have worked to my advantage.

If everyone is going one way, it's not necessarily a bad decision to take a slightly different path. I've been able to build a career and a business on the alternate path.

Re:Google embraces, extends, and extinguishes by scarface74 on Friday May 04, 2018 @11:57AM (#16995560)
I don't remember a time where some variant of Unix wasn't the preferred server operating system.
Re:Google embraces, extends, and extinguishes by pjmlp on Friday May 04, 2018 @01:47PM (#16996579)
Well, I remember when servers were some variation of VAX, IBM and Unisys mainframes and most small business were happy using Novell Netware.

During those days UNIX based systems were just yet another option, mostly on government and university departments.

I learned UNIX thanks to our OS teacher carrying around a single PC tower (remember those?) with Xenix on it, where we would take 30m turns to try out our C programs, prepared in advance with Turbo C on MS-DOS.

80's and early 90's, before Linux and BSD actually took off as a free way (free beer) to get access to UNIX.

Re:Google embraces, extends, and extinguishes by bryanlarsen on Friday May 04, 2018 @11:05AM (#16995080)
Kubernetes experience is very marketable, I believe.
Re:Google embraces, extends, and extinguishes by scarface74 on Friday May 04, 2018 @11:34AM (#16995358)
Kubernetes isn't tied to hosting on Google' cloud. Amazon offers EKS and Microsoft offers a hosted service.
Re:Google embraces, extends, and extinguishes by bryanlarsen on Friday May 04, 2018 @12:55PM (#16996076)
Which is why nobody is advertising for GCP skills and why Kubernetes skills are going to be much more valuable than AWS and Azure skills.
Re:Google embraces, extends, and extinguishes by scarface74 on Friday May 04, 2018 @01:14PM (#16996262)
Kubernetes is just a really small part of knowing AWS infrastructure. I would go as far as saying that k8s is just another level of complexity that most implementations don't need. Serverless Docker solutions like AWS Fargate and native AWS services on top of it are a lot easier to set up and maintain.
Re:Google embraces, extends, and extinguishes by jacksmith21006 on Friday May 04, 2018 @09:06AM (#16994257)
Google completely gave away K8s including even the brand. So not sure what you think Google will do?

They are giving away everything they learned with Borg and you somehow take it as a negative? Really?

Do think from a business standpoint not sure why Google would give up the K8s brand but it is good for everyone.

I really worry that Google will stop opens sourcing and more important sharing so much through papers when people b*tch so much.

Starting to be hard to see why they even do it any longer. Who else have a huge competitive advantage with something like Borg and just gives it away to the industry?

But I am someone super curious and just probably spend more time on

https://research.google.com/pubs/papers.html

Read most of the paper. But then what they presented at NIPS. My favorite paper right now is

https://arxiv.org/abs/1712.01208

Bigger example is how much they give away in terms of AI/ML.

Re:Google embraces, extends, and extinguishes by hellepardo on Friday May 04, 2018 @09:11AM (#16994295)
> Bigger example is how much they give away in terms of AI/ML.

You think they are publishing all these papers and open-sourcing TensorFlow for the good of the community solely?

No... if the entire ML community publishes papers with TensorFlow as code, then now Google can sell that many more TPUs and that much more access to their cloud systems to run the models...

Re:Google embraces, extends, and extinguishes by jacksmith21006 on Friday May 04, 2018 @09:15AM (#16994324)
No not solely. But often it is mutual benefit. But the one that seems most crazy is giving away Borg.

I was shocked they gave away the K8s brand. Now that was ballsy and surprised necessary.

Re:Google embraces, extends, and extinguishes by LamaOfRuin on Friday May 04, 2018 @09:17AM (#16994338)
>Do think from a business standpoint not sure why Google would give up the K8s brand but it is good for everyone.

Because with Borg they only had an advantage with their own products' operations. That's nice, in that they are able to maintain a decent amount of technical superiority and have their product reliability be leading or tied for first, but it's not a clear line between this and them generating more revenue. They want their cloud offering to compete with AWS and Azure, which is why they want K8s to gain mindshare. GCE has huge potential for growth and significant switching costs; meaning they need to get new products on it sooner rather than later, and developer mindshare is the way to do that.

Re:Google embraces, extends, and extinguishes by jacksmith21006 on Friday May 04, 2018 @09:26AM (#16994410)
I listen to a interview with a higher level Googler that explained why and it made a ton of sense to me.

They were tired of writing something up as a paper and then someone implemented as OSS and then companies and engineers knew the OSS implementation instead of their own.

So instead do a OSS version and then drive what everyone learns.

I found this a bit funny. One solution would have been to not write the papers. But so glad Google does. Spend a lot of time on their research site.

https://research.google.com/pubs/papers.html

Re:Google embraces, extends, and extinguishes by pjmlp on Friday May 04, 2018 @10:56AM (#16994999)
Still not as much as Microsoft Research.
Re:Google embraces, extends, and extinguishes by cromwellian on Friday May 04, 2018 @11:59AM (#16995591)
Quantity != Quality though. Arguably, a lot of Google published research papers have spawned entire software categories.

The original papers on Map/Reduce, Bigtable, Dremel, Dapper, Spanner, directly spawned a bunch of successful open source implementation projects as well as companies.

Re:Google embraces, extends, and extinguishes by jacksmith21006 on Friday May 04, 2018 @02:09PM (#16996794)
Google papers have fundamentally changed the industry.
Re:Google embraces, extends, and extinguishes by hellepardo on Friday May 04, 2018 @09:09AM (#16994281)
I have always felt that, in a "typical" open-source project, I could approach the community, join it, build some rapport, and eventually help with the decision-making process (assuming I was any good and people in the community liked me).

But with Google-run (or other company-run) open-source projects, it's a completely different story. Want to be a TensorFlow maintainer? Great, sure, you can contribute, but you'll never get to make any decisions about the direction of the project itself unless you're inside Google.

To me it's a disappointing semi-abuse of the term "open source"... it doesn't mean what I think it should...

Re:Google embraces, extends, and extinguishes by cromwellian on Friday May 04, 2018 @11:53AM (#16995521)
This is true of any large project. Do you think you’ll be able join Linux and make decisions about Linux kernel architecture?

The larger and more important a project, the more bureaucratic it will be, and the more people you’ll be competitive with for attention.

Re:Google embraces, extends, and extinguishes by golergka on Friday May 04, 2018 @09:23AM (#16994387)
> assuming I was any good and people in the community liked me

...you could always find other people who think the same way and create a fork.

Re:Google embraces, extends, and extinguishes by FLUX-YOU on Friday May 04, 2018 @09:15AM (#16994317)
If you are good enough to contribute to Tensorflow and be able to come up with new directions, why not just get a job at Google so you can do that?

This isn't unlike any other open source project ran by people who don't want to consider all input from others. Not all suggestions are worth serious consideration, after all. Open source doesn't mean I'm beholden to your every whim just because you've contributed. That's what forks are for.

Re:Google embraces, extends, and extinguishes by hellepardo on Friday May 04, 2018 @09:19AM (#16994353)
> why not just get a job at Google so you can do that?

Don't want to move to California? Can't pass the interview? Salary too low in the offer? There could be lots of reasons.

I agree with your other points, but what I'm saying is not exactly that. What I'm saying is that TensorFlow is not a community-led project in the way that other open-source projects are. It's Google-led, and if you don't like it, you can go away or fork it (...and if you fork it, it's unlikely you'll get many users given the Google marketing machine).

(The same complaints have been and are being regularly made against Red Hat, and also Fedora, given that Fedora is basically "experimental" Red Hat.)

Re:Google embraces, extends, and extinguishes by FLUX-YOU on Friday May 04, 2018 @09:39AM (#16994508)
Convince Google it's better as a community-led project. Maybe they will have some convincing counter-points about why it's better the way that it is that we're not considering. They've likely already been through this debate.

Consider if Tensorflow were half as useful as it is today if it was community-led and it was led by a shitty community.

Re:Google embraces, extends, and extinguishes by tetromino_ on Friday May 04, 2018 @10:57AM (#16995011)
You don't necessarily have to move to California - see https://www.google.com/about/locations/
Re:Google embraces, extends, and extinguishes by kogir on Friday May 04, 2018 @09:13AM (#16994303)
I don’t understand your XMPP example. Google didn’t and doesn’t decide the fate of XMPP. Most developers who targeted it only cared about where the users were, and features supported by the most popular servers and clients.

Would you have preferred that those developers reverse engineer something closed source Google developed instead? Libpurple has advanced messaging not at all, only been great for interoperability.

Re:Google embraces, extends, and extinguishes by dontchooseanick on Friday May 04, 2018 @12:00PM (#16995602)
I've worked on XMPP.

Google talk / now hangouts once was an XMPP pusher. XMPP is decentralized. As with mail servers you can advertise your own XMPP servers, see https://wiki.xmpp.org/web/SRV_Records , you can even publish a per-user DNS record with an OTR key.

I shut down my XMPP endpoint a few monthes ago because Google had stopped supporting XMPP bridging to hangouts. Now that everybody uses hangouts (coders) or slack (teams/corporates) or whatsapp/messenger/skype (everyone else) there is no incentive to support it anymore (I speak now only IRC with coders)

They gave their support and a lot of traction to the protocol, and removed it on a second step (once advocates of XMPP, _yes, like me_ said to everyone, "yes you should prefer gmail+talk, they are supporting open standards")

Looks like an approaching-with-good-intentions and then disrupting strategy to me. Even if you disagree on the responsibility of XMPP rise and fall, they are the bull in a china shop : don't drink the kool aid, you'll end up broken.

Re:Google embraces, extends, and extinguishes by jononor on Friday May 04, 2018 @01:40PM (#16996499)
Gmail/Google was one of the biggest platforms that supported XMPP. Then they removed it completely, with of course made supporting XMPP less meaningful/attractive for others.
Re:Google embraces, extends, and extinguishes by seba_dos1 on Saturday May 05, 2018 @01:46PM (#17002817)
Uhm... Google wasn't the first to use XMPP. XMPP is also as open as a project can be. It's still going strong, providing a modern, mobile friendly IM protocol - it just happens without Google now. XMPP never was a Google project of any kind. They just played EEE by having the biggest server and making it incompatible with the rest of the network. That's a completely different story that what it seems from your post - no external development happened to progress Google's service in any way.

Google backstabbed XMPP, sure, but it's healing pretty well. I don't see how any time spend by developers on XMPP is wasted just because Google backed off. They took their userbase with them, sure, but if you want modern, open, federated IM network, XMPP is a sane choice. The only proper alternative these days is Matrix, but that's more of an IRC replacement.