There's even an open source cloud: https://www.openstack.org/ This is being used by some large companies (e.g. HP), and can also be used by companies who want to run a VM setup on their own hardware like they probably already do with VMware, but want a cloud-like API for it.
At the moment, there's a fair amount of diversity out there for things like:
- server hardware, network hardware, etc
- specific linux distribution
- webserver "brand" (apache, nginx, etc) and version
- database choices (oracle, postgres, mysql, etc) and versions
- application server level (java/j2ee/tomcat, node, python, php, etc)
I would argue that most of that diversity is currently driven by non-cloud applications. You do, of course, have choices in the cloud as well. However, they all have a lower friction path that would naturally consolidate the current broad array of choices down to fewer solutions.
The hard bit is trying to visualize if the majority of non-cloud solutions are eventually going to move to the cloud, or will there always be a significantly large non-cloud solution set out there? There's also the question of how many of the cloud customers would leverage the providers' default tech stack choices versus customers making their own choices.
This has been promised, at least, since the 90s. So far its really not happening. If anything the exact opposite is.
>There needs to be a protocol to decentralize social the way bitcoin decentralized money.
In the currency wars, bitcoin isn't even a player. Its quite the niche, like FB alternative diaspora*. Heck, by now we should all have local copies of wikipedia and only do nightly diffs right? Everyone wants their own personal decentralized copy of wikipedia, right?
I think centralization is not only going to be popular due to cost savings and ease of management, but the natural way to go for information technology. Complex systems often consolidate balkanized parts into larger wholes. This is also why we have ~200 countries instead of tens of thousands of city-states. Or why so many markets in technology break down into natural monopolies/duopolies.
The network effect, branding, lock-in, and minimizing customer confusion are fairly significant forces here on the customer end. Not to mention, the ability for large organizations to buy up smaller ones and also having the financial resources to soak up all the hot talent. From the abuse end of things, there's defeating smaller competitors with crony capitalism like patent abuse or predatory pricing, both of which larger entities have an easier time of doing.
I actually see this as a consequence of technology. Before git, version control was mostly centralized. Before Linux, WebKit and open source explosion, the OSes and browsers were manufactured by one or two main companies, and compared to what we expect now, the IE monopoly and other siloed monopoly products seemed terrible. I predict facebook will seem that way in the future when we have choice where to host our own social node, and which plugins to install.
There is a lot of power imbalance with centralization. The consolidation of telcos led to the net neutrality debate, but the problem is central control by huge players. Amazon can squeeze publishers. Apple rules app developers with an iron fist. Twitter caused a ruckus knocking companies off their playform whenever they felt like it. Google can disconnect a website and have its traffic plummet. Etc etc.
I think that when the technology exists, decentralization helps the consumer, through competition and raising their expectations.
In a capitalistic market, natural monopolies and oligarchies will form. So long as competitors can enter (i.e. arent legislated out) of the market there's nothing wrong.
Similarly, forcing businesses that work better at scale to become smaller will increase prices and decrease efficiency. Obviously, this is a somewhat idealized situation, so in reality it's going to be somewhere in between (because companies, governments, and people are all willing to not play "fair" in a market). Being a libertarian, I just error on the side of letting the market decide, as opposed to legislators deciding how the market should be.
The same reasons that most people prefer to "just use" gmail, facebook, amazon etc. are the same reasons that people prefer to "just live in" a state and not figure everything out themselves. So the libertarians who envision a world where people pay competing rating agencies, security agencies, firefighting companies etc. face the same challenge as those who want people to use OSS software instead of hosted platforms developed and supported by big companies. The smaller stuff will always be more rinkydink, but the competition will drive expectations of freedom and quality up for everyone, even those using the states / corporations.
Its not one or the other. These two systems can work hand-in-hand. FOSS commingling with commercial centralization has been quite the success story. FOSS on its own as PoC's or nerd toys chatted about in IRC isn't enough to get real market penetration.
https://www.ted.com/talks/clay_shirky_on_institutions_versus...
What we have is centralization in phone manufacturers when it comes to Apple, but not Android. Google tried to re-centralize by making the Play Store but Amazon and others already forked Android. In fact, Chrome is a fork of WebKit which Apple started for Safari! And that is how we had such an acceleration of features. Open source and commercial competition.
I don't deny FOSS commingles with commercial offerings. Rather I'm saying that when platforms became freely forkable, the quality AND the freedom rose dramatically. That has yet to happen in social -- as you say, Diaspora* didn't take off.
But once there is a viable FOSS social tech platform, it will disrupt all the big players. Who needs http://twitter.com/MyBrand when I can just add the Twitter module to mybrand.com !
You can also see the problems with Facebook campaigning for India to have a billion people have their internet signal go to Facebook's server farms. Besides state spy agencies, this is also simply bad engineering. Early internet apps like email worked even if email.com was down. I could use the local network.
I think we will actually soon have software that runs on social mesh networks and that's going to enable everyone in the world to share files, communicate and collaborate faster and wasting less energy than going through a central hub. You'll see a re decentralization of the internet in the next several years.
Longer term, I think Amazon's cloud will continue to eat market share, especially as an entirely new generation of techies comes online whose only experience has been working with AWS and other cloud providers. Kind of like the transition from desktop software in the 80s/90s to server-based web apps in the 90s-early 00s. If you're building a company / app today, the first move is to go to the cloud and get on AWS. That's the "safe" business move now rather than spending capital to buy hardware, getting a sysadmin, etc. etc. etc.
For the market, I'm hoping we see someone develop a set of common APIs that abstract away common specifics across AWS/Google Cloud/Azure and let you run a generic web stack and dynamically move parts of your app around based on pricing. If Google drops below AWS and your demand is spiking, you setup new instances on Google, rather than spinning up AWS instances, and use a micro service API to interact with the database. Behind the scenes, the API layer tracks the different clouds you configured for and dynamically moves most pieces of your application except data storage as your target conditions change. I envision spot cost, demand, latency, and host availability / provisioning time all playing a role in this cost algorithm. Eventually you end up with agnostic applications that don't ever live anywhere. They migrate and dynamically adjust across different clouds turning each individual cloud in part of a larger, more agnostic cloud for some services.
At an IAAS level, you can use BOSH to manage stateful systems AWS, OpenStack, vSphere, Azure and GCE. The last two are basically new contributions by Microsoft and Google.
Which matters because once you have BOSH, you've done a lot of the heavy lifting required to install Cloud Foundry.
We are not at the point where there is seamless multi-provider support. But it's a foreseeable capability. We see a lot of capabilities, because of the flexibility of the underlying components of Cloud Foundry -- particularly BOSH ( low-level system deployment and liveness), Diego (container placement, replacement and observation) and Garden (container runtime).
My usual disclaimer: I work for Pivotal, which donates the majority of engineering effort to Cloud Foundry. I should doubly add that saying "it could plausibly do thing X" doesn't mean I'm able to commit us to it.
http://www.walmartlabs.com/2015/10/walmartlabs-oneops-open-s...
We've done some POCs around integrating an agent inside Docker containers and it works like a charm, too. Multi host networking for containers, regardless of your provider's networking limitations :)
If anyone's curious: https://wormhole.network
Edit: Fixed wording.
Given the difficulty Intel has seen getting the new E5 and E7 chips out I personally don't see any of the others as competition when it comes to producing a power efficient X86-64 chip for the DC at scale.
I think the greater danger isn't other X86-64 chip manufacturers though. The only real competition is AMD/ATi + GlobalFoundry and their chipset process is a generation or two behind. The greater danger is the market moving to an entirely new architecture. Think about ARM in the datacenter. Less power consumption, smaller boxes, less cooling requirements. DCs could make a strong argument it's cost efficient for them to jump and if a major one jumps, developers will probably follow. Most of what we build today uses a software stack built on top of a managed language (Java/Python/Ruby) and doesn't rely on being native compiled. It reduces the opportunity cost of switching.
I think it mostly has to do with the client-server technology of the web today. Once effective technology exists to decentralize key aspects of the platform - hosting, accounts, etc. - you'll see less consolidation in one provider. You'll also see a lot less feudalism when it comes to the power dynamics of users and organizations vs services (like Uber, Facebook, Google, etc.)
There needs to be a protocol to decentralize social the way bitcoin decentralized money.
Disclaimer: I'm working on just such a protocol and reference implementation (http://qbix.com/platform/features/distributed)