It effectively gives you, the SaaS vendor, the best of both worlds:

  - Access to markets you can't serve otherwise.
  - Development agility with frequent deployments.
  - Scalable ops/support for cloud/private deployments.

Outside the excitement of containers and composability I'm not sure that the problem has been fundamentally simplified. The challenge is that a fully "operational" system from a customer perspective has to fully integrate with the rest of the neworks services. That's primarily around all the other elements such as monitoring, intrusion detection, backups and secure remote access: granted the article starts to address some of these. As a SaaS vendor you need absolute repeatability and a constrained set of 'additional services' to integrate with - otherwise the support matrix is going to grow massively. From a customer perspective I can see the advantages in being able to flexibly move across the boundaries depending on the circumstance.

Symmetry between servers has a value.

The main board, raid controller, network, and usually the storage is going to be planned out meticulously ahead of time based on the maximum load the server is going to see during it's lifetime. Often, Processor and Memory come down to "If we needed X Feature and we didn't have it" or "If we took the server down for 1 hour to upgrade it, how expensive is that?".

I initially pictured someone picking the highest spec option they can afford when setting up a new service, rather than choosing based on actual demand of each node.

> "If we took the server down for 1 hour to upgrade it, how expensive is that?"

Putting my CI / DevOps hat on for a sec: who takes production servers down for upgrades without some level of HA to avoid downtime? ;)

Every service gets run in two environments: test and prod. Both are co-located on the same Kube cluster in different namespaces. We also don't put any datastores in Kube. That stuff still lives in OpenStack for now. Ceph can make it possible but for fast disk I/O, it's tough to beat the local SAS bus.

Personal anecdote - we have everything on-prem using a pretty standard vSphere setup, I've got a couple PostgreSQL databases that aren't even that heavily used (I mean, they top out around 20-40 tx/sec) backed by a hybrid Tegile array. Randomly throughout the day my IOWAIT starts spiking because even over 8Gb fiber channel our storage latency starts spiking because everything else is on the same storage (well over 200 full VM's), or sometimes I need to run a table scan over a 40GB table for a one-off query and the storage bottlenecks the crap out of us because only our small active set is cached on the SSD's.

You can run databases on remote storage, people obviously do it, but there's a reason why even on AWS the best practice is to use your instance disks instead of EBS volumes if you care about performance. Noisy neighbors suck.

GCE seems poorly engineered if the profit margins are indeed low (normal, that is).

Interestingly, the "3 cores for 1 user acing one" would explain the pricing of "preemptible" instances - which can cost as little as 1/3rd of full instance. And the main difference is that they are fully rescheduled every 24h or more often, and there's no live-migration for maintenance...

… I am now wondering if GCE runs fault-tolerant VMs. Because if it does, holy moly

A GCE VM spins up in less than a minute. I've seen them come up in 20 seconds.

Is it?

CF is meant to provide a platform, not an API. The whole point is to liberate the developer from caring about how the app is brought to life so that they can focus on building the app. For operators the point is that no badly-behaved app can bring down production for everyone, so they can lower the drawbridge and let developers move faster.

I imagine that my counterparts at Red Hat would say the same about OpenShift. It's there to let you go faster.

Disclosure: I work for Pivotal, the majority donor of engineering to Cloud Foundry.

What am I meant to be estimating, exactly?

We would probably build this around Kube's Ingress API. We're still a few backlogged projects away from addressing this internally. For now, we manage VIPs by hand. About 1-3 new VIP requests a week.