What I like about Kubernetes is that it’s a platform you get things like exponential backoff, replication, liveness and health checking and so forth as part of the platform, and plugins for metric collection and monitoring are fairly standard. These are things I don’t even feel most PaaS have done a great job on across the board, so I can certainly see the appeal of having Kubernetes under the hood.

Yeah that is understandable, k8s is a platform with a plethora of capabilities. On the other hand, the vast majority of organizations using k8s have multiple users that have multiple work streams and multiple applications (not true by definition, but close). This lends itself to a code as configuration approach, and makes managing orchestration through a UI risky and error prone. We review code, not clicks.

Further, K8s exposes an YAML interface because it is relatively human readable and of course is machine readable. I'm sure the k8s community encourages users to build on top of this. Want to really do code as config? Write a nice DSL in your favorite scripting language to produce these YAMLs. Submit that as source. You've now automated away the vast majority of toil related to k8s YAML config, because your code can be:

1. modular by design

2. tested

3. tracked as code

etc etc

I've been experimenting on a personal project with https://www.pulumi.com/ where you can do the configuration and deployment in Python or JavasScript and I think I can recommend it.

Do you have pointers on how to merge and manage Pulumi modules(?) when multiple languages are used?

How does it compare to Terraform, which it is based on?

Yeah, there's a whole category of apps that run on heroku with maybe 10 dynos tops that could definitely be deployed on kubernetes with manual / web ui-based configuration.

PaaS came before K8s, it's the minimum that K8s is better than PaaS.

I actually don’t see why a PaaS couldn’t do all of this better. Especially at larger companies, where containers and monitoring have been the norm internally forever, it is unclear why PaaS offerings still feel primitive... maybe there is some technology limitation here.

Edit: disclaimer, I have not used a PaaS service in a couple years.

What I was getting at was, maybe K8s folks should look ahead (serverless) and not back (PaaS).

Serverless is not a strict upgrade to containerized service deploys in any sense.

Why not?

It takes containers to their logical conclusion by generalizing them to a function while removing quite some operational overhead on the way.

I think PaaS, autodeployed from a branch, is the best system for most orgs.

> In all seriousness I'm sure this is useful in limited cases, but for most the right answer isn't some fancy UI, its middleware or some layer on top of kubernetes that allows you to automate away the toil-y parts of YAML, but keep the flexibility.

I think Dhall is in the right category of solutions for this problem. YAML needs to be reusable and text templates a la Helm are an amusingly bad solution. Moreover, I think static typing is a very nice feature for things that generate YAML since it’s so easy to make mistakes with these kinds of tools. I’ve used Starlark for these kinds of tasks in the past and the lack of static typing has been a sore point.

IMHO I feel I get most of the way with operators and Kustomize.