There's (IMO) a mix.

Kubernetes and cloud native software make a lot of use of TLS for mutual auth.

A standard Kubeadm cluster (very vanilla k8s) has 3 distinct Certificate authorities, all with the CA root keys online.

On top of that things like Helm, FluentD and Istio will make use of their own distinct TLS certs.

One of the most "fun" pieces is that k8s does not support certificate revocation, so if you use client certs for AuthN, then a user leaving/changing job/losing their laptop can lead to a full re-roll of that certificate authority :)

If you issue user certs, it is best to do it from software and have them live for 8h max.

Also recommended: keep api behind a jumpbox.

I've seen short-lived certs as a suggested workaround for the lack of revocation and as a user that might be the best option.

That said the distributions I've seen that make use of client certs, don't do that (typical lifetime for a client cert is 1 year), so I'm guessing a load of people using k8s will have these certs floating about...

It’s a mental shift similar to using iam assume role sessions rather than static keys with perms right on em.

Powerful creds should be limited in lifespan and machine issued to users in a transaction that involves mfa.

We built our own but Gravitational Teleport I think has these patterns in a product.

The other issue even if you get certs right is service tokens (an admin could have grabbed a copy of one and they can be used from outside cluster) so want to keep that api wrapped IMHO.

As an aside, if you want revocable, non expiring user creds you would be better served by bearer tokens.

True, the only real in-built option in k8s land for that is Service accounts, which aren't designed for user auth. but can be used for that purpose.

With certs that short-lived, you're just reinventing Kerberos, badly.

Using a similar pattern (tickets I guess) doesn’t mean you’re reinventing something.

You might equally say we’re reinventing “authenticated sessions”. More in common with jwt cookies tbh.

We don’t want to run krb infrastructure so we don’t do that.

The runtime dependency on ldap or NIS, plus keeping krb itself HA, fed and happy plus OS dependent PAM setup make krb fairly undesirable in a production cloud environment if ssh certs and kube certs would suffice.

> so that the certs you're using aren't in fact self-signed but signed by a private CA?

I find that most people confuse or combine "self-signed" with "signed by a private CA". For a lot of uses, the configuration pains are the same to the user: "I have to load this cert into the CA root trust store". They don't realize how much better a private CA really is.

And of course, PKI would be so much more useful with "name constraints" so you don't have to trust a private CA for all domains just the one domain you care about.

k8s tends to use a private PKI. I do wonder if we're comparing secure verses insecure solutions though. Nobody ever has TLS issues with swarm, puppet, or aws, or are they just trusting the network?