If you are running on cloud, you would probably want your cloud provider to give you service secrets and rotate them somehow. AWS/Google Compute metadata service or Azure Key Vault are capable of doing this but I don't think they entirely map the microservices world because ACLs are set on the VM instances, not microservices specifically.
Our service is built on top of docker deployed on CoreOS with fleet and etcd. Most of our secrets & runtime configuration was stored in etcd, which was our attempt to store the config in the environment (http://12factor.net/config).
With Kubernetes, life is much simpler. Gone are the silly fleet configuration files, and the bootstrap scripts I used to configure etcd. Moreover, Kubernetes' secrets volume means I can have my configuration and secrets easily plugged in.
There are definitely other great solutions out there, but I'm sold on Kubernetes.
https://github.com/kubernetes/kubernetes/issues/10439#issuec...
Also on access control, any process with root on any node in your cluster can get access to all your secrets (since the kubelet needs to be able to do so). There are no user access controls either; any cluster admin can dump all the secrets.
This stuff is clearly documented, so it's not an indictment on k8s; I just get the feeling that the feature isn't really ready for production use yet.
That still leaves the secret in plain view on the nodes that run the pod that needs the service. It would be great to be able to umount the secret when not needed anymore.
http://kubernetes.io/v1.1/docs/user-guide/secrets.html
I can't remember which issue this was on, but it seemed like there was some discussion on their GitHub project about making pluggable secrets backends (HashiCorp's Vault was mentioned).
Kubernetes' secrets API is still very basic, but I think the fundamental concept is very sound and has a great foundation to continue building on.