Having used vault quite a lot, I'm not really sold on it. Do you see any value in this tool? I've done the whole sidecar mess, middle of the night three keys unlock like we're arming a nuclear weapon and everything, and most of the time it's just been a total faff and imo security theatre vs actual security.

End of the day, the secrets are being written to a .properties file or /proc/<pid>/env somewhere anyway and can be read by whomever has the permissions or the shellcode to do so..

If you're just storing secrets as static key/value, sure.

Vault's real value is in rotating credentials automatically. Some of that value has eroded over time, e.g. Kubernetes now having IAM Roles for Service Accounts. But Vault handles it for databases, and there are a variety of plugins to handle it for other usecases as well, e.g. https://github.com/martinbaillie/vault-plugin-secrets-github

I have been curious about this, too. I think I'd want something like vault to issue OTPs that can be exchanged for secrets over a socket to a sidecar, where the OTP is made available as an environment variable set by the orchestrator (eg k8s). If the token is used twice, lock it all down. If it's read but not confirmed to have been received by the service (through some method... dunno), lock it down.

Thanks for coming to my brain dump.

You can use kubernetes auth in k8s which exchanges you a short lived token for your k8s service acc jwt

If you don’t need the Shamir part of Vault, create fewer key shares.

If you integrate properly throughout the stack (i.e “not being negligent”) then secrets will not hit properties files, rotation will happen correctly, and you will be able to audit everything.

You can also do this using a native secret management system if you’re in a a public cloud, but Vault is, for the most part, just better.

What do you suggest? That is, what is proper? I haven't found a good example showing how to use it without environment variables, files*, or http interfaces with shared secrets, but it has been a while since I dug in and I could have missed something obvious.

edited to add "files"

Obviously if your app is compromised all its secrets are too. Hopefully one doesn’t pull the entire secrets backend to a single app and has audit logs to assess the actual impact and what else needs to be rotated. Also Vault is not just about kv secrets, there’s also pki, ssh and more

This is one example we put out https://github.com/avantoss/vault-infra for vault that supports auto-unlock and other DR hosting features. Once you set it up you remove access to buckets, keys, and ssh and it runs pretty self contained. Upgrades are pretty seamless. One of the most stable and reliable pieces of our infrastructure. Relies on AWS but could be supported with most cloud platforms

With its PKI endpoint, it makes it pretty easy to stand up an internal CA. No faffing about with arcane openssl commands.

The 3 of 5 keys thing is just a default and easily changed.

Set to autouseal with a transit or cloud (or hms if you pay them $$$) and you don’t need to use sss unless to recover root token (or if somebody leaves)