[1] https://learn.hashicorp.com/vault/operations/ops-reference-a...
But if you cut Consul for backend, if you are not using consul for other service discovery and Nomad etc..., you can simplify that deployment a whole lot. You make sure you open the cluster communication port, setup a Application Load Balancer in front of the cluster to balance traffic and serve SSL, configure auto-unseal using AWS KMS (since you are not using it too often, 1$ is OK to have AWS manage your master key), deploy Vault on every cluster instance, and use something managed like DynamoDB as your backend. I think this is a pretty simple yet scalable setup. The amount of cloud lock-in is pretty minimal and can easily be replaced with HAProxy setup.
If I'm already working 100% in Amazon, I'm tempted to use Secrets Manager rather than justify the cost in hours to deploy and maintain a Vault cluster.
Interested in your opinion.
We started by using https://github.com/segmentio/chamber/, but because of the way we decided to structure our secrets we decided to write a clone of it ourselves (https://github.com/micvbang/confman-go).
In practice, we have a client written in Python that grabs configuration from Parameter Store and puts it into the process' ENV variables at startup. This allows us to avoid vendor lockin in the rest of our code, since we still just fetch all config from the environment. I like it so far :)
To a sister comment who mentions parameter store - that UI is the biggest leaking bag of horse manure I've ever had the displeasure of using. We made the mistake of using it, and moved over to Vault at the first available opportunity. Vault isn't a silver bullet either, but the UI is at least usable.
By itself AWS Secrets Manager is fine, but as with everything AWS, billing is pretty opaque. If you couple this with a poorly planed Lambda setup, costs can potentially uncontrollably scale.
For what you are paying, AWS Secrets Manager seems too barebones/rigid.
No, but you’re bound to Vault?
I agree that Vault is likely the best solution to this (dynamic secrets are super useful), but “not being coupled to something” is not a benefit you get from it.
Yes, you are tied to Hashicorp Vault once you adopt it, but it isn't much of a problem because you can easily redeploy it in a different cloud.
It might be true to keywhiz, so my comment wouldn't be valid to this point.
This way I am not bound to docker swarm, or keywhiz, or god forbid AWS Secrets Manager.
As of now, I am still exposing secrets with Env Vars, but the next step is to use Vault directly. Vault has been pretty reliable so far. It is using AWS KMS for managing the master key and a scalable DynamoDB table for high availability backend.