Off topic, but seeing as you mentioned it - do you see the integrated Raft storage becoming the preferred solution? Without it, there's (IMHO) a bit of a chicken-and-egg bootstrapping problem between Vault and Consul.
(Note “support” above is literal human support. “Support” in terms of enabling that use case will be around indefinitely but planned forever, we’ll always support different storage backends. But for our paying customers, we only support Consul and integrated storage for the sake of providing good service.)
I need an equivalent to Windows' Active Directory in Linux world that ideally can also federate with/masquerade as AD. Can Vault be such thing?
Vault is not such a thing.
AD is basically LDAP + Kerberos. Neither Vault nor Keycloak can fill that role. You would need Samba 4 setup as a domain controller [1] for that.
If you need AD but for Linux desktops you could take a look at FreeIPA [2].
If your feeling brave you can combine an LDAP server (like Wren:DS, 389 or OpenLDAP) with a Kerberos implementation like MIT Kerberos and create an AD like experience yourself. Though I would not recommend that.
[1] https://wiki.samba.org/index.php/Setting_up_Samba_as_an_Acti...
If you can use public cloud, have you considered Azure Active Directory? We're a Linux shop using it very successfully for on-prem and cloud systems.
you don't even need a ldap/ad in the background but you can connect to one.
however it's strictly for authentication and authorization - basically users + roles
vault is different usecase but can probably be connected.
You can even do cross-forest trust to AD, log into your Linux systems as users who live in AD...
Only for bootstrapping the TLS key of the vault servers we needed another tool.
Sure there is sops.
We opted for google/tink
This gives us asymmetric encryption with KMS for bootstrapping the vault AMIs on AWS.
We create Vault AMIs with Packer and put encrypted TLS keys inside the AMIs. This way we have immutable machine images, which can crash/restart using an Autoscaling group and we can attach all decryption processes to the instance role of the EC2 machines
Broadly speaking, if you're looking at Vault to solve a specific problem X for a specific consumption type Y on a specific platform Z, then it _probably is overkill_ (I wouldn't say "overhyped" :)). i.e. "encrypted key/value via env vars on AWS Lambda". "X via Y on Z." The power of Vault is: multiple use cases, multiple consumption modes, and multiple platforms supported with a single consistent way to do access control, audit logging, operations, etc.
I can't stress that "single consistent way to do access control, audit logging, operations, etc." enough. Multiple security use cases dangling off that consistency is really important as soon as you hit N=2 or N=3 security use cases.
If you need say... encrypted KV and encryption-as-a-service and dynamic just-in-time credentials and a PKI system (certificate), and you need this as files and as env vars, and you need this on Kubernetes and maybe also on EC2, then Vault is -- in my totally biased opinion -- going to blow any other option out of the water.
That's a somewhat complex use case but its something Vault excels at. For simpler use cases, Vault is making more and more sense as we continue to make Vault easier to use. For example, we now provide a Helm chart and official K8S integration so you can run Vault on K8S very easily. And in this mode, developers don't even need to know Vault is there cause their secrets show up as env vars and files just like normal K8S secrets would.
Also, this article is from June 2019 and in 10 short months we've made a ton of progress on simplifying Vault so it gets closer to that "X via Y on Z" use case. Here are some highlights I can think of off the top of my head but there are definitely more, this is just from memory:
* We have integrated storage as an option now, so you don't need separate storage mechanisms.
* Our learn guides went from basically zero to lots of content which makes it much easier to learn how to use Vault: https://learn.hashicorp.com/vault
* We have an official, feature-packed Kubernetes integration to do stuff like secret injection and rotation automatically. We also publish a Helm chart to run Vault on Kubernetes. https://learn.hashicorp.com/vault?track=getting-started-k8s#...
We're looking at ways to make running Vault much, much easier. More on that later this year. :)