Re:App Engine’s New Go 1.11 Runtime
by CydeWeys on Wednesday October 17, 2018 @10:54AM
(#18239643)
Neat, looks like they're using a solution similar to what we're doing, except instead of using Cloud KMS to encrypt information stored in Google Cloud Storage, we're putting our information in Datastore.
The reason being that these secrets are absolutely essential for many tasks that our registry needs to perform, and the past six years of experience have shown us that Datastore has better availability than GCS. We haven't seen Datastore ever go down unless all of Cloud is down too, whereas we have seen outages isolated to just GCS. Datastore also has lower latency (since it's a DB, not bulk file storage).
One caveat is that the maximum size of a single entity in Datastore is 1 MB -- if you're encrypting stuff larger than that then you'll need to shard (ugly) or just use GCS. Since none of our secrets are anything close to that large, it works just fine.
The reason being that these secrets are absolutely essential for many tasks that our registry needs to perform, and the past six years of experience have shown us that Datastore has better availability than GCS. We haven't seen Datastore ever go down unless all of Cloud is down too, whereas we have seen outages isolated to just GCS. Datastore also has lower latency (since it's a DB, not bulk file storage).
One caveat is that the maximum size of a single entity in Datastore is 1 MB -- if you're encrypting stuff larger than that then you'll need to shard (ugly) or just use GCS. Since none of our secrets are anything close to that large, it works just fine.