Re:Encrypting Postgres Data at Rest in Kubernetes
by bob1029 on Sunday October 31, 2021 @07:47PM
(#29061601)
> But I would be gladly proven wrong.
All I can do is support your position. Our customers are interested in the idea of encryption-at-rest of any sensitive business data that we have on disk at any time.
I work in banking, so I am ethically bound to provide the most brutally-honest takes regarding security models to our customers. When developing proposals for encryption-at-rest we have to cast everything in the worst possible light.
We started with looking at things like DPAPI but that is almost a joke for our use cases.
We are more worried about something along the lines of - A senior IT administrator has access to all application servers with full admin/root and can trivially dump customer databases to the dark net for crypto. When modeling for this scenario, we have to consider both physical and digital controls. Digital controls alone will never provide a comprehensive solution for this kind of adversary. Our approach looks something like this:
The customer has to install HSM(s) in their environment. This would typically be something living in a nested secure network or USB dongle directly attached to each server. These secure environments/items must not ever be physically accessible unless at least 2 employees are present at the same time. Each of our application servers which transact PII must be configured to talk to the HSMs. Multiple HSMs provide redundancy in a production environment, mitigating downtime risk. The cryptoscheme is a key-wrapping approach, where working keys are managed per user work item. When a work item is requested for the first time, the HSM has to unwrap its working key (which is stored encrypted on the application server). There is a bit of a mild tradeoff between security and performance here. Hypothetically, we could require the HSM encrypt the payload itself on every I/O, but for our product that would quickly saturate the capacity of even the most expensive HSMs. The policy is that an untouched work item has its key zeroed out in memory after X seconds. Working keys that timed out of memory are marked for rekey on next retrieval.
We have also modeled for remote debugger scenarios, and believe that simple firewall and other access controls can mitigate most of those vectors. This is very much a defense-in-depth strategy, so I am only presenting a small piece of that puzzle here.
All I can do is support your position. Our customers are interested in the idea of encryption-at-rest of any sensitive business data that we have on disk at any time.
I work in banking, so I am ethically bound to provide the most brutally-honest takes regarding security models to our customers. When developing proposals for encryption-at-rest we have to cast everything in the worst possible light.
We started with looking at things like DPAPI but that is almost a joke for our use cases.
We are more worried about something along the lines of - A senior IT administrator has access to all application servers with full admin/root and can trivially dump customer databases to the dark net for crypto. When modeling for this scenario, we have to consider both physical and digital controls. Digital controls alone will never provide a comprehensive solution for this kind of adversary. Our approach looks something like this:
The customer has to install HSM(s) in their environment. This would typically be something living in a nested secure network or USB dongle directly attached to each server. These secure environments/items must not ever be physically accessible unless at least 2 employees are present at the same time. Each of our application servers which transact PII must be configured to talk to the HSMs. Multiple HSMs provide redundancy in a production environment, mitigating downtime risk. The cryptoscheme is a key-wrapping approach, where working keys are managed per user work item. When a work item is requested for the first time, the HSM has to unwrap its working key (which is stored encrypted on the application server). There is a bit of a mild tradeoff between security and performance here. Hypothetically, we could require the HSM encrypt the payload itself on every I/O, but for our product that would quickly saturate the capacity of even the most expensive HSMs. The policy is that an untouched work item has its key zeroed out in memory after X seconds. Working keys that timed out of memory are marked for rekey on next retrieval.
We have also modeled for remote debugger scenarios, and believe that simple firewall and other access controls can mitigate most of those vectors. This is very much a defense-in-depth strategy, so I am only presenting a small piece of that puzzle here.