This is actually easier said than done. One of the major concerns is you don't want some dude with a security clearance taking work home with him. So a lot of effort goes into things like selinux being used to control what programs can access files.

Here you have a fundamental conflict. A root user can reconfigure the controls to allow access (that is absolutely necessary from a practical perspective), and so you end up having to trust the sysadmins.

Or you could set up other systems that would read the files, decrypt them only within special programs. Now you have two problems..... Moreover the sysadmin might be able to pull passphrases as they are typed, keys as they are uploaded, etc. It is not that easy to manage.

In the end a determined sysadmin can get the information and there isn't really a way you can lock him or her out fully.

A determined sysadmin can certainly get the information, but can he do so undetected?

A sufficiently careful one probably could for some time, especially if he was quite aware of what was checked and could work around such checks initially.

DBA's and system administrators can usually use their privileges to install tools or change settings in such a way that they gain access to the raw data. Whether it goes undetected or not is another matter.

In the end someone or some process has access and a sysadmin / DBA could pretend to be that someone or that process.

Once you have physical access to a box it is game-over, the only thing that might still trip you up is an auditing system that is not part of that box.

You shouldn't make it easy either, but if someone configures a system they are in a position to bypass the security on that system. It all ends with trust.

You can set things up so there's no way for any single to bypass logs or other audit controls. A trivial way is to have all logins go via an administration host which logs separately , and never give admin rights on that machine to people with admin on the protected systems.

Yep. That's the way to do it. But a surprising number of companies have things set up so that a single individual can get at whatever data he or she wants without any logs or with logs that can be tampered with. Typical reason in small companies: there is only one person with the required skills.

What is surprising is that even the NSA did not follow these practices after all if there is a place where such a system would make sense it would be there, and they certainly could not make the excuse that they're small.

It boggles my mind as well. I worked for a credit scoring company, and I must say they took their security really seriously (much more so than the NSA, it looks like).

They created separate positions, with deliberately and carefully separated permissions and often conflicting duties, so that one party watched over the other and ensured no overreaching, and no single person being able to access data unmonitored (as rdl exemplified). In all the time I worked there as a contractor, I never saw production data.

Looking over the Internet, it seems standard practice:

http://www.sans.edu/research/security-laboratory/article/it-...

And of course what Schneier says:

"The first is compartmentalization. Trust doesn't have to be all or nothing; it makes sense to give relevant workers only the access, capabilities and information they need to accomplish their assigned tasks. In the military, even if they have the requisite clearance, people are only told what they "need to know." The same policy occurs naturally in companies."

"Make sure a single person can't compromise an entire system. NSA Director General Keith Alexander has said he is doing this inside the agency by instituting what is called two-person control: There will always be two people performing system-administration tasks on highly classified computers."

Given that NSA internal controls seem to be less than commercial best practice in high security settings, and yet people (probably correctly) assume NSA is superior to known science in many areas, there are three reasonable scenarios:

1) NSA actually is fairly competent overall, and feigns incompetence as a way to let people bypass legal controls when "useful".

2) NSA is actually incompetent overall, and people like tptacek who say they're wildly beyond the open world are wrong. They obviously have a huge budget and legal protection (legal or illegal) to do whatever they want, but a lot of what someone would want to do is something even I could pull off given the NSA's level of resources.

3) NSA has really fucked up priorities, and for some reason (limited resources, cynical political calculations, expediency, etc.) put all of their effort into offense and none into defense, not even from their #1 threat (insiders). Maybe they also put too much faith in the vetting process and external shell and less on protections within the chewy center. This was pretty common in the commercial world 10-15 years ago (when "use a firewall and AV filters" was the leading security advice). If NSA actually went backward from the 1980s when they strongly believed on internal controls to actually cross over and be inferior to the commercial world, we're kind of screwed.

I present a 4th scenario:

These systems are on 'known good networks' staffed by people who have some of the most extensive vetting you can get.

These are the folks who invented (or funded the development of) RBAC and the like. SELinux came out of NSA, for example. The tech exists, but it's a hastle to work with. If you've got fully-cleared, all-known-good actors, why bother?

It's not like it's on the internet, or accessible in some fashion.

"Maybe they also put too much faith in the vetting process and external shell and less on protections within the chewy center."

They put a lot of effort into screening staff, but they're also the most attractive target for well-funded adversaries (Russia, China, etc.).

The only stuff I've ever seen come out of NSA and actually used by anyone in government that hasn't been crap has been hardware. SELinux, etc. are a sideshow. Their OS protections as deployed elsewhere in government are primarily COTS or were developed by external contractors (HBSS, etc.), and are actually shittier in a lot of ways than best commercial practice.

I can see why trust their people and their vetting process. They get to read your email and snoop through your browsing history before they hire you.