The security policy was draconian to the extent I’m sure it was well intentioned but led you to do things in the least secure way possible as it was the only way to complete a contract.
I.e the servers on the other end running Windows 7 (in late 2019) where so old they didn’t have the required cpu instruction set to run some required software. Likewise input lag was extremely noticeable to the point you hit a key, wait one second then press again thinking something went wrong only to have to press delete some seconds later.
How did I deliver that project? Developed on my own local machine, emailed the artifact to the cooperate email address obfuscated, log in to the corporate laptop, then Citrix, ssh the artifact up to the cloud ec2 servers.
That’s another one. The cloud ec2 servers. No public outbound internet, no internal trusted repositories. What was the accepted way of setting up the servers? Going to random internet sites, downloading random binaries to your Citrix account, scp’ing then to the servers. Trying to explain how stupid this is gets no where in organisations with thousands of people. When you mention trusted artifact repositories, immutable / reproducible builds, deployment pipelines the answer was we don’t have this as they didn’t meet security guidelines.
This was a tier 1 bank. Experience is the bigger the company the worse things are due to the size and different teams/departments being so disconnected.
When we little employees roll our eyes and say "this doesn't make sense," we're telling ourselves a comforting lie because the situation you describe DOES make sense- from the organizational perspective.
Management did everything their rules allow to make the computing environment safe, but you, the individual, hacked an unsafe circumvention into the workflow by finagling some personal website and using SCP. I bet none of that process is documented- and if it is, whoever made that documentation is now the responsible party and fall guy.
The setup you describe sounds less like draconian security and more like a half baked solution that confused usability hurdles for security. Just because security usually brings lower usability, it was probably (wrongly!) assumed that lowering usability will bring security.
The bank was using remote desktops over Citrix from HP workstations (thin clients). It worked really really well, including copy/paste.
If not for the initial login and citrix log, I don't think it would be possible for a developer to figure out he is working on a remote desktop.
1) App whitelisting. This is not the strongest mitigation (attackers with exploits can move in memory), but it stops drive-bys and 99% of malware, plus it massively reduces noise on the network.
2) Reduce lateral movement. If the attacker accesses an employee laptop, encrypts it, but can't move to other devices, that's not a very compelling ransom.
You don't known Carbon Black is there until something doesn't work. That "not working" propagates an alert up to the security team.
a) You just kill ~85% of malware (rough estimate, probably technically higher, but I'm basing that on stats around interpreter-based/LOLBAS malware). Anything that isn't targeted is probably dead in the water.
b) You know exactly what's running on everyone's computers, more or less, so you have a way easier time baselining and building monitoring.
It's honestly easy-mode for security. But it's hard to roll out to a company a decade after it's been running, so you really need to do it early on. And most orgs don't care about security until after a breach, at which point they're too large and slow to get something like that done.
I'm interested in learning how to do this right, because I've only ever seen it done wrong. How do you streamline the process for getting programs approved? How do you accommodate developers who need to generate and run code?
Yeah, absolutely. But I consider this to be, when done right, a good thing. A single security team is going to drown trying to scale your company's security asymmetrically with your company's growth. For every 1000 developers you might have 5-50 security engineers depending on how serious the company is about it. Spreading out security work across the company scales extremely well.
Of course, you want to minimize burden too.
There are a number of ways you can go. Personally, at my company, we're so small that it's trivial. We rarely onboard new employees and pretty much all approvals are handled within a few minutes of the employee getting their laptop.
As you scale it may make more sense to crowdsource this. 'Upvote' is one such tool. https://github.com/google/upvote
Upvote-like systems basically give you reputation based approvals. If you can convince a number of your coworkers that the app is worth installing, you can install it. I've seen a number of systems built this way that combine crowdsourced approvals with other forms of reputation.
At my company we use Chromebooks. Apps/Extensions are only allowed once approved. The Linux environment is where development happens.
This is nice because there's a strong separation between environments. That said, we don't do any sort of application whitelisting in the Linux environment today, due to a lack of tooling support.
If we did, what I'd like to do is just have simple rules like "If the binary was created by gcc etc allow it to execute". Santa allows for process-tree based rules like this.
Another option is developer VMs like in EC2. These can be nice for devs because they're often much more powerful than a laptop anyways. It requires a bit of tooling to work smoothly though with local IDEs and whatnot.
The app whitelist impairs my efficiency, and honestly takes a fair amount of fun out of work. I can no longer use many of the tools that help me, and instead am confined into the corporate-approved structure. This is both a productivity issue and a job satisfaction issue.
Trying to separate our development workstations, and dev environment overall, from the rest of the sordid mess, is extraordinarily difficult. We're trying to do similar what you recommend, with developer VMs running in our internal ESX cluster. However, that technology conflicts with Microsoft's Hyper-V (we're a MS shop). We're trying to get to Docker/k8s, but we can't run Docker inside a VM that runs in ESX. So we're trying to offload the actual execution of the system, even for local testing, into DevSpaces. But there's a lot to figure out, and DevSpaces is a young product. Further, private AKS environments are a new thing in Azure, and we've a couple of times now run into roadblocks with MS's own growing pains.
Your "upvote" system seems to have merit, but with, I dunno, 50 or 60 devs, spread across several teams focused on different tech, it doesn't seem like it would scale well. Coming up with the right mix of apps, and convincing our Risk team, is very difficult. Especially when so much of their pushback seems confusing (like, DBeaver was denied as a FOSS product, but approved when we paid for the Enterprise version).
Out of curiosity, like what?
> Trying to separate
Yeah, like I said, starting early is going to make things much easier. If you're trying to get to this place later on, it's just wayyyyyy harder and most orgs can't get there. This is often the case with security - if you build your code, infra, policy, etc, with security in mind from day 1 it's 1000s of times simpler than doing it even just a few years later.
> Coming up with the right mix of apps, and convincing our Risk team, is very difficult.
Oh yeah, a risk team can be the real killer. That's why upvote is nice - security only gets involved if something is flagged. But if your risk team isn't willing to work with you that's a problem, and it sounds like yours isn't doing the work consistently.
To me, approval should be easy. Even if a malicious app is approved that's often still a huge win - the attacker can't use many of the tools and techniques they're used to. Obviously you want to avoid that compromise too, but it isn't the whole entire goal.
We whitelist some vendors entirely. Our app whitelist request form is:
* What risks are there with this extension/ app?
* Is there another approved app that can do this, and if so, why do we need this one?
* How will this app help you? What is it for?
That first question is really important because people usually have a decent understanding of what the app should/ shouldn't be doing, or if the risk should be trivial. It's all about spreading the assessment out.
98% of the time this will be triggered by an admin performing a task that just happens to touch that resource, but it is incredibly helpful the other 2% of the time.
Meanwhile in the real world, a company I develop for implemented the most draconian security measures to "prevent ransomware".
Development environment is a virtual machine at the other end of the world, with disabled copy-pasting from and to the local system. A complete separation between safe internal network and unsafe development environment. No copy pasting from and to email, etc. Horrible environment to work in.