> Not perfect but sometimes perfect is the enemy of good.

This. Add enough hurdles, and people will

a) spend all the energy they're willing to spend on your process, hate it, and as a result never do anything except what you force them to, and in particular, never voluntarily reduce permissions unless forced (because they might need them later and then it'll be a pain). They'll also see you as an enemy, not a partner, which is not the place a security team should be in.

b) optimize for not what is best, but what is least affected by your wall of process, even if it's less secure (e.g. because it's a legacy system that you didn't get around to locking down yet).

c) outsource to less secure vendors, and get it approved because management knows that getting it deployed internally would take forever due to the process

d) in the most extreme case, set up uncontrolled shadow IT with zero security controls and hide it from you - and because they already spent all their energy dealing with "perfect", they don't want to hear the word "security" ever again, and the security posture of their shadow IT shows this.

Thank you for acknowledging this. I worked at a place like that and it put me off security to the point where I didn't want to ever deny any person or app or service permission to anything and was just happy with securing the edge nodes of our network. I got over it and now I'm in a position where my neck is on the line if there is a security breach I feel differently but I can still relate to myself back then and understand the need to be pragmatic and massage people into following the process rather than forcing them.

Years ago I wrote a program to let various services run their course, query Cloudtrail for successful calls madero different AWS services, and attempt to find a minimal set of IAM permissions (not applicable for S3 at the time). The idea was to run an exhaustive test suite with expected allowed actions only and deny anything else. I believe AWS has a similar tool now for IAM but it’s not a problem that’ll be resolved satisfactorily for everyone given the combinatorics of IAM possible. Lateral movement in IAM roles and credentials is tough and even today not every action that IMO should be flagged is reported (IAM role assumption failures across accounts is silent when I checked early last year).

Did you open source it? If not, you definitely should.

Someone has actually been working on a project like this. While not 100% complete it's the best working one I know of. Can definitely relate to the problem being described here, especially when writing IAM policies for terraform deployments.

https://github.com/flosell/trailscraper

Probably can’t be open sourced given IP under contracts but I could try to re-write it. There’s some new services in IAM that could be leveraged to make it more accurate and cheaper to use, too.

Thank you for putting this idea in my head! I’ve been trying to get better at expressing infrastructure as code, and one of the big blockers has been how adding new services to e.g. Terraform is tough when you don’t know all their permissions they need (see also https://github.com/hashicorp/terraform/issues/2834 for example).

Using a test AWS environment to stage and then checking CloudTrail to see what was actually called would be a step forward. Having software to extract it would be even better.

The "I don't know" case is the most painful to ask for since it feels like I am always second guessed.

If I have the permission, I can tell its value in a few minutes but instead I have spend hours doing due diligence to try to justify my answer and still am not confident asking.

I usually end up getting what I need in the end so the admins don't see it as a big issue, but it is a subtle thing that can kills productivity for anything related to AWS for everyone else.