Terraform used to be ahead of CloudFormation in AWS features support, but now it's the opposite. If you consider the cost of Terraform Cloud or Enterprise plus all you need to run and the free CloudFormation, the latter seems like a great deal.

The Hashicorp stack already supported this kind of usecase, it's called Sentinel: https://www.hashicorp.com/sentinel

I do agree that it's a pity that Sentinel isn't free.

Not only it's not free, but it closed-source, too. When others (such as Scalr, Env0, SpaceLift) are using OPA, HashiCorp uses something that's not portable and that's not even using HCL. Their offering ignored not-invented-there tools such as tfsec, tflint, etc. - they only integrated recently with partners, who possibly do revenue sharing with them. Meanwhile, competitors of TFC are not sleeping and innovating way faster (Env0, for example, supports Terragrunt, which fixes many of the core Terraform issues they've been neglecting for years, such as the lack of dynamic providers). The problem is that HashiCorp is dysfunctional for the past two years. Their focus shifted to who knows what, but many providers are barely maintained (such as TLS). They have over 166 engineering jobs openings on their website - you can tell they are struggling! I guess their engineers are too busy interviewing candidates, but I assume their pay rates are below market as the positions keep staying open for months or there's a steady turnover. They jumped on the cdktf bandwagon to compete with Pulumi, but I'm not sure if that's a practical use of their resources especially when it's not even supported by their commercial offering.

I never understood this sentiment. Hashicorp offers Sentinel for free [1], publicly. The documentation is also open [2]. There's also open rulesets for CIS benchmarks [3] (minimal but demonstrates what Sentinel can do), and a ton of public drop-in rules for several clouds [4]. There's helpers [5,6] for common use cases, supports custom integrations [7] and it can be run in CI [8].

[1]: https://docs.hashicorp.com/sentinel/downloads [2]: https://docs.hashicorp.com/sentinel/intro/getting-started/ru... [3]: https://github.com/hashicorp/terraform-foundational-policies... [4]: https://github.com/hashicorp/terraform-guides/tree/master/go... [5]: https://github.com/hashicorp/terraform-guides/tree/master/go... [6]: https://github.com/hashicorp/terraform-guides/tree/master/go... [7]: https://github.com/hashicorp/sentinel-sdk [8]: https://docs.hashicorp.com/sentinel/commands/apply

I am not sure when they made it free to use (and not sure under what terms exactly), it's part of their most expensive TFC offering and always has been, but why do they believe that people are interested in a closed-source project, that's pretty much not used anywhere else, unlike OPA? I am a TFC commercial user and I'm not interested in using Sentinel as if I decide to switch to Scalr, all that work would be lost of the license doesn't allow me to use it, and if the competitor platforms don't support it due to the licensing constraints.

It looks like it dropped a few months after TFC[1] with a bunch of additional features. At least these days, Teams supports Sentinel, and in personal use cases, I've had TFC be backend and threw a git pre-commit hook at it for Sentinel on my machine. My guess is that for some time, they wanted to focus on enterprise sales and Sentinel was an easy way to upsell. SSO still makes Business worth it though.

[1]: https://www.hashicorp.com/blog/announcing-hashicorp-sentinel...

Yah... this kind of feature would be killer for Terraform...

I can't tell if this should have a /s on it but terraform does have this feature (local-exec, remote-exec) and you don't need Terraform Enterprise, the free one does just fine.

You can also write your own providers etc etc. The whole reason hooks are needed is because you don't have direct access to the execution engine.

If you really want access to the execution engine, looks like one team found a way. https://orca.security/resources/blog/aws-cloudformation-vuln...

I honestly cannot imagine how sophisticated cloud software will be in ten years. So much will be hashed out, that it will be truly like building software with legos. The idea of using libraries or abstractions over infrastructure will grow and eventually mature. I think software wont be automated, but the interfaces we implement for custom logic will get smaller and smaller

We are almost there. See AWS CDK. You could install an entire app from NPM, if someone had released such a thing. Right now the ecosystem is still building legos though.

Small rant--I generally enjoy using AWS products, but WOOF, figuring out all the IAM permissions necessary to get CloudFormation to work is a real PITA.

CloudFormation basically needs full account permissions to do it's job. We use separate AWS accounts to provide isolation. It's not great, but we've found it workable.

Agreed, but you can make it manageable if you do it a lot. First give your CF runner full permissions. Then once you have your CF done and working, take away all permissions from your CF runner. Then add them back one by one as it fails.

You can even build a script to run it, check the errors, update IAM, and then run it again.

Eventually you'll have a set of IAM least privilege rules. It's a pain but it's secure.

Do the actions from CloudFormation not show up in CloudTrail?

They do and there is even a button in iam to do just that. Also exposed as api to regernate the policy based on usage.

Would you mind pointing me to a doc about regenerating policies based on usage?

The pain I'm dealing with is having to look at the event logs to determine which CF task failed due to permission, then add that to the CF policy.

> Would you mind pointing me to a doc about regenerating policies based on usage?

First result in Google gift “generate IAM policies from usage”

https://docs.aws.amazon.com/IAM/latest/UserGuide/access_poli...

Instead of hooks triggering off of cloud formation, wondering why they couldn’t be triggered at a lower level so the console or programmatic API mutations could trigger.

Depends on exactly what youre after. AWS Config acts as an inventory and audit of resources, with some enforcement state features. Eventbridge events are produced on resource state change and CloudTrail on API actions. Those are super easy to wire with “post” hooks using eventbridge rules.

Those dont really provide for generic “pre” hooks as they fundamentally represent changes. Getting arbitrary customer logic in to the API call chain has a lot of scary implications for fault isolation, timing, resource use, etc. In my opinion it would also provide very very sharp edges for users to cut themselves on. And, of course, getting consistent behavior and coverage across the ~300 existing AWS services is a pretty big implementation challenge.

Disclaimer: Principal at AWS, but no inside knowledge of the Cloudformation changes or mentioned services.