Could you share some of the issues you're experiencing with the provider? We've also hit some problems with the provider but worked around most of the issues in our open-source modules (eg, https://github.com/mineiros-io/terraform-github-repository) and by using Terramate (eg, renaming repositories causes re-creation in the provider, with Terramate we're dynamically creating move statements to prevent this. Check out Terramate if you haven't done it yet: https://github.com/mineiros-io/terramate)

Well, there are a lot of open issues [0] with no outlook to get them ever closed.

We've had issues with branch protection rules (the same code works with old actors in the state, but new actors erroring out without any reason), organization secrets, etc.

[0]: https://github.com/integrations/terraform-provider-github/is...

Regarding Terramate - I've reviewed it, it definitely is a better reincarnation of Terragrunt not using templating but HCL, yet, it does not solve all of our issues, so, we chose CDKTF for the glue code.

The fundamental difference is that Terragrunt is a wrapper for Terraform, compared to Terramate, which generates native Terraform code that runs in all kinds of Terraform tooling such as CIs, etc. Also, Terramate adds new concepts such as eg change detection, which is helpful when building pipelines. Terramate may also be used together with Terragrunt.

For details please see https://blog.mineiros.io/introducing-terramate-an-orchestrat... and https://blog.mineiros.io/terramate-and-terragrunt-f27f2ec403...

I didn't mean it behaves the same, but it accomplishes a similar goal. Definitely Terramate is the better tool and integrates with Terraform Cloud unlike Terragrunt.

Thank you!

Thanks for sharing those insights. I'd love to hear what kind of issues you managed to resolve with CDKTF compared to using HCL other than being more developer centric?

I personally like CDKTF, but for me, it's still in a too early state of development to build a stable product on top.

One example is the handling of dynamic providers. Imagine, instead of having the inventory of environments hardcoded in directory structure, you have them in a configuration setting. Or, another example, having a dynamic list of Kubernetes clusters, which you need to install Helm charts into, for which you need dynamic aliased Kubernetes providers.

Interesting - dynamic providers and configurations are exactly what we're doing with Terramate also. Seems like we're having a very similar approach while using different tooling. Thanks for sharing the insights with me!

I know you can do it - but not as dynamically as I want, too. Maybe I'm not getting something, but I don't see the `for_each` or `count` equivalent in `generate_hcl` [0].

Also, stacks are predefined, not dynamic. With CDKTF, I can generate any number of stacks with one piece of code. In addition, I can define stack dependencies - let's say, my dev database stack depends on dev IAM stack - something that Terraform does not do and developers need to remember the order of `terraform apply` across stacks.

[0]: https://github.com/mineiros-io/terramate/blob/main/docs/code...

We do not try to replace existing solutions or technologies. They all have their well known pros and cons.

While working with customers we identified that even Terraforms HCL configuration requests too much knowledge from users. So we tried to come up with a tool where a team can take ownership of the Terraform part while the actually users can just write configurations (also in HCL). You can for sure achieve the same with CDKTF, there is no doubt.

Terramate gives us the same orchestration features that you mentioned as you can define an order of execution between stacks. You can even enforce stacks to always run if another stack is triggered. Change Detection based on our Git Integration allows us to just run stacks that actually have changed and thus reducing wait times for very big set ups to a minimum, which makes our solution very scalable.

We also currently work on easier stack management features to be able to create stacks with a single call (already available) or clone stacks for either promoting them through the environment e.g. from staging to production or to just duplicate resources easily. Those features will be used to create one-click solutions to request a new respository with a default config (as defined by the company) in less than 30 seconds + time to review (if desired) + very fast deployment.

The goal here was to create a great user experience (UX) for teams that normally do not use Terraform or programming languages as GitHub is also used by other departments than engineering like marketing, hr or sales - but also allowing the team managing this solution to easily extend it to own needs.

Our solution is currently based on Terraform but might even evolve to something else as we also see the bottleneck in the provider where both Terraform or CDKTF will face the same issues in the end.

On the dynamic side of things we are working right now on adding support for dynamic block generation on generate_hcl, which should help.

Regarding dependencies, did you took a look on the orchestration features on Terramate ?

https://github.com/mineiros-io/terramate/blob/main/docs/orch...

They should be enough to let you easily define ordering and not depend on developers doing the ordering manually.

Great news!