If you like CDK, then i highly recommend pulumi

Same here. Pulumi got some things spectacularly right. Their Crosswalk for AWS [0] is magical, especially if you're building serverless apps using Lambda or Docker-based services on top of EKS/ECS. I'd suggest sticking to the TypeScript runtime and major clouds where the support is the best.

_Full disclosure_ - Marcin at Spacelift here, and we chose to support Pulumi in addition to Terraform, so this opinion is necessarily biased.

[0] https://www.pulumi.com/docs/guides/crosswalk/aws/

Can second that. Pulumi with Typescript is just absolutely awesome.

I don’t get why people like scripting it, declarative is fantastically simple. I don’t want to trace through loops, if statements, functions It does from these files declare my infra to these scripts when parser thru define my infer

I get Logic in yaml/json is less ergonomic but templing makes us for it

I suspect you haven't tried Pulumi. It is declarative. Sometimes I just want to generate the declarations rather than type them out by hand.

I was more replying to those saying these products now offer a typescript feature

It's probably good when you are already using Typescripts in other parts of your day job. While I like the idea of using a real programming language in place of a declarative DSL you trade in the awkward parts of the latter with the idiosyncrasies of async programming and promises.

I specifically chose Typescript for my IAC because of the outstanding static typing among all scripting languages.

Pulumi needs to work on their documentation. I looked at it about 18 months ago and the Python support was barely there.

Tried it again last month and it's better, but I had to do a lot of trial and error. ie. (Try to figure out how to provision an EC2 instance with a root volume. They copied part of the docs from terraform but sections are missing.)

I tried Pulumi with Python and kept running into show stopper bugs.

I'm currently about to switch to pulumi/python (we're a python show).

What showstoppers did you run into? Any insight welcome if that prevents me from hitting a wall I've not seen yet :)

My first project was trying to deploy a Redshift cluster. It flat out didn’t work. Did 2-3 calls with their tech sales before giving up on it. And then cdktf came out shortly after that.

How do they differ?

So regular CDK is basically a program-driven CFN generator.

Pulumi has a similar model where you build a resource graph at runtime BUT it's also got the execution engine built-in to the tool.

What this means in practice is that you can create resources (like a kube cluster) and then use them as providers (e.g provision state tracked resources with kube api) all in the same operation.

You can also (in your infracode or an importable module) define "dynamic providers", meaning you can easily extend the execution engine to do custom things related to your workload.

As an example, imagine you want to create a cluster, deploy an app, then provision (state tracked) some app resources like an admin user and group via the app's REST API. You can do that without too much fuss.

Neither terraform nor CDK can really do those things very well. TF is not powerful enough language-wise, and in CDK the execution phase is locked away from you.

I don't think I 100% understand.

Could you elaborate a bit more?

Let's say I have an app with API Gateway, Lambda, and DynamoDB.

I would provision them with one of those tools CDK or Pulumi.

How would these tools differ in their provisioning steps?

You wouldn't see much _practical_ difference between CDK and Pulumi (or Terraform!) for that use-case. The workflow would feel almost identical.

Under the hood different things are happening.

Your CDK program would run with all the resources you declared. That would generate CloudFormation script(s) that get submitted to the CloudFormation service for evaluation. The CloudFormation service (running inside AWS) is the "execution engine" and is responsible for creating and tracking the state of your resources.

Pulumi would run your code, build an object graph, then do the "execution" itself - invoke all the required AWS API calls to create your resources (the API Gateway, the Lambda, etc etc) from where the CLI is running. The CLI would also be writing out all the resource state to somewhere.

The tradeoffs are in line with what you might expect. The Pulumi approach is more powerful, but you "own" more of the complexity since it lives on your side of the responsibility boundary.

Some people prefer AWS to be the execution engine; they feel it's more reliable to let AWS provision resources and keep track of state. They like it that AWS is responsible and will fix bugs.

Others prefer the increased control of "owning" the execution engine. This means being able to debug it or extend it with third party or custom providers that let you provision new resource types. They're happy that they don't need to wait for AWS to fix things, they can do it themselves if they have to.

This is not the only difference between the two tools but it is one of the most fundamental ones.

Ah, thanks for this explanation!

Nice.

Does Pulumi support stuff like Cloudflare Workers, FaunaDB, or Auth0?

Workers yes, Auth0 yes, FaunaDB not built-in but you can add it (there is an experimental provider someone made).

>"So regular CDK is basically a program-driven CFN generator."

What is CFN here? Cloudformation?

Yes.

+1 to Pulumi.

xyzzy123 already described the differences between Pulumi and terraform, but I want to add one key way in which they are similar:

Pulumi uses terraform under the hood. We get all of the reliability of terraform, but with a much more powerful runtime engine.

I guess it depends on what you mean by "under the hood". As far as I know it doesn't use Terraform during runtime but it uses the Terraform resources for generating language definitions. It has a lot of interoperability tools as well such as a "terraform bridge" and a tool that converts Terraform projects.

Thank you for clarifying -- I didn't know that!

The 'not using HCL' bit is really the only positive I've found so far because everything else has been more difficult than just using TF directly. I think my goals were slightly off from the beginning, because this is really just replacing one CLI with another for me at this point.

What I want: Use Terraform programmatically, i.e. call "cdktf deploy" or similar FROM node or python and give users some scripts they can use where I can abstract away some of the difficulties of learning to use Terraform natively for simple use cases (i.e. deploy an S3-based frontend host). Ideally, I had intended to distribute some npm-installable packages which would run this stuff.

> What I want: Use Terraform programmatically, i.e. call "cdktf deploy" or similar FROM node or python and give users some scripts they can use where I can abstract away some of the difficulties of learning to use Terraform natively for simple use cases (i.e. deploy an S3-based frontend host). Ideally, I had intended to distribute some npm-installable packages which would run this stuff.

This is actually exactly the use case we’ve designed the Pulumi Automation API (https://www.pulumi.com/blog/automation-api/) to support.

Allowing modern IaC technology (like Pulumi or Terraform) to be easily embedded into custom software solutions, instead of just being something humans work with directly, is a huge potential enabler for the next wave of cloud infrastructure management tooling.

> What I want: Use Terraform programmatically, i.e. call "cdktf deploy" or similar FROM node or python and give users some scripts they can use where I can abstract away some of the difficulties of learning to use Terraform natively for simple use cases (i.e. deploy an S3-based frontend host).

Maybe not node/python, but I'm pretty sure you can use terraform as a package in go. If not, there is always the "make temp dir, write/download files necessary tf files, run terraform apply"

That's good to know at least; will give the go API a look. The latter option you're recommending is essentially what I went with (Node bin script that shells out to run cdktf commands).

There is little value in abstracting HCL al la troposphere. just write native

Thanks for the share, didn't heard about it before, I see the advantage in a use-case that we have :

We have are actually working on : "How to manage Terraform resources". We ended up having a conflict dev <-> ops where dev teams are messaging the terraform guys to create resources. For example, to create a database, it will take only 1 hour writing HCL, but 2 weeks of emails to align on the specs.

We are currently building something on top, to have resources that can be created in a self-service mode by the devs themselves. (Behind the scenes, it uses Terraform modules to generate resources that will comply with the company policies).

For the moment, it's a bunch of Jenkins pipelines. Having a CDK can actually help us a lot there. (Can plug it to a CMDB database, have a UI on top, etc)

Are all variables, conditionals, templating, and loops done in Python? Or is some of that still needed on the Terraform side?

Basically you create the desired state DAG in procedural code, rather than the TF DSL. Blithe diffing and applying are the same.

Can you inspect inputs from terraform resource attributes or data sources in the procedural evaluation?

> Can you inspect inputs from terraform resource attributes or data sources in the procedural evaluation?

No... the high level programming language really just serves as a bridge or translation layer to a Terraform compatible JSON file. Those sorts of evaluations don’t happen to the actually plan/apply. However, you may find it useful to make direct API calls to your cloud provider in cdktf stacks. For instance, I mostly use data lookups but if I want to perform string operations on that sort of data I would use boto3 instead.

I really wish CDK supported writing a Terraform plugin in any of those languages.

I really wish they would support CDK in Go!

Getting started with the AWS Cloud Development Kit and Go >https://aws.amazon.com/blogs/developer/getting-started-with-...

We were talking about Terraform CDK here, not AWS CDK. It is cool that AWS CDK supports Go though!

>"On a related note, CDK for Terraform allows DevOps practitioners to use a variety of programming languages instead of HCL."

Out of curiosity what is it that people generally don't like about HCL?

Bizarre syntax, crazy resource paths, whole directories just to define one function, etc etc etc.

Yet one more thing to learn. No types (hints in IDEs).

v2 has types, and my IDE ( VS Code ) does hints

The IntelliJ HCL plugin has full refactoring support, auto completion etc. It’s a shame it’s not open source, since there must be a Kotlin implementation of an HCL2 parser underpinning it.

Thanks for this, I was unaware!

Why would you use CDK with Terraform instead of CDK with CloudFormation? The latter seems like a more reasonable choice as it is native to AWS.

CloudFormation is really slow and lags behind in supporting all AWS features.

Terraform is super fast, has great docs and vast coverage for managing just about any type of cloud resource.

Why "practitioner"? it really does not feel like the correct term.

It's a philosophy not a title https://www.visualscript.com/devops/devops-is-not-a-title-it...

But that still does not align with the use of the word practitioner, it's usually a formal profession with licensing such as medicine or law.

That's just one way to use it. Practitioners practice something. For example you can find https://en.wikipedia.org/wiki/Martial_arts using "practitioner" for people training martial arts. There are practitioners of religion, techniques, etc.