Re:Infrastructure as Code, Part One
by johncolanduoni on Saturday April 13, 2019 @04:46PM
(#19655608)
I agree with this in general, but I think there’s room for more programmatic ways of calculating what the desired state is in tools like Terraform. There are already a lot of built-ins in terraform that are deterministic but procedural ways of calculating what the state is, without infecting the declarative nature of the reconciliation process.
We also were frustrated at the verboseness and vagaries of the terraform syntax.
We ended up preprocessing them using jinja2, injecting our variables via its context, and using its cleaner syntax for expressing conditions, loops, etc. Now we have the best of both worlds.
... until your jinja templating breaks for some obscure reason, or the underlying terraform resource upgrades (they follow the provider api after all), or if you need to do some state surgery and partition one terraform state into multiple ones, or until you want to import resources into the generated terraform or....
Apologies, I don't mean to be a downer. But... you're making your present life easier but making the code harder to maintain for future maintainers of your codebase.
What's the more maintainable alternative? Is writing repeated constructs etc out individually really more maintainable, or are you referring to other ways of getting these things reduced?
I highly recommend taking a look at the AWS CDK. It's a relatively new project, still in beta and hasn't gotten a lot of attention yet. It allows you to use real programming languages to generate Cloudformation templates. In my eyes this is the best of both worlds.
Just plain terraform modules, for now.
I’m eagerly awaiting 0.12 so the language would have similar features as jinja templating.
If that's what you want to achieve, you can generate the terraform files from any external system. Basically template it.
That's what sparkleformation does for cloudformation stacks.
Re:Infrastructure as Code, Part One
by closeparen on Saturday April 13, 2019 @07:07PM
(#19656343)
When there is live traffic or data in the infrastructure you’re modifying, the procedure matter a great deal.
Just as in databases, you end up playing lots of “read the query plan, restate the query to try and get a better one” games.
You’re not telling AWS how to do something you’re telling it what you want the end state to be and let it figure out what needs to be created, updated, deleted, or replaced and the dependency chain what can be run in parallel when you create or update the template.
There are linters and editors for CloudFormation that help you with autocomplete (?) and to warn you when you are specifying a a resource type. You can even add your own custom definitions to the linters for custom resource types that you create.
CloudFormation just like generic SQL doesn’t by itself have the concept of loops. CloudFormation does support custom transforms and macros that you can create in any language and you can write programs that generate CF in many languages using the CDK where it will perform validation. I haven’t used it so I am being really hand wavy.