News for hackers. Stuff that matters.
Re:Terraform 0.15 General Availability by dxxvi on Wednesday April 14, 2021 @04:11PM (#26812400)
My tiny brain still don't get why people like terraform. Do people need to look at both terraform docs and aws/azure/gcp docs when writing a .tf file? The fact that terraform saves/remembers the resource states is like a double-edged sword: we cannot manually fix some minor mistakes of ours when creating resources because that'll mess up terraform
Re:Terraform 0.15 General Availability by dasil003 on Wednesday April 14, 2021 @04:36PM (#26812721)
Have you ever set something up manually via AWS console, then 6 months later totally forgot the steps you took and end up wasting a lot of time reverse engineering what you did in order to make a comparatively small change?

After that perhaps you vowed to take better notes, so the next time you do it that way, but then 6 months later you find that you missed some detail, or there was some changes in between that were not recorded.

So then you decide you're going to do everything by API and save the commands, so you write a bunch of bash scripts that execute against the AWS CLI. Over time you add several more for different operations (eg. adding an instance, configuring new ssl cert, etc), but the list of scripts grows very long, and you find that each one makes a lot of implicit assumptions about the state of the infra when its run, so you end up with varying degrees of confidence in the scripts depending on how often you run them.

Now you are primed for Terraform. At this point you realize the hard part about cloud configuration is state management. Furthermore, you realize there are some common patterns of how different components and APIs interact with regard to serial dependencies and operation idempotency, however the specifics vary by service and by use case. Terraform gives you a standard substrate for state management, and a framework for developing service "providers" that know how to interact with APIs and map them to state. All of this happens in code which is declarative, can be code reviewed, and state which can be centrally tracked and shared among a large team for a clear audit trail.

There's definitely a learning curve, but once you learn it the overhead is pretty small compared to the benefits, even for small teams IMHO.

Re:Terraform 0.15 General Availability by takeda on Thursday April 15, 2021 @02:56AM (#26816955)
I'm not parent poster, but I believe (when he mentioned looking at TF and AWS documentation) he most likely meant to use TF instead of CloudFormation. And frankly I think CF is far more robust.
Re:Terraform 0.15 General Availability by aprdm on Wednesday April 14, 2021 @09:28PM (#26815201)
The jump from what you mentioned in paragraph 2 to 3 is not necessarily Terraform. You can use other tools, like Ansible, that IMO have a much better framework than Terraform. I use Terraform for extremely simple stuff that is easy to destroy/recreate. Projects of bigger scale IMO are better served with Ansible and friends.
Re:Terraform 0.15 General Availability by weitzj on Wednesday April 14, 2021 @11:39PM (#26815966)
Ansible looks fine when you start, but gets painful.

You need to write idempotent ansible from the start. No excuses.

Then you would implement a retry logic in ansible since your cloud APIs will fail for weird reasons or you really must wait that a resource in AWS is truly ready before you run another task. ( for example vpn gateway in AWS must be ready and then you can attach a vpn connection, or a route53 zone record has to propagate before you do something with it. But in the meantime your other tasks can continue in parallel. Only the vpn tasks needs to wait)

Then after a while with all your resources created with your idempotency implemented in ansible, your code will roughly first check ( with timeouts, retries) if you actually can skip tasks since there is nothing to do. The checks increase, your playbook takes longer and longer to execute. Maybe you try to parallelize it in some way. In the meantime a colleague of yours executes the same playbook in their machine or pipelines, and now you see weird side effects. Maybe you start implementing some locking behavior.

At this point we did not do any code reviews yet and we did not answer the question: “how should the infrastructure look like?”

This especially means: how do you actually delete resources with Ansible? So you Start to introduce a kind of “state” for your resource.

So, if you squint hard enough, idempotent, resilient, parallelizable, thread-safe, fast ansible with a state is what terraform solves. If you look at tools like terratest you even get to do unit/integration testing for your infrastructure code.

So as soon as you are more than 1 person handling infrastructure following best practices like code review and testing, without getting insane, use terraform.

If you are alone and don’t want to follow the above best practices, you still have to be very good in writing idempotent, resilient ansible code.

My hypothesis is that the intersection of people who write high quality (idempotent, resilient) ansible code but people who do not care about code review/testing, is quite small.

Once you do not work in isolation any more, either in a team or as a consultant who needs to hand over their work (and Teach people how idempotent ansible works), I would favor terraform any time.

You could even call ansible from terraform if you need some ansible integration, where ansible has a nicer api for you. But let terraform handle the retries, state management and ansible could just be the executing part. So ansible is more like a fancy bash script/function which you call in an orderly manner.

Re:Terraform 0.15 General Availability by aprdm on Thursday April 15, 2021 @12:59PM (#26823101)
> You need to write idempotent ansible from the start. No excuses.

Absolutely. And code reviews, etc. just like code.

> Then you would implement a retry logic in ansible since your cloud APIs will fail for weird reasons or you really must wait that a resource in AWS is truly ready before you run another task. ( for example vpn gateway in AWS must be ready and then you can attach a vpn connection, or a route53 zone record has to propagate before you do something with it. But in the meantime your other tasks can continue in parallel. Only the vpn tasks needs to wait)

Yes, Ansible modules have a retry built in or a wait_for statement. You can also control the serialization serialize: (or was it parallel), or run_once or when. You have a couple of tools at your disposal.

> At this point we did not do any code reviews yet and we did not answer the question: “how should the infrastructure look like?”

Why not? We usually start with some diagram in README.md or Confluence or a Design Proposal before the code. Code never gets released without a merge request.

> This especially means: how do you actually delete resources with Ansible? So you Start to introduce a kind of “state” for your resource.

state: absent, you can use in about every module.

> So, if you squint hard enough, idempotent, resilient, parallelizable, thread-safe, fast ansible with a state is what terraform solves. If you look at tools like terratest you even get to do unit/integration testing for your infrastructure code.

I believe terraform hides way too much and is too magic. Troubleshooting when terraform goes wrong is really really hard. You have two states to look on -- the cloud state and the TF state.

And now you need to know two tools too, since you will likely need Ansible anyways to configure and operator the hosts later.

> So as soon as you are more than 1 person handling infrastructure following best practices like code review and testing, without getting insane, use terraform.

Beg to differ, 100s of engineers sharing ansible galaxy roles with semantic versioning and whatnot in a very good way at my org.

> Once you do not work in isolation any more, either in a team or as a consultant who needs to hand over their work (and Teach people how idempotent ansible works), I would favor terraform any time.

Maybe you're right. If I'm doing freelance I will likely use Terraform to interact with Cloud. IMO Ansible, just like coding, requires a bit of a community around it with best practices, reviews and etc.

Re:Terraform 0.15 General Availability by weitzj on Thursday April 15, 2021 @02:44PM (#26824568)
> Beg to differ, 100s of engineers sharing ansible galaxy roles with semantic versioning and whatnot in a very good way at my org

I understand we both have our experiences when using ansible compared to terraform. Good to know that this scales

Regarding your point:

> state: absent, you can use in about every module.

What I meant is that you probably would create 2 pull request. The first one to actually delete things for your infrastructure and then a second pull request to delete your ansible code. I am still not sure how changing things work in this manner, where you destroy and recreate things in a single logical Pull request/commit for example. I guess tracking multiple pull requests in a single ticket might work.

> And now you need to know two tools too, since you will likely need Ansible anyways to configure and operator the hosts later.

I agree. Well, I would use ansible beforehand with Packer to build immutable images/AMIs which are launched by an auto scaling group and a final configuration via cloudinit. Therefore I would not have a chance to apply ansible during autoscaling.

Thanks for your input and valuable discussion. Stay safe

Re:Terraform 0.15 General Availability by apple4ever on Thursday April 15, 2021 @12:36AM (#26816236)
Thank you for this. I've been hemming and hawing over straight Ansible or Ansible + Terraform and you made it clear why the later makes sense.
Re:Terraform 0.15 General Availability by themacguffinman on Wednesday April 14, 2021 @11:10PM (#26815823)
I'm no tooling expert but that seems like the opposite of what you should use Terraform for. Imperative frameworks like Ansible are basically fancy ways to organize and execute custom scripts, you still have to implement your own custom state management logic in Ansible commands/scripts. With larger & more complex infrastructure, you want to do this as little as possible because it's hard to consistently get it right, so Terraform basically steps in and implements state management for you. Even if you end up needing to write your own TF drivers, regular devs don't have to deal with it to make infra changes. Declarative templates are much simpler and easier to think about, all the complex state management stuff is hidden away.
Re:Terraform 0.15 General Availability by aprdm on Thursday April 15, 2021 @12:54PM (#26823035)
Ansible has a bunch of declarative modules. Writing imperative Ansible should basically never happen. My team maintains 50-80 galaxy_roles and it's super rare that we have to build any shared galaxy role on top of imperative logic.
Re:Terraform 0.15 General Availability by themacguffinman on Thursday April 15, 2021 @10:12PM (#26828598)
True, and interesting to see that it has worked out so well for your team. I suspect the difference I see in practice is that Ansible playbooks make it much easier to hack in imperative shell scripts in an Ansible task, so it's more likely to happen. I agree that a disciplined & experienced team can do the right thing with either tool and perhaps Ansible's API is nicer if you use it right. This makes me see Ansible in a slightly different light, perhaps I've been unfairly influenced by some Ansible playbooks I've seen in real life.
Re:Terraform 0.15 General Availability by darthrupert on Thursday April 15, 2021 @01:52AM (#26816578)
No, Ansible is not good for this. It's good for maintaining clusters of hosts, but its stateless nature makes it close to worthless for infrastructure.
Re:Terraform 0.15 General Availability by aprdm on Thursday April 15, 2021 @12:53PM (#26823013)
Well I guess I beg to differ. Been creating and managing 10ks of hosts across multiple on prems datacenters and clouds for some time now.
Re:Terraform 0.15 General Availability by apple4ever on Sunday April 25, 2021 @02:09PM (#26935001)
How do you deal with this with Ansible instead of Terraform?

I'd love to have a single tool, but Ansible's style seems like a poor fit especially compared to Terraform.

Re:Terraform 0.15 General Availability by darthrupert on Monday April 19, 2021 @01:17PM (#26865325)
Perhaps we have a different view of what infrastructure means? Managing hosts isn't it by my view.
Re:Terraform 0.15 General Availability by aprdm on Thursday April 22, 2021 @05:41PM (#26908521)
Possibly! Care to share your view?
Re:Terraform 0.15 General Availability by dkobran on Wednesday April 14, 2021 @04:39PM (#26812758)
Great explanation!
Re:Terraform 0.15 General Availability by cmeacham98 on Wednesday April 14, 2021 @04:27PM (#26812605)
> we cannot manually fix some minor mistakes of ours when creating resources because that'll mess up terraform

This is a feature, not a bug. Terraform is a tool to (reproducibly) enshrine your infrastructure in code. 'Minor manual fixes' are often left undocumented and known only to 1-2 people, and suddenly become major problems when your infrastructure comes crashing down and nobody knows why it used to work.

Re:Terraform 0.15 General Availability by dastx on Wednesday April 14, 2021 @04:30PM (#26812640)
Reproducible infra, gitops, automation and much more.

For me, the biggest thing is, when I go into AWS I struggle to find everything that is intrinsically linked to another resource. Say you have a lambda, to find which iam is linked to it, and what permissions it has is 2 separate tabs, then another for e.g. security groups, probably more tabs for other things. While using aws-cli makes it slightly easier, it's still a lot of effort to do this effectively.

With terraform I can look in one repo that has all the above, often in the same file too. Finding out what your infra looks like is a lot easier.

Regarding the state, you should not be touching your infra outside your code, if you do (e.g. while you're testing in dev), you should make the same changes in tf once you've confirmed it's what you want, and otherwise you undo those changes.

With further automation (e.g. tfcloud) you can even enforce these things by auto applying workspaces which ensures manual changes are always undone.

Re:Terraform 0.15 General Availability by moufestaphio on Wednesday April 14, 2021 @04:41PM (#26812787)
> we cannot manually fix some minor mistakes of ours when creating resources because that'll mess up terraform

In my opinion if you're doing manual fixes you're doing it wrong. Let's say you do your manual fix in your Dev environment. Do you remember to do it in Prod/whatever other environments you have? Are you sure you did the EXACT same thing? Did you change 5 other things trying to fix it first?

You end up with so many different deployment environments that are unique 'snowflakes', and when something breaks in one it might not affect the others cause they're in totally different states.

It's a nightmare.

In my opinion, infrastructure as code is the only way to do it in a serious environment.

Re:Terraform 0.15 General Availability by kstrauser on Wednesday April 14, 2021 @08:19PM (#26814788)
I think there's a middle ground if you're not sure how to fix a mistake in Terraform but you know how to do it in the console:

* Make your changes by hand

* Right afterward, run "terraform plan" to see how Terraform would undo your changes

* Edit your Terraform config to reflect those changes, and run "terraform plan" again to make sure you caught everything. Repeat until it's a no-op.

Now you've got a log of what you've done in a Git-ready format, and you can repeat it elsewhere, and you've learned how to make that console change in code.

Re:Terraform 0.15 General Availability by pm90 on Thursday April 15, 2021 @02:45AM (#26816882)
This. I’m surprised at how many folks don’t realize you can do this and capture your changes in terraform by looking at the plan and making tf code changes until the plan doesn’t show a diff
Re:Terraform 0.15 General Availability by te_chris on Thursday April 15, 2021 @03:39AM (#26817287)
You can also fetch cloud resource state with terraform, without running plan - I can’t remember the exact command. You can use this to import new resources into tf
Re:Terraform 0.15 General Availability by matwood on Wednesday April 14, 2021 @09:35PM (#26815238)
You can do it that way, but I find the tf docs easier and more concise to use than clicking around the AWS UI.
Re:Terraform 0.15 General Availability by kstrauser on Thursday April 15, 2021 @12:33AM (#26816221)
I do too, but that was an invaluable tool when we were first switching over and learning the ropes.
Re:Terraform 0.15 General Availability by Chilinot on Wednesday April 14, 2021 @04:26PM (#26812572)
Terraform allows us to implement development practices into our sysadmin lives. Such as code reviews, etc.

For example, at my work this is what i do to apply changes to our AWS setup:

1. Fetch the latest version of our git repo.

2. Create a new git branch named after the Jira ticket im working on.

3. Solve the jira ticket by modifying the terraform code accordingly.

4. Submit a pull request and assign one of my colleagues as reviewer.

5. They review my solution, tell me to correct some issues that there might be, or straight up approves my solution.

6. My PR is merged into master.

7. I download the latest master version and apply the codebase.

This way we always have at least two people verify any changes to our infrastructure, minimizing the risk of fuckups and ensures solutions are as good as possible.

Re:Terraform 0.15 General Availability by hrktb on Wednesday April 14, 2021 @06:42PM (#26814019)
Reading the sister comments, I kind of understand the appeal of terraform for huge/multi cloud infra systems.

Now, managing changes in code doesn’t look too far from dealing with kubernetes’ json/yml config and applying them to the current cluster, provided it would be trickier when expanding to multiple cluster or doing complex orchestrating.

I guess TF makes a lot more sense for on-premise, bare metal VMs ?

Re:Terraform 0.15 General Availability by erik_seaberg on Wednesday April 14, 2021 @07:04PM (#26814210)
Also cloud hosted services. E.g., terraform knows how to manage hundreds of AWS resource types (IAM roles, DynamoDB tables, S3 buckets), sometimes even before CloudFormation does.

There seems to be some kind of https://github.com/aws-controllers-k8s/community in early beta, but I don’t know how much it supports so far or whether it can diff a resource and modify it in-place.

Re:Terraform 0.15 General Availability by DenseComet on Wednesday April 14, 2021 @06:53PM (#26814108)
I think the right way is to use TF to provision the kubernetes cluster and underlying resources such as storage connections, dns, etc, and then use kubernetes to deploy the app.
Re:Terraform 0.15 General Availability by vbsteven on Thursday April 15, 2021 @02:03AM (#26816632)
Use Terraform for bringing up the Kubernetes cluster and other related cloud resources (storage, load balancers, DNS, etc).

Once you have a cluster up and running you can just use kubernetes yml to manage the cluster.

It is possible to also use Terraform to manipulate kubernetes resources (or AWS ECS or another container platform) but I personally like a clear separation between infrastructure (bringing up the environment using Terraform) and scheduling work on a cluster (using kubectl)

Re:Terraform 0.15 General Availability by hrktb on Thursday April 15, 2021 @03:01AM (#26816992)
So TF would come where you would otherwise use gcloud commands, if I get it.

One part of the cluster building that caught me off-guard with GCP is that cluster options effectively change over time, as the k8s versions march forward.

For instance some feature go out of beta and become the default, and trying to rebuild a cluster exactly similar to one built year ago would require specific “don’t use that new feature” flags added to it. Or using these new features require adapting the other resources to the new configurations.

I guess TF is still useful in that it helps audit and reproduce the same infra inbetween changes, and will be nicer as an interface than bash scripts, even if it won’t save from dealing with nitty gritty of the platform.

Re:Terraform 0.15 General Availability by vbsteven on Thursday April 15, 2021 @03:38AM (#26817284)
Indeed, I see TF mostly as an automation/idempotency tool for setting up and maintaining infrastructure components like Kubernetes clusters, ECS clusters, load balancers and all that. Something you would normally do with gcloud or aws-cli or by clicking around in GCP/AWS web console.

Yes, the "provider churn" is real: defaults on cloud platforms can change and platform-specific TF providers can change as well. The way I usually deal with that is to make sure my TF repo's stays in shape by applying them regularly. If you haven't applied a TF repo in over a year chances are real that your repo has rotted. In a similar way to an old Ansible playbook that has rotted because some apt packages have new dependencies. Keep applying them regularly and manage changes in small chunks instead of once a year.

Auditing is really useful. I would recommend preventing developers from applying TF to production directly, and have it all managed by a CI pipeline.

Also with some careful planning and structuring of your TF repository. It's pretty straightforward to duplicate environments: for example: spin up an extra development environment to experiment with a newer K8s version, or to validate some infrastructure changes.

Re:Terraform 0.15 General Availability by sciurus on Wednesday April 14, 2021 @08:37PM (#26814896)
> I guess TF makes a lot more sense for on-premise, bare metal VMs ?

No, not necessarily. If you have an API for orchestrating your on-premise infrastructure (openstack etc), then sure.

Where terraform shines is managing resources in your cloud provider.

I.E. creating the GCP project and GKE cluster that you need before you can apply your k8s YAML. Or creating the cloudsql databases, GCS buckets, etc that the apps running in your k8s cluster need.

Re:Terraform 0.15 General Availability by weitzj on Thursday April 15, 2021 @06:20AM (#26818563)
> I guess TF makes a lot more sense for on-premise, bare metal VMs ?

Sure. And also for Kubernetes json/yml. You don't just create infrastructure but it will also change -> delete/create.

And establishing a process how to delete your Kubernetes json/yml files in some form of codereview or in some way so you know what it actually deployed, you end up with a tool like terraform.

Re:Terraform 0.15 General Availability by kstrauser on Wednesday April 14, 2021 @04:29PM (#26812628)
"I just spent a couple of hours standing up a specific resource, which was kind of a pain in the neck. Oh, now I need 23 more of them! That'll take another 30 seconds to generate the configuration and create the new resources."

That's why I like Terraform.

Edit: "Also, I created a thing 2 years ago and now I need another one. Oh, here it is in Terraform." It's incredibly nice not to have to re-learn how to make that thing, and remember what its quirks and dependencies are.

In short, I consider Terraform exactly like I consider shell scripts: it's not perfect, not by a long shot, but once you have it working it tends to stay working and you don't have to invent it again the next time you need to do something.

Re:Terraform 0.15 General Availability by matwood on Wednesday April 14, 2021 @09:37PM (#26815262)
> "Also, I created a thing 2 years ago and now I need another one."

Let me add, this thing was changed 2 years ago to add an option...why? Git blame to the rescue...

I probably use git history more in my tf code than in my regular programming.

Re:Terraform 0.15 General Availability by pm90 on Thursday April 15, 2021 @02:46AM (#26816894)
Absolutely invaluable to have this, especially for infrastructure changes which often involve ugly hacks to get around limitations but which may just not be clear at first.
Re:Terraform 0.15 General Availability by solatic on Wednesday April 14, 2021 @04:28PM (#26812615)
Because the state exists regardless. When you write code that interfaces with AWS, there's always something that exists on the other side of the AWS API. The question you have is, how do you programmatically keep a copy of that state on your side of the API?

The naive approach that tries to do this without state in code goes something like:

a) invoke the remote API to look for something that should exist b) if it doesn't exist, invoke the remote API to create it c) use the property of the thing to do something else

What Terraform does is cleanly separate your state (don't look stuff up you already know exists) from the API (which is handled by an open-source library, aka the provider) from your configuration (which can now be declarative). Because the state and API calls are abstracted away, the configuration itself is much more clean, easy to reason about, and easier to maintain.

Re:Terraform 0.15 General Availability by mr_C on Wednesday April 14, 2021 @04:48PM (#26812857)
Maybe an unpopular opinion but what you just described as "naive" is arguably a better solution than Terraform's overengineering.

I use Ansible to manage multiple clouds (Openstack, AWS...etc) using a mix of custom modules and public collections. I don't need a "state", I couldn't care less if resources exist or not, upgrades between versions are smooth, module/collection upgrades doesn't interfere with all the existing resources we already have... every time I run a playbook I know that everything will end up just like I want it to be. Not bad for a naive approach I guess.

Re:Terraform 0.15 General Availability by diroussel on Wednesday April 14, 2021 @05:07PM (#26813095)
If you have an ansible playbook that creates a certain resource, and you delete that code. Next time you run it, it won’t delete the resource because there is no state management.

You have to add code to as only to be sure to remove the non-longer needed resource. But how long does that code need to stay there.

Ansible is supposed to engender a decorative approach, but it’s very easy to slip into procedural code. Whereas terraform is much more declarative.

Re:Terraform 0.15 General Availability by mr_C on Wednesday April 14, 2021 @05:23PM (#26813293)
Who cares if there is a dangling dns records somewhere or an extra allocated floating ip? In practice you could just set state:absent to whatever you are trying to remove or just remove it manually, the latter is most of the time faster than dealing with state management once you have a behemoth in prod that no one wants to break.
Re:Terraform 0.15 General Availability by ncallaway on Wednesday April 14, 2021 @08:01PM (#26814660)
> Who cares if there is a dangling dns records somewhere or an extra allocated floating ip

What if instead of a dangling dns record, its 15 large EC2 instances?

Yes, you can come up with examples of trivial dangling resources, but it's just as easy for me to come up with non-trivial examples of dangling resources.

Re:Terraform 0.15 General Availability by mr_C on Thursday April 15, 2021 @04:37AM (#26817720)
I came up with trivial examples because no one forgets about non-trivial resources. In my opinion, if you decrease some instance count from 18 to 3, you'd rather waste 1 minute deleting 15 instances than dealing with all the problems a state management brings to the table.
Re:Terraform 0.15 General Availability by ncallaway on Thursday April 15, 2021 @01:53PM (#26823880)
> no one forgets about non-trivial resources

The number of articles I've read about someone who left a non-trivial number of resources running unused in AWS and were later surprised by a large bill would seem to be a counterexample to that point.

Re:Terraform 0.15 General Availability by sciurus on Wednesday April 14, 2021 @08:40PM (#26814912)
> Who cares if there is a dangling dns records somewhere

You should care. This opens you up to subdomain takeovers, which have real security implications.

https://developer.mozilla.org/en-US/docs/Web/Security/Subdom...

Re:Terraform 0.15 General Availability by erik_seaberg on Wednesday April 14, 2021 @06:32PM (#26813917)
Some cloud resources will cost money every month, forever. (I think this is an unappreciated side of the AWS business model; it’s not cost-effective to have a dev confirm that each resource can be safely decomm’d.)

There’s also a risk that your legacy environments only work because some dangling resource wasn’t cleaned up, and a new clone of the environment will fail.

Re:Terraform 0.15 General Availability by ezrast on Thursday April 15, 2021 @01:29AM (#26816471)
> In practice you could just set state:absent to whatever you are trying to remove

If you do this, or in fact anything with Ansible, be REAL careful about double-checking what your tags actually match before committing. Since it doesn't track state, anything in your cloud environment is fair game.

I was not careful once, and that was a bad week for me.

Re:Terraform 0.15 General Availability by solatic on Thursday April 15, 2021 @12:54AM (#26816317)
Part of the benefit of Terraform is the ability to set up ephemeral resources and tear everything down afterwards with "terraform destroy", which is useful for setting up one-off experiments and tests. That kind of cleanup is completely impossible with Ansible.
Re:Terraform 0.15 General Availability by mr_C on Thursday April 15, 2021 @04:41AM (#26817772)
Not true, just set state to absent and run your play again.
Re:Terraform 0.15 General Availability by m1keil on Wednesday April 14, 2021 @09:07PM (#26815077)
You pay for some of these (like dangling IP addresses not in use) and some others have a max quota (like security groups).
Re:Terraform 0.15 General Availability by matwood on Wednesday April 14, 2021 @09:44PM (#26815310)
Any dangling resources may cost money and/or open up security concerns.
Re:Terraform 0.15 General Availability by llbeansandrice on Wednesday April 14, 2021 @04:59PM (#26813006)
I haven't used Ansible so maybe I'm incorrect here but aren't tf and Ansible solving slightly different problems?

Terraform feels like infrastructure management to me. We use it to provision underlying resources: Networking, Clusters, Nodes, Alerts, etc. All of the actual code deployments are entirely separate.

Ansible is more of a configuration management right?

Re:Terraform 0.15 General Availability by timdev2 on Wednesday April 14, 2021 @07:40PM (#26814490)
Right. And you can use something like terraform-inventory[0] as a dynamic inventory source in Ansible. So TF manages all the bits and bobs floating around in AWS, and then ansible takes over to manage configuration on whatever EC2 instances are involved.

[0]: https://github.com/adammck/terraform-inventory

Re:Terraform 0.15 General Availability by throwdbaaway on Wednesday April 14, 2021 @08:02PM (#26814666)
Not really. In terms of functionalities, I consider ansible as a superset of terraform, without all the state management stuffs. Distributed systems are hard, and I will just let my cloud providers to be the single source of truth of all the states.

Here's a comment from the author of ansible when terraform was first released: https://news.ycombinator.com/item?id=8100036

> One of the things shown in the Ansible examples are how to do a cloud deploy in one hop, i.e. request resources and also configure the stack all the way to the end, from one button press, and can also be used to orchestrate the rolling updates of those machines, working with the cloud load balancers and so on, throughout their entire life cycle -- all using just the one tool.

Re:Terraform 0.15 General Availability by ggregoire on Wednesday April 14, 2021 @06:35PM (#26813945)
I've never used them but there are modules to provision stuff on AWS: https://docs.ansible.com/ansible/latest/scenario_guides/guid...

Edit: seems limited to ec2 and s3: https://docs.ansible.com/ansible/latest/collections/amazon/a...

Although you could provision other services with some custom modules using the aws-cli.

Re:Terraform 0.15 General Availability by matwood on Wednesday April 14, 2021 @09:48PM (#26815335)
Yeah. I use packer and ansible to build/configure AMIs. Terraform manages the configuration that launches said AMIs (through things like autoscaling, etc...).
Re:Terraform 0.15 General Availability by matwood on Wednesday April 14, 2021 @09:43PM (#26815302)
> upgrades between versions are smooth

I use tf and ansible regularly. I wouldn't call ansible upgrades exactly smooth, they deprecate features just like anyone else.

Re:Terraform 0.15 General Availability by apple4ever on Thursday April 15, 2021 @12:43AM (#26816270)
How do you ensure you have say 2 web servers created and connected to a load balancer? Is that part of your custom module?
Re:Terraform 0.15 General Availability by mr_C on Thursday April 15, 2021 @04:25AM (#26817622)
Using ec2_instance_info to check if the instance exists filtering by name (eg: selectattr('tags.Name', 'defined') | selectattr('tags.Name', 'equalto', server_name) ) and then standard ec2_instance module, same for lb.
Re:Terraform 0.15 General Availability by darkwater on Wednesday April 14, 2021 @04:17PM (#26812470)
To answer your questions: you generally look in the Terraform docs which are well written and always updated, because autogenerated. The state forces you and especially the team to almost never touch things directly. And if you do, you feel nasty for it.
Re:Terraform 0.15 General Availability by 2ion on Wednesday April 14, 2021 @04:24PM (#26812548)
What else to use? All other tools operate at the same level as terraform, be it cloudformation or anything else. It's just drivers for the cloud API in question, each with their own drawbacks, idiocracies, limitations and workarounds. In a sense, these are all equal effort for the user.
Re:Terraform 0.15 General Availability by throwdbaaway on Wednesday April 14, 2021 @08:13PM (#26814746)
Ansible? Compared to terraform, it should require less effort from the user, without having to worry about states. Compared to aws cloudformation or gcp cloud deployment manager, it should require less effort as well, without having to learn the different idiosyncrasies of these proprietary tools.

> It's just drivers for the cloud API in question

Exactly, and if you are on gcp, both the ansible modules and the terraform modules are even generated from one code base: https://github.com/GoogleCloudPlatform/magic-modules

Re:Terraform 0.15 General Availability by akvadrako on Thursday April 15, 2021 @04:57AM (#26817907)
I used ansible for AWS a few years ago - it's terrible. Not by design, just that most of the modules are buggy and incomplete.
Re:Terraform 0.15 General Availability by verdverm on Wednesday April 14, 2021 @04:15PM (#26812452)
A manual fix requires a git commit, as most use Terraform with GitOps and IaC.

The point of using TF for many is reproducible infra and change approval workflows. So avoiding manual changes via the web consoles is what people are striving for.

Re:Terraform 0.15 General Availability by temp0826 on Wednesday April 14, 2021 @04:26PM (#26812574)
I spent a lot of time in both sets of docs for sure (along with perusing the console and creating test resources manually to see what options and switches I might be overlooking).

I think tf really shines when you start using multiple providers to manage things outside of the cloud though. In my last gig I was for example using the workspaces feature and auth0 provider to have separate auth stacks for our different envs, being able to use values created by one providers resources in another's was nifty

Re:Terraform 0.15 General Availability by llbeansandrice on Wednesday April 14, 2021 @04:24PM (#26812543)
There are docs for terraform which are fairly general and then provider specific docs as well (aws/gcp/azure/etc). We like it because it is easy to recreate environments and entire infrastructures. It also makes it much easier to review infrastructure changes.

>manually fix

You can if you need to. TF can ignore changes to certain aspects of resources like desired count of ECS tasks which might autoscale up and down and you can always make the change manually and then update the code to reflect the change. That's a bit of a no-no though. Like pushing code without a review.

Re:Terraform 0.15 General Availability by takeda on Thursday April 15, 2021 @02:53AM (#26816932)
Well... I'm puzzled about it on AWS. I don't know about Azure, but on GCP I tried their equivalent of CloudFormation and it was absolutely POS. Reported a bug in their repo, and got response acknowledging the bug, but it's in specific component's API that is owned by another team (within Google of course) and there's not much they can do about it.

Like WTF? Do they think I have better chance reaching them than them? Tried then TF and it was much simpler and everything worked as expected.

On AWS though, seriously, TF is inferior.

Re:Terraform 0.15 General Availability by csomar on Wednesday April 14, 2021 @08:58PM (#26815028)
> we cannot manually fix some minor mistakes of ours when creating resources because that'll mess up terraform

I find this to be the best advantage to Terraform. These small changes make your infrastructure harder to re-produce in the future; and unless you have strong documentation rules, you lost what is there in your infrastructure.

Re:Terraform 0.15 General Availability by matwood on Wednesday April 14, 2021 @09:31PM (#26815218)
> we cannot manually fix some minor mistakes of ours when creating resources because that'll mess up terraform

I had an old boss who fixed minor mistakes in stored procedures on the live server and then wondered why they broke again when builds went out...

Re:Terraform 0.15 General Availability by manojlds on Thursday April 15, 2021 @02:26AM (#26816762)
You can definitely manually fix something and then just have to update the terraform code and it should be fine. But it depends on what kind of manual fix you are doing as some changes are inplace while some require recreation.
Re:Terraform 0.15 General Availability by darthrupert on Thursday April 15, 2021 @01:48AM (#26816559)
You are not supposed to do anything manually ever in a cloud environment managed by a declarative tool. That's not just for terraform but for all of them.
Re:Terraform 0.15 General Availability by jen20 on Wednesday April 14, 2021 @04:36PM (#26812710)
You absolutely can manually fix and sync state:

`terraform refresh` does this.

(Disclaimer: I am an ex-core-maintainer of Terraform)