News for hackers. Stuff that matters.
Re:Mocking Terraform AWS Using Docker by JBerlinsky on Friday June 22, 2018 @12:32AM (#17371329)
I'm not sure I follow the use case here. Perhaps it's just a function of my use cases for Terraform.

Terraform comes with `terraform plan`, which does a cursory validation of the infrastructure-as-code, diffs it against the current state of the world in various accounts, and reports back with what it expects to execute. While there are some cases where `terraform plan` succeeds and `terraform apply` errors, they tend to be cases that I don't think mocking would catch. For example:

- Certain AWS resource naming restrictions are not handled by `terraform plan`, but are caught by the AWS API, causing `terraform apply` to fail.

- The validity of AWS ACM certificates (issued vs pending) is not handled by `terraform plan` -- `plan` assumes that any certificate ARN provided is valid and issued, whereas the AWS API will error if the certificate is not valid, prompting `terraform apply` to fail.

There are plenty more examples (duplicate security group rules come to mind as a particularly egregious offender). I'm just not sure I see what value this provides -- if I'm missing something, please do let me know!

Re:Mocking Terraform AWS Using Docker by diegopacheco on Friday June 22, 2018 @12:40AM (#17371364)
All fare and reasonable points JBerlinsky! All you said make sense to me.

IMHO the main use cases for me is: - Speed up development time by doing tests locally - don't need to commit and wait a build kick it and see whats wrong. - Able to be part of the bigger scheme where you can test your whole "infrastructure" or part of it at least we still limited by localstack.

Re:Mocking Terraform AWS Using Docker by JBerlinsky on Friday June 22, 2018 @12:49AM (#17371403)
To your points:

>> - Speed up development time by doing test locally - don't need to commit and wait a build kick it and see whats wrong

Combined with the article linked, it sounds like you're working in an environment where the only machine authorized to interact with AWS, or retain the Terraform statefile, is a Jenkins build server. Might I suggest investigating Terraform Enterprise (despite the name, it's rather affordable, for what you're getting)? TFE integrates with Github (and other VCS) pull requests, allowing a pull request to trigger a `terraform plan`. This gives a pretty good understanding of whether your changes are going to succeed or not, and are pretty quick to run -- I work on some rather large projects, and have never waited more than a few minutes for a plan to complete.

If you're allowed to pull down the statefile, you can also just run `terraform state pull > terraform.tfstate && terraform plan`.

>> - Able to be part of the bigger scheme where you can test your whole "infrastructure" or part of it at least we still limited by localstack.

In my experience, mocking systems like `localstack` are never quite up to the "standard" of the systems they intend to mock (AWS in this case). Of particular note, AWS's APIs, especially for newer products, tend to have lots of "gotchas" that are undocumented or hard to trace back -- things that are extremely unlikely to be covered well in a mocking framework.

If this works well for you, great! Just giving my two cents, as someone who has thought a lot about this and wound up right back at `terraform plan`.

Re:Mocking Terraform AWS Using Docker by diegopacheco on Friday June 22, 2018 @12:54AM (#17371431)
In the end of the day is about COST but we could have more machines doing builds and better ones but still takes more time and effort them being local. Terraform enterprise is not available for me. Since my project just uses Open Source besides AWS.

AWS moves pretty FAST, but I don't know anything better than LocalStack fro AWS mocking.

Re:Mocking Terraform AWS Using Docker by diegopacheco on Friday June 22, 2018 @12:58AM (#17371450)
I Also understand this is far from perfect and I appreciate all your comments. I just wish AWS had something like localstack with all endpoints mocked would make everybody life so much better. Not to mention if they could use same(consistent) names for everything i.e API, CLI, BOTO etc...
Re:Mocking Terraform AWS Using Docker by JBerlinsky on Friday June 22, 2018 @12:57AM (#17371443)
You might also want to look into https://github.com/runatlantis/atlantis, which provides the benefits of Terraform Enterprise, in an open-source, self-hosted format.
Re:Mocking Terraform AWS Using Docker by diegopacheco on Friday June 22, 2018 @01:01AM (#17371461)
I did not know that one - looks pretty nice. Thanks for sharing!
Re:Mocking Terraform AWS Using Docker by JBerlinsky on Friday June 22, 2018 @01:02AM (#17371468)
My pleasure! At the end of the day, whatever works best for your workflow is ideal, of course.
Re:Mocking Terraform AWS Using Docker by diegopacheco on Friday June 22, 2018 @12:59AM (#17371453)
I did not know that one - looks pretty nice. Thanks for sharing!
Re:Mocking Terraform AWS Using Docker by jmspring on Thursday June 21, 2018 @11:46PM (#17371128)
"Terraform is a good tool for infrastructure provisioning. However to test terraform it could be pretty difficult. So you will create some terraform scripts and upload to the cloud a run some slow Jenkins job?"

One can test Terraform scripts without requiring it be hooked into Jenkins. If I have an account with a supported provider, testing during development of said script(s) isn't really that hard.

Re:Mocking Terraform AWS Using Docker by diegopacheco on Friday June 22, 2018 @12:08AM (#17371212)
Yes, it's possible to test it is local if you have credentials. But for security reasons, lots of times companies don't provide credentials. Also, it's your local dev machine you don't want mess it up right? So docker is awesome being ephemeral.

Sometimes when you use PACKER and you have a SHARED infrastructure a build could take 40mim to run all the things with Terraform/Ansible that kind of a mainframe :D Unfortunately this exit :(

Re:Mocking Terraform AWS Using Docker by timmytwotime on Friday June 22, 2018 @09:49AM (#17373924)
I'd also add that as software is going CI/CD, so is infrastructure. Good points in this post OP.
Re:Mocking Terraform AWS Using Docker by diegopacheco on Friday June 22, 2018 @03:12PM (#17376553)
agree