Terraform modules and some of best-practices - March 2019
9 likes3,897 views
This document summarizes best practices for using Terraform modules. It discusses: - Writing resource modules to version infrastructure instead of individual resources - Using infrastructure modules to enforce tags, standards and preprocessors - Calling modules in a 1-in-1 structure for smaller blast radii and dependencies - Using Terragrunt for orchestration to call modules dynamically - Working with Terraform code by using lists, JSONnet, and preparing for Terraform 0.12
![- [ ] How to write modules?
- [ ] How to call modules?
- [ ] How to work with the code?](https://image.slidesharecdn.com/terraform-modules-meetup-singapore-2019-190318054343/85/Terraform-modules-and-some-of-best-practices-March-2019-32-320.jpg)
![- [x] How to write modules?
- [x] Do not write, if you can
- [x] Avoid providers in modules, provisioners
- [ ] How to call modules?
- [ ] How to work with the code?](https://image.slidesharecdn.com/terraform-modules-meetup-singapore-2019-190318054343/85/Terraform-modules-and-some-of-best-practices-March-2019-50-320.jpg)
![- [x] How to write modules?
- [x] How to call modules?
- [x] 1-in-1 works beter over time
- [x] Orchestration = Terragrunt
- [x] Dynamic values in tfvars
- [ ] How to work with the code?](https://image.slidesharecdn.com/terraform-modules-meetup-singapore-2019-190318054343/85/Terraform-modules-and-some-of-best-practices-March-2019-67-320.jpg)
![- [x] How to write modules?
- [x] How to call modules?
- [x] How to work with the code?
- [x] Lists in Terraform 0.11 can be painful
- [x] Perceive Terraform easier](https://image.slidesharecdn.com/terraform-modules-meetup-singapore-2019-190318054343/85/Terraform-modules-and-some-of-best-practices-March-2019-86-320.jpg)
![- [x] How to write modules?
- [x] How to call modules?
- [x] How to work with the code?
- [x] Terraform 0.12 beta!](https://image.slidesharecdn.com/terraform-modules-meetup-singapore-2019-190318054343/85/Terraform-modules-and-some-of-best-practices-March-2019-87-320.jpg)
Terraform modules and some of best-practices - March 2019
- 1. Terraform modules and some of best- practices Anton Babenko @antonbabenko March 2019
- 2. Anton Babenko Terraform AWS fanatic since 2015 Organiser of HashiCorp UG, AWS UG, DevOps Norway, DevOpsDays Oslo I 💚 open-source: terraform-community-modules + terraform-aws-modules antonbabenko/pre-commit-terraform — clean code and documentation antonbabenko/terraform-docs-as-pdf antonbabenko/modules.tf-lambda — generate Terraform code from visual diagrams www.terraform-best-practices.com medium.com/@anton.babenko @antonbabenko — Twitter, GitHub, Linkedin
- 3. Collection of open-source Terraform AWS modules supported by the community. More than 2 mil. downloads since September 2017. (VPC, Autoscaling, RDS, Security Groups, ELB, ALB, Redshift, SNS, SQS, IAM, EKS, ECS…) github.com/terraform-aws-modules registry.terraform.io/modules/terraform-aws-modules
- 4. Write, plan and manage infrastructure as code www.terraform.io
- 11. Google Cloud Deployment Manager Azure Resource Manager
- 14. Why Terraform and not AWS CloudFormation, Azure ARM, Google Cloud Deployment Manager? • Terraform manages 100+ providers, has easier syntax (HCL), has native support for modules and remote states, has teamwork related features, is an open-source project. • Provides a high-level abstraction of infrastructure (IaC) • Allows for composition and combination • Supports parallel management of resources (graph, fast) • Separates planning from execution (dry-run)
- 15. Terraform — universal tool for everything with an API GSuite Dropbox files and access New Relic metrics Datadog users and metrics Jira issues All Terraform providers
- 16. Let’s start!
- 18. Modules in Terraform are self-contained packages of Terraform configurations that are managed as a group.
- 19. Resource modules Create resources in a very flexible configuration Open-source
- 20. Resource modules
- 21. Resource modules
- 22. Resource modules
- 23. Resource modules
- 24. Q: Why use resource modules instead of resources? A: Resources can’t be versioned, but modules can. Resource modules
- 26. Infrastructure modules Consist of resource modules Enforce tags and company standards Use preprocessors, jsonnet, cookiecutter
- 31. Types of Terraform modules Resource modules (github.com/terraform-aws-modules , for eg) Infrastructure modules
- 33. Tip №0 Check Terraform Registry before writing resource modules
- 37. Size
- 39. Things to avoid in modules
- 40. Exception: logical providers (template, random, local, http, external) Providers in modules — evil
- 42. Provisioner — evil Avoid provisioner in all resources
- 43. Provisioner — evil Avoid provisioner in all resources
- 44. Provisioner — evil Avoid provisioner even in EC2 resources
- 45. Provisioner — evil Avoid provisioner even in EC2 resources
- 48. null_resource provisioner — good
- 49. Traits of good Terraform modules Documentation and examples Feature rich Sane defaults Clean code Tests Read more: http://bit.ly/common-traits-in-terraform-modules
- 51. Call Terraform modules Amount of resources and code keeps growing How to organize and call? How to orchestrate calls?
- 52. All-in-one Good: Declare variables and outputs in fewer places Bad: Large blast radius Everything is blocked at once Impossible to specify dependencies between modules (depends_on)
- 53. 1-in-1 Good: Smaller blast radius Possible to join invocation Easier and faster to work with Bad: Declare variables and outputs in more places
- 54. Which way do you group your code? All-in-one or 1-in-1?
- 55. Correct MFA (Most Frequent Answer): Somewhere in between
- 56. What kind of orchestration tool do you use? -target Makefile …
- 58. No really, do not try this at home!
- 64. tfvars can’t contain dynamic values :( Orchestration = Terragrunt
- 65. Orchestration = Terragrunt tfvars can’t contain dynamic values, so I have fixed it :)
- 66. before_hook + shell script https://github.com/antonbabenko/modules.tf-lambda/blob/master/templates/ terragrunt-common-layer/template/common/scripts/ update_dynamic_values_in_tfvars.sh or use modules.tf
- 68. Add new features Usually it is easy…
- 69. Create new or use existing
- 70. Create new or use existing
- 71. Create new or use existing
- 72. Create new or use existing
- 73. Work with lists
- 74. Work with lists
- 75. Work with lists
- 77. Work with stateful lists
- 78. Work with stateful lists
- 79. Work with stateful lists
- 80. Work with stateful lists
- 81. Integration
- 82. Integration
- 83. Auto-integration
- 84. Edge cases Different AWS regions (version of S3 signature, EC2 ClassicLink, IPv6) Date of creation of AWS account Limits on resources in AWS Services and features availability
- 85. Avoid in Terraform Not secret arguments should not be specified as command line arguments => put them in tfvars Reduce usage of "-target" and "-parallelism" "Terraform workspaces" evil in=> separate by directories Dependency hell in modules
- 88. Terraform 0.12 HCL2 — simplified syntax Loops ("for") Dynamic blocks ("for_each") Correct conditional operators (… ? … : …) Extended types of variables Templates in values Links between resources are supported (depends_on everywhere) Read more — https://www.hashicorp.com/blog/announcing-terraform-0-1-2-beta
- 89. Summary Write less and simpler (Terraform 0.12 won’t fix your code for you!) Use existing modules and utilities
- 90. How to handle secrets in Terraform? • Can you accept secrets to be saved in state file in plaintext? Probably not. • AWS IAM password & access secret keys — use PGP as keybase.io • AWS RDS — set dummy password and change after DB is created • AWS RDS — use iam_database_authentication_enabled = true • EC2 instance user-data + AWS KMS • EC2 instance user-data + AWS System Manager’s Parameter Store • AWS Secrets Manager • https://github.com/opencredo/terrahelp • Other options: • Secure remote state location (S3 bucket policy, KMS key)
- 91. What are the tools/solutions out there? • Terraform Registry — collection of public Terraform modules for common infrastructure configurations for any provider — https://registry.terraform.io/ • Terraform linter to detect errors that can not be detected by `terraform plan` — https://github.com/wata727/tflint • Terraform version manager — https://github.com/kamatama41/tfenv • A web dashboard to inspect Terraform States — https://github.com/ camptocamp/terraboard • Jsonnet — The data templating language — http://jsonnet.org • terraform-compliance - BDD style Terraform validation/compliancy check — https://github.com/eerkunt/terraform-compliance
- 92. Atlantis — Start working on Terraform as a team A unified workflow for collaborating on Terraform through GitHub, GitLab and Bitbucket https://www.runatlantis.io https://github.com/terraform-aws-modules/terraform-aws-atlantis
- 93. Bonus
- 98. ✓ cloudcraft.co — design, plan and visualize ✓ terraform-aws-modules — building blocks of AWS infrastructure ✓ Terraform — infrastructure as code
- 99. Infrastructure as code generator — from visual diagrams to Terraform https://github.com/antonbabenko/modules.tf-lambda Demo video: https://www.youtube.com/watch?v=F1Ax1zfZbiY
- 100. 1. Go to cloudcraft.co 2. Sign up, sign in (free account) 3. Draw your AWS infrastructure 4. Click "Export" 5. Click "Terraform code export" Try it yourself!
- 101. modules.tf — generated code ✓ Potentially ready-to-use Terraform configurations ✓ Suits best for bootstrapping ✓ Enforces Terraform best-practices ✓ Batteries included (terraform-aws-modules, terragrunt, pre-commit) ✓ 100% free and open-source (https://github.com/antonbabenko/ modules.tf-lambda) ✓ Released under MIT license