Terraform best-practices-and-common-mistakes-dev ops-west-2021
1 like77 views
The document summarizes a presentation on best practices and common mistakes for using Terraform for infrastructure as code. It covers topics like project structure, separating configurations from reusable modules, managing environments through variables, feature branching and CI/CD pipelines for Terraform workflows, and anti-patterns to avoid when building modules. The presentation provides guidance on structuring projects, making resources optional, and using Terraform appropriately while avoiding attempts to manage content or do configuration management with it.
Terraform best-practices-and-common-mistakes-dev ops-west-2021
- 1. Terraform Infrastructure as Code Best Practices and Common Mistakes Given by Derek C. Ashmore DevOps West 2021 June 9, 2021 ©2020 Derek C. Ashmore, All Rights Reserved 1
- 2. Who am I? • Professional Geek since 1987 • AWS since 2010 • Azure since 2017 • Terraform since the 0.5.x days • Specialties • Application Transformation • Infrastructure Automation • Yes – I still code! ©2021 Derek C. Ashmore, All Rights Reserved 2
- 3. Discussion Resources • This slide deck – https://www.slideshare.net/derekashmore/presentations • Sample code on my Github – https://github.com/Derek-Ashmore/ • Slide deck has hyper-links! – Don’t bother writing down URLs • Assumptions – You have used Terraform (at least played with it) – You know basic functionality • Keep track of your questions – live Q&A at the end ©2021 Derek C. Ashmore, All Rights Reserved 3
- 4. Agenda Intro and Level Set Environment Management Modularity Summary / Q&A ©2021 Derek C. Ashmore, All Rights Reserved 4
- 5. Terraform Terminology • Configuration vs Module – Terraform Module designed for reuse – Terraform configuration is the outer layer • TFVars files – Provides variable input for an environment • Like properties file – Used with the –var-file option ©2021 Derek C. Ashmore, All Rights Reserved • Resources – Controls a cloud asset • Data lookup – Searches for cloud assets • Variables – Different values per context • Function and Expressions – Built-ins that gather/manipulate values
- 6. How Terraform Works • Terraform is “Declarative” – Like SQL • Reads all files with extension .tf – Figures out execution order • Supports variables and functions • Plugin architecture – Supports many clouds and products ©2021 Derek C. Ashmore, All Rights Reserved
- 7. Agenda Intro and Level Set Environment Management Modularity Summary / Q&A ©2021 Derek C. Ashmore, All Rights Reserved 7
- 8. Project Structure • Separate Configurations from Modules – Documents what’s designed for reuse and what is not • TFVars files provide environment specifics – All environments use the same automation – Easy to add environments – Different back-end state per environment ©2021 Derek C. Ashmore, All Rights Reserved
- 9. Project Structure Anti-Pattern • Separate configurations per environment • The good – Code is often simpler – Easier to add/subtract capabilities per environment – Separate state if using local file system default • The bad – Has code duplication – Harder to establish new environments – Environments can be inconsistent • Works in dev, but not prod ©2021 Derek C. Ashmore, All Rights Reserved
- 10. Making Environment Differences Configurable • Use Conditionals – No “If-Then” capability – Boolean indicators • Resources using count • Dynamic blocks ©2021 Derek C. Ashmore, All Rights Reserved
- 11. Optional configuration through ‘try’ • Use for complex inputs with optional fields • Try suppresses exceptions • Specify Terraform defaults with null ©2021 Derek C. Ashmore, All Rights Reserved
- 12. Environment Management Best Practices • Always run Terraform through tooling, not on your desktop – CI/CD Tools such as Jenkins or Terraform Cloud – Benefits • Audit history • Terraform and provider version control • Consistent runtime environment • Always require a plan before the apply – Require approval step before going on to the apply • Utilize cloud security constructs – AWS IAM instance roles for Jenkins agents – Azure Managed Identities for Jenkins or Azure DevOps agents • Always use back-end state ©2021 Derek C. Ashmore, All Rights Reserved
- 13. Agenda Intro and Level Set Environment Management Modularity Summary / Q&A ©2021 Derek C. Ashmore, All Rights Reserved 13
- 14. Terraform Usage Evolution • In the beginning – Use Source Control – Use Back-end state • As #Coders grows – Feature branches – CI/CD Pipelines • As #Configurations grows – Separate repo for modules • Or Terraform registry – Implement versioning • Never use main/master! • Further reading ©2021 Derek C. Ashmore, All Rights Reserved
- 15. Feature Branching • DevOps Team Discipline is Key • Feature Branches – Never edit main/master directly! – Update using Pull Requests • Should live less than one day! – Single targeted enhancement – One developer only – Long-lived branches prone to merge conflicts – Prefer rebase to merge • Further reading ©2021 Derek C. Ashmore, All Rights Reserved
- 16. CI/CD Pipelines • Provides consistent runtime environment – Terraform version – Cloud security policy • Audit history / Admin security • Pipeline approvals – Force Plan execution – Force manual approval before apply or destroy – Automatic “Apply” nullifies benefit of doing the plan ©2021 Derek C. Ashmore, All Rights Reserved
- 17. Modularity Anti-Patterns • All of these examples come from the field – Module creation before it’s needed – Modules that only contain one resource – Inappropriate Data lookups in modules – Undocumented modules – Use modules referencing main/master ©2021 Derek C. Ashmore, All Rights Reserved
- 18. Module creation before it’s needed • Should have at least two consumers before module is created • Classic YAGNI • Hard to track down consumers after release • Impossible to remove unused modules ©2021 Derek C. Ashmore, All Rights Reserved
- 19. Modules that only contain one resource • Amounts to a thin proxy – No value-add • Unnecessary complexity • Not as well documented as the underlying Terraform resource • Every module should have at least two resources! ©2021 Derek C. Ashmore, All Rights Reserved
- 20. Inappropriate Data lookups in modules • Data lookups fail if nothing is found – Error if the consumer configuration creates the resource • Makes assumptions about execution context • Data lookups belong in configurations, not modules, as they do know context ©2021 Derek C. Ashmore, All Rights Reserved
- 21. Undocumented Modules • Force consumers to read/understand module code – Costs them time • Makes it hard to use • All modules should have a README.md: – Example module call – Release Notes – Variable list – Output list ©2021 Derek C. Ashmore, All Rights Reserved
- 22. Use modules referencing main/master • Always consume referencing specific versions – Version upgrades are planned work • Source code ©2021 Derek C. Ashmore, All Rights Reserved • Recipe for unplanned work – Consumers can break unexpectedly when modules change • Always version modules
- 23. Agenda Intro and Level Set Environment Management Modularity Summary / Q&A ©2021 Derek C. Ashmore, All Rights Reserved 23
- 24. Secrets Handling • Secrets include – Credentials (account/password) – SSL Certificates – SSH Keys • Manage secrets separately – Digital Vault • Terraform looks the secret up – CI/CD Pipeline “Secret” variable • Anti-pattern: Terraform generating password – Easy to get out of sync with reality – Secrets have different life-cycle ©2021 Derek C. Ashmore, All Rights Reserved
- 25. Simplicity is Key • Eliminate unused variables – Always remove dead code • Don’t replicate derived values – Derive once in locals and use • Variable defaults – Inappropriate defaults common • Environment-specific names • Globally unique names ©2021 Derek C. Ashmore, All Rights Reserved
- 26. Avoid the Hammer and Nail Problem • Terraform is good for: – Creating cloud assets – Changing attributes on cloud assets ©2021 Derek C. Ashmore, All Rights Reserved • Terraform is not good for: – Maintaining content on cloud assets • VM configuration management – Use Ansible, Chef, etc. • Image pipelines – Use Packer – Don’t “remote control” • Use Terraform to execute Ansible or Packer
- 27. Session Summary • How to structure projects • Manage environments using tfvars – Not configurations • How to make resources optional • Use CI/CD tooling • Appropriate uses for Terraform ©2021 Derek C. Ashmore, All Rights Reserved • Module Anti-Patterns – Module creation before it’s needed – Modules that only contain one resource – Inappropriate Data lookups in modules – Undocumented modules – Use modules referencing main/master
- 28. Thank you! • Derek Ashmore: – Blog: www.derekashmore.com – LinkedIn: www.linkedin.com/in/derekashmore • Connect Invites from attendees welcome – Twitter: https://twitter.com/Derek_Ashmore – GitHub: https://github.com/Derek-Ashmore – Book: http://dvtpress.com/ • Please fill out the survey form! • Click the “Subsessions” tab for live Q&A ©2021 Derek C. Ashmore, All Rights Reserved 28