AWS Community Day CPH 2024 - Three problems of Terraform
0 likes160 views
The document discusses three key problems with Terraform: dynamic state location, deploying identical configurations in multiple environments, and managing environment-specific parameters. It explores various solutions and issues related to resource referencing across different states, deployment setups, and the use of workspaces. Additionally, it touches on collaboration challenges and competition with platforms like OpenTofu, while highlighting the benefits of using specific automation tools like Terragrunt.
AWS Community Day CPH 2024 - Three problems of Terraform
- 1. NORDICS DGI Byen’s CPH Conference 2024
- 2. NORDICS The three problems of Terraform Andrey Devyatkin | 2024-05-07
- 6. www.fivexl.io | hello@fivexl.io Do it. Do it better. Do it right. Alex Lindsay
- 7. www.fivexl.io | hello@fivexl.io Andrey Devyatkin Co-Host @ DevSecOps Talks podcast Principal AWS Consultant AWS Community Builder Security and Identity Co-Founder @ FivexL AWS User Group Leader UG Las Palmas de GC
- 11. www.fivexl.io | hello@fivexl.io three conceptual problems Dynamic state location Deploying the same configuration to multiple environments Environment specific parameters A way to address differences between environments Cross-state resources lookup A need to reference resources from different states
- 15. www.fivexl.io | hello@fivexl.io $ ls README.md .terraform main.tf terraform.tfstate
- 16. www.fivexl.io | hello@fivexl.io terraform { backend "s3" { bucket = "my-cool-startup-infra-state" key = "terraform/main.tfstate" region = "us-east-1" } }
- 17. www.fivexl.io | hello@fivexl.io Assumptions AWS S3 backend
- 18. www.fivexl.io | hello@fivexl.io So far so good No need for the wrapper
- 20. www.fivexl.io | hello@fivexl.io We need to deploy the app to the second environment
- 22. www.fivexl.io | hello@fivexl.io We need to change backend configuration depending on env
- 23. www.fivexl.io | hello@fivexl.io terraform { backend "s3" { bucket = "my-cool-startup-infra-state" key = "terraform/main.tfstate" region = "us-east-1" } }
- 25. www.fivexl.io | hello@fivexl.io Can we use Terraform workspaces?
- 26. www.fivexl.io | hello@fivexl.io https://developer.hashicorp.com/terraform/cli/workspaces As of 2023-05-18
- 27. www.fivexl.io | hello@fivexl.io https://developer.hashicorp.com/terraform/cli/workspaces#when-not-to-use-multiple-workspaces As of 2023-05-18 Okay, what is the real life use case then? 🤔
- 28. www.fivexl.io | hello@fivexl.io https://developer.hashicorp.com/terraform/cli/workspaces#alternatives-to-workspaces As of 2023-05-18
- 31. www.fivexl.io | hello@fivexl.io But why so many directories? Can’t we just use the same directory somehow? 🤔
- 32. www.fivexl.io | hello@fivexl.io https://developer.hashicorp.com/terraform/language/settings/backends/configuration#partial-configuration As of 2023-05-18
- 33. www.fivexl.io | hello@fivexl.io terraform { backend "s3" { bucket = "my-cool-startup-infra-state" key = "terraform/main.tfstate" region = "us-east-1" } }
- 34. www.fivexl.io | hello@fivexl.io terraform { backend "s3" {} } terraform init -backend-config "bucket=my-cool-startup-infra-state" -backend-config "key=terraform/main.tfstate" -backend-config "region=us-east-1"
- 35. www.fivexl.io | hello@fivexl.io Do we share S3 bucket between environments?
- 36. www.fivexl.io | hello@fivexl.io Assumptions AWS dev/production S3 backend bucket/state per env with predefined name
- 37. www.fivexl.io | hello@fivexl.io terraform { backend "s3" {} } terraform init -backend-config "infra-state-7YYYYYYYYY62" -backend-config "key=terraform/main.tfstate" -backend-config "region=us-east-1"
- 38. www.fivexl.io | hello@fivexl.io Why did we name s3 bucket this way? 🤔
- 39. www.fivexl.io | hello@fivexl.io format("infra-state-%s", data.aws_caller_identity.current.account_id)
- 40. www.fivexl.io | hello@fivexl.io Would exposing the account id get us into trouble? 🤔
- 41. www.fivexl.io | hello@fivexl.io As of 2024-05-07 https://medium.com/@maciej.pocwierz/how-an-empty-s3-bucket-can-make-your-aws-bill-explode-934a383cb8b1
- 43. www.fivexl.io | hello@fivexl.io Control Tower naming convention
- 44. www.fivexl.io | hello@fivexl.io # debatable format("infra-state-%s-%s", data.aws_caller_identity.current.account_id, data.aws_region.r.name) # paranoid edition format("infra-state-%s", sha1( format( "%s-%s",data.aws_caller_identity.current.account_id data.aws_region.r.name)))
- 45. www.fivexl.io | hello@fivexl.io Why not just use env suffix like -prod or -dev? 🤔
- 46. www.fivexl.io | hello@fivexl.io If we are using the same dir then how to be with .terraform? 🤔
- 47. www.fivexl.io | hello@fivexl.io https://developer.hashicorp.com/terraform/cli/config/environment-variables#tf_data_dir As of 2023-05-18
- 50. www.fivexl.io | hello@fivexl.io AWS_DEFAULT_REGION env variable? Use aws-vault for env setup
- 51. www.fivexl.io | hello@fivexl.io $ ls README.md .terraform.7YYYYYYYYY62 .terraform.8XXXXXXXXX28 main.tf
- 53. www.fivexl.io | hello@fivexl.io How do we specify different parameters for different environments?
- 54. www.fivexl.io | hello@fivexl.io https://developer.hashicorp.com/terraform/language/values/variables#assigning-values-to-root-module-variables As of 2023-05-18
- 56. www.fivexl.io | hello@fivexl.io $ ls README.md 7YYYYYYYYY62.tfvars 8XXXXXXXXX28.tfvars .terraform.7YYYYYYYYY62 .terraform.8XXXXXXXXX28 main.tf $ cat 7YYYYYYYYY62.tfvars # dev instance_type = "t4g.micro" $ cat 8XXXXXXXXX28.tfvars # prod instance_type = "t4g.large"
- 59. www.fivexl.io | hello@fivexl.io Do we really need a wrapper for this?
- 61. www.fivexl.io | hello@fivexl.io What if we add more applications?
- 64. www.fivexl.io | hello@fivexl.io How do I get VPC id from network stack to my application stack?
- 66. www.fivexl.io | hello@fivexl.io Terragrunt remote state resoltion
- 67. www.fivexl.io | hello@fivexl.io AWS SSM Parameters AWS S3 Self-containing modules Are the other ways? Tooling
- 68. www.fivexl.io | hello@fivexl.io Self-contained modules? 🤔 Create resources Look up resources Provide policies …
- 70. www.fivexl.io | hello@fivexl.io two more problems Licensing Are you competing with HashiCorp? Team work How do you work together with the same state?
- 71. www.fivexl.io | hello@fivexl.io OpenTofu Not a big gap at the moment Unclear longevity Community requested features Has a momentum
- 72. www.fivexl.io | hello@fivexl.io Terraform is a cli tool That operates on a shared resources
- 73. www.fivexl.io | hello@fivexl.io Conventional CI/CD vs TACOS Terraform Automation and Collobaration Systems
- 74. www.fivexl.io | hello@fivexl.io Commit UnitTest Lint Build Deploy Test Promote CI/CD Server TACOS Server
- 75. www.fivexl.io | hello@fivexl.io Benefits of using TACOS Lock down access to the state Better visibility Less shoulders bumping Extra features https://www.reddit.com/r/Terraform/comments/lkylzk/scalr_vs_spacelift_vs_atlantis_vs_env0_bake_off/
- 76. www.fivexl.io | hello@fivexl.io Conclusion and recap
- 77. www.fivexl.io | hello@fivexl.io three conceptual problems Dynamic state location Deploying the same configuration to multiple environments Environment specific parameters A way to address differences between environments Cross-state resources lookup A need to reference resources from different states
- 78. www.fivexl.io | hello@fivexl.io two more problems Licensing Are you competing with HashiCorp? Team work How do you work together with the same state?
- 80. www.fivexl.io | hello@fivexl.io Tools like Terragrunt offer nice extra features Which are making going back harder
- 81. www.fivexl.io | hello@fivexl.io conventions vs wrappers