Configuration management II - Terraform
2 likes449 views
Terraform can be used to automate the deployment and management of infrastructure as code. It allows defining infrastructure components like VMs, networks, DNS records etc. as code in configuration files. Key benefits include versioning infrastructure changes, consistency across environments, and automation of deployments. The document then provides details on installing Terraform, using common commands like plan, apply and import, defining resources, variables, modules and managing remote state. It also demonstrates creating an EC2 instance using a generated AMI.
Configuration management II - Terraform
- 2. What do we have and what do we need? ● We have an automated system to build a provisioned image of a machine using Ansible + Packer. ● We need to ○ automate the deployment of the image we built in an easier way than shell scripts. ○ keep our entire infrastructure as code in order to version changes in it. 2@XSerrat
- 3. 3@XSerrat
- 4. Terraform > Why do we need it? ● Build, change and version our infrastructure using configuration files. ● We can use it as a documentation of our infrastructure. ● Be aware of what we have configured. ● Be able to manage our entire infrastructure (instances, repositories, DNS, and other SaaS) using the same syntax: Terraform configuration files. ● We can re-use our configuration to build new applications. 4@XSerrat
- 5. Terraform > More benefits ● Avoiding any surprises when Terraform manipulates our infrastructure: execution plan step shows what terraform will do before being executed. ● Graph of all infrastructure components. ● Building non-dependent components in parallel to increase efficiency. ● Avoiding possible human errors when we are doing complex changes. ● Manages ANYTHING that has an API. 5@XSerrat
- 6. Terraform > Installation ● We can find the package to install Terraform here: https://www.terraform.io/downloads.html ● In MacOS you can use the following command: $ brew install terraform 6@XSerrat
- 7. Terraform > CLI > Usage 7@XSerrat
- 8. Terraform > CLI > Usage > init ● It initializes all we need to apply changes to our infrastructure. ● It downloads new providers we have just add to the project. https://www.terraform.io/docs/commands/init.html 8@XSerrat
- 9. Terraform > CLI > Usage > plan ● It loads all files and shows the changes that Terraform has detected when it has compared our infrastructure with Terraform configuration files. https://www.terraform.io/docs/commands/plan.html 9@XSerrat
- 10. Terraform > CLI > Usage > apply ● It shows what Terraform will change of our infrastructure as plan command. ● It also asks if we want to apply the changes typing yes . https://www.terraform.io/docs/commands/apply.html 10@XSerrat
- 11. Terraform > CLI > Usage > graph ● It allows the generation of a graph of the resources we have configured with Terraform and how they are connected. https://www.terraform.io/docs/commands/graph.html 11@XSerrat
- 12. Terraform > CLI > Usage > import ● It allows us to retrieve the whole resources of our infrastructure and set them to the Terraform state. ● In the future, this command allows us to generate also the Terraform configuration of an existing infrastructure! https://www.terraform.io/docs/commands/import.html 12@XSerrat
- 13. Terraform > Configuration files ● Terraform uses sets of text files called configuration files using the syntax HCL (HashiCorp configuration language). ● Two formats: ○ Terraform format (ending: *.tf): Recommended format because it supports comments and it is more human-readable than JSON. ○ JSON format (ending: *.tf.json): It is the format used for machines to create / modify. 13@XSerrat
- 14. Terraform > Configuration files ● When we execute Terraform, it loads all .*tf files of the same directory and then, it shows what changes will do. ● Files are loaded in alphabetical order except “overrides” files that are loaded at the end overriding the previous files components. An override file can be “override.tf” or a file ended by “_override.tf”. ● Except “override” files, if there are two resources with the same name in the other files, a validation error will occur when executing Terraform. 14@XSerrat
- 15. Terraform > Configuration files > Components ● In Terraform configuration files there are some components: ○ Provider: Responsible to creating and managing resources. ○ Resource: Defines a resource that exists in the current infrastructure such as an EC2 instance. ○ Data source: It allows a way to fetch data outside Terraform configuration. ● The order of the definition of each component including variables does not matter. 15@XSerrat
- 16. Terraform > Components > Providers ● Manage the lifecycle of a resource: create, read, update and delete. ● By default, the name of all resources of a specific provider starts by the name of the provider. e.g: the resource aws_instance belongs to aws provider. example.tf Usage: https://www.terraform.io/docs/configuration/providers.html Providers: https://www.terraform.io/docs/providers/index.html 16@XSerrat
- 17. Terraform > Components > Providers ● When a new provider is used in our configuration files, we need to download and initialize it using a specific command: $ terraform init *Note that this command needs to be executed in the directory where our configuration files are placed. 17@XSerrat
- 18. Terraform > Components > Resources ● A resource is a component of our infrastructure. It can be a virtual machine, a container, dns records, an email provider... example.tf Usage : https://www.terraform.io/docs/configuration/resources.html AWS resources: https://www.terraform.io/docs/providers/aws/index.html 18@XSerrat
- 19. Terraform > Components > Resources > Dependencies ● Resources normally depends on other resources. But, when a resource does not depend on another it is created in parallel. aws_eip depends on the aws_instance aws_instance without any dependency 19@XSerrat
- 20. Terraform > Components > Resources > Provider ● It is possible we have multiple providers. For example, multiple aws providers due to different regions. ● Resources have a provider meta-parameter to associate the resource to the expected provider: west provider east provider 20@XSerrat
- 21. Terraform > Components > Data Sources ● Data sources allow Terraform configurations a way to retrieve information from our infrastructure that is not present in configuration files. ● They can also be used to compute new values on the fly. Data source of the most recent aws_ami who has a tag “Component” with the value “web” https://www.terraform.io/docs/configuration/data-sources.html 21@XSerrat
- 22. Terraform > Provisioners ● We use provisioners to execute scripts on local or remote machine as a part of resource creation or destruction. Provisioner who executes a script locally 22@XSerrat
- 23. Terraform > Provisioners ● By default provisioners are executed on creation time. ● We can add the meta-parameter when = “destroy” to execute the provisioner on destruction time. Provisioner executed on creation time Provisioner executed on destruction time https://www.terraform.io/docs/provisioners/index.html 23@XSerrat
- 24. Terraform > Provisioners > Taint status ● When a provisioner is executed on creation or destruction and the execution fails, the resource is marked as taint. ● When a resource is marked as taint, the next time we execute terraform apply, the creation or destruction will be executed again. ● It is important to execute scripts that can be executed multiple times without problems. 24@XSerrat
- 25. Terraform > Variables ● We can define variables to parametrize our configurations. ● Two types: ○ Input variables ○ Output variables https://www.terraform.io/docs/configuration/variables.html 25@XSerrat
- 26. Terraform > Variables > Input variables > Assignment ● We can move variables into another file with *.tf extension: ● Terraform will load all files ending in .tf of the same directory. variables.tf required optional 26@XSerrat
- 27. Terraform > Variables > Input variables > Assignment ● We can assign variables also from: ○ Command-line flags: We can pass them with any kind of Terraform command $ terraform apply -var 'access_key=foo' -var 'secret_key=bar' ○ File: We can define a terraform.tfvars or *.auto.tfvars and Terraform populate variables with this values. Also we can pass the file as a CLI parameter. $ terraform apply -var-file=“secret.tfvars” -var-file “production.tfvars” secret.tfvars 27@XSerrat
- 28. Terraform > Variables > Input variables > Assignment ○ Environment variables: We can define TR_VAR_access_key and then Terraform populate the value of the ENV variable to the access_key variable. Only for string types! ○ UI input: When we execute apply Terraform asks for variables without a default value. variables.tf These variables will be prompted 28@XSerrat
- 29. Terraform > Variables > Input Variables > Usage ● We use the variables in our providers, resources and data sources using interpolations. file.tf using variables 29@XSerrat
- 30. Terraform > Variables > Types ● Lists ● Maps variables.tf terraform.tfvars Dynamic lookup Static lookup variables.tf terraform.tfvars 30@XSerrat
- 31. Terraform > Variables > Output variables ● Due to the huge quantity of variables that Terraform manages, as a user we want to know only some of them. We can specify it using output variables. ● Each output variable will be printed when apply command is executed. outputs.tf 31@XSerrat
- 32. Terraform > Variables > Local variables ● They are equivalent to local variables in any programming language. 32 https://www.terraform.io/docs/configuration/locals.html @XSerrat
- 33. Terraform > Modules ● Modules are self-contained packages of Terraform configuration that are managed as a group. They allow us: ○ Reuse components ○ Improve the organization ○ Try to treat pieces of infrastructure as a black box ● A module contains: ○ Other modules ○ Input variables ○ Output variables ○ Resources https://www.terraform.io/docs/modules/index.html 33@XSerrat
- 34. Terraform > Modules > Registry ● Terraform Registry is a repository of modules written by the Terraform community. 34 https://registry.terraform.io/ @XSerrat
- 35. Terraform > Modules > Registry 35Autoscaling module for AWS @XSerrat
- 36. Terraform > Modules > Inputs ● A module has a specific name like resources and the mandatory input variable “source” that contains the path where this module can be retrieved. ● The source value can contain a module from Terraform registry, a repository or a directory path to a custom module. 36@XSerrat
- 37. Terraform > Modules > Outputs ● A module can have some output variables about what the module has just created. ● To use a variable we need to use this nomenclature: module.<module_name>.<output_variable_name> ● In this example we are printing the output variable “asg_name_servers”: example.tf Output shown in the shell 37@XSerrat
- 38. Terraform > Modules > Execution ● After adding a module we need to execute terraform init. ● Then we can execute terraform apply: ● A module can contain nested modules inside in order to decompose complex systems into manageable components. 38@XSerrat
- 39. Terraform > Modules > Execution ● Then, if we execute the terraform destroy, Terraform will also destroy all resources created by the module: 39@XSerrat
- 40. Terraform > Modules > Structure ● We can create a module and use it in our Terraform configuration. A module has this shape: ● Basic files: main.tf, variables.tf and outputs.tf . 40 Module example: https://github.com/hashicorp/terraform-aws-consul @XSerrat
- 41. Terraform > State ● Stores all managed resources of the infrastructure. ● Maps real resources to our Terraform configuration. ● Improve performance for large infrastructures: ○ In a small infrastructure, Terraform plan and apply refresh the state from the real infrastructure. ○ In a large infrastructure, the use of the state file it is important to Terraform to work well. ● It is a file created in the working directory called terraform.tfstate. ● We can inspect and modify the state using a the CLI terraform state with options to restore a previous state (Terraform stores a backup) 41 More information: https://www.terraform.io/docs/state/index.html State CLI usage: https://www.terraform.io/docs/commands/state/index.html @XSerrat
- 42. Terraform > State > Remote state: Backend ● Determines how the state is loaded and how an operation such as apply is executed. ● By default, Terraform uses such a “local” backend. ● Using a remote state we obtain the following benefits: ○ Working in a team: share the state with teammates ○ Keeping sensitive information (the state) off disk ○ Remote operations ● Backends are responsible for locking the state when someone else is making changes in the state file. Not all backends support locking. 42 Backends available: https://www.terraform.io/docs/backends/types/index.html More info about remote state: https://www.terraform.io/docs/backends/types/remote.html @XSerrat
- 43. Terraform > Workspaces ● Each Terraform configuration has an associated backend to define how operations are executed and where the Terraform state is stored. ● By default, there is a workspace called “default” ● They are useful when we want to test some changes before modifying the main production infrastructure. 43 https://www.terraform.io/docs/state/workspaces.html @XSerrat
- 44. Terraform > Extras > PhpStorm Plugin ● We have a PhpStorm plugin to support the HCL language: ● Highlights, navigation and autocompletion! Github repository: https://github.com/VladRassokhin/intellij-hcl 44@XSerrat
- 45. Terraform > Extra info ● Best practices and some extra tools: https://es.slideshare.net/AntonBabenko/terraform-modules-and-bestpractices- september-2018 ● All interesting presentations by Anton Babenko, active contributor to Terraform: https://es.slideshare.net/AntonBabenko/presentations 45@XSerrat
- 46. DEMO: Create an instance using a generated AMI 46