MongoDB World 2019: Terraform New Worlds on MongoDB Atlas
2 likes902 views
Melissa Plunkett, a Senior Product Manager at MongoDB, delivers a training session on using MongoDB Atlas with Terraform for cloud automation, emphasizing the importance of APIs and infrastructure as code. The presentation also introduces HashiCorp's Vault for managing secrets and improving data security, while outlining future plans for integrating MongoDB features with these tools. Participants are encouraged to provide feedback and connect during the event.
![Create a Cluster:
"US_EAST_1" : {
"analyticsNodes" : 0,
"readOnlyNodes" : 1,
"electableNodes" : 2,
"priority" : 6 }
}
}]
}'](https://image.slidesharecdn.com/mdbw19-mplunkett-final-givenatworld-190717211840/85/MongoDB-World-2019-Terraform-New-Worlds-on-MongoDB-Atlas-20-320.jpg)
![Create a MongoDB Database User in the Project:
-X POST
"https://cloud.mongodb.com/api/atlas/v1.0/groups/5ce4d39d79358
e873fc12c1f/databaseUsers?pretty=true"
--data '
{ "databaseName" : "admin",
"roles" : [ {
"databaseName" : "admin",
"roleName" : "readWriteAnyDatabase"
} ],
"username" : "spock2",
"password" : "badpass123” }'](https://image.slidesharecdn.com/mdbw19-mplunkett-final-givenatworld-190717211840/85/MongoDB-World-2019-Terraform-New-Worlds-on-MongoDB-Atlas-21-320.jpg)
![Add an IP to the Project IP Whitelist:
--request POST
"https://cloud.mongodb.com/api/atlas/v1.0/groups/5ce4d39d79358
e873fc12c1f/whitelist?pretty=true"
--data '
[
{
"ipAddress" : "192.0.0.15",
"comment" : "IP address added via API"
}
]'](https://image.slidesharecdn.com/mdbw19-mplunkett-final-givenatworld-190717211840/85/MongoDB-World-2019-Terraform-New-Worlds-on-MongoDB-Atlas-22-320.jpg)
![HashiCorp’s Vault
• Atlas community Secrets Engine examples:
$ vault write database/roles/readonly db_name=atlas
creation_statements=‘{ "db": "admin", "roles": [{ "role":
"readAnyDatabase" }] }' default_ttl="1h" max_ttl="24h"
$ vault write database/config/atlas plugin_name=atlas
allowed_roles="readonly" apiID="public API key"
apiKey="private API key" groupID=”project id"
$ vault read database/creds/readonly](https://image.slidesharecdn.com/mdbw19-mplunkett-final-givenatworld-190717211840/85/MongoDB-World-2019-Terraform-New-Worlds-on-MongoDB-Atlas-42-320.jpg)
MongoDB World 2019: Terraform New Worlds on MongoDB Atlas
- 1. Melissa Plunkett, Sr. Product Manager Terraform New Worlds on MongoDB Atlas themantissa Melissa Plunkett, Sr. Product Manager Terraform New Worlds on MongoDB Atlas themantissa
- 2. Welcome! • Sr. Product Manager in Cloud, focus Automation • Formally a Solution Architect • Before that an ops gal – Started with SGI IRIX and Red Hat pre RHEL. • Tech Mantra: Learn it, Master it (enough). Script it. Rest. Repeat. • And a Trekkie so …
- 3. Welcome Cadets! Starfleet Academy for Automation
- 5. Cadets, Have You Used? • MongoDB Atlas? • MongoDB Atlas API? • Terraform? • Vault? • Bonus!!! Star Trek Fan? Star Wars Fan? Both?
- 6. MongoDB Atlas Starfleet is focused on building applications that allow us to explore strange new worlds & seek out new civilizations • MongoDB Atlas was selected earlier this year • Focus on apps not ops • Can quickly spin up on multiple cloud providers in multiple planetary* regions! • Easy to use UI *currently limited to Earth
- 7. MongoDB Atlas UI Demo
- 8. MongoDB Atlas – Exploring the API MongoDB Atlas API means UI isn’t required to do much of what we need to do repeatedly to support developers • REST • API key w/ Digest Auth over HTTPS • API keys has RBAC - capabilities match the permissions assigned to the Key • API key whitelist https://docs.atlas.mongodb.com/api/
- 9. MongoDB Atlas – Exploring the API - Resources Organization Projects {Group-ID} Clusters Project IP Whitelist Database Users Invoices Custom MDB Roles Global Cluster Configuration Alerts Alert Configurations Maintenance Window LDAP Configuration Continuous Backup Snapshots Continuous Backup Snapshots Schedule Continuous Backup Restore Job Cloud Provider & On Demand Snapshots Cloud Provider Snapshot Restore Job Cloud Provider Snapshot Bkup Policy Checkpoints VPC Programmatic API Keys Monitoring and Logs* Performance Advisor* Auditing Encryption at Rest Root Atlas Users Events Teams https://docs.atlas.mongodb.com/api/
- 10. MongoDB Atlas –API Resources Reference Organization Projects {Group-ID} Clusters Project IP Whitelist Database Users Invoices Custom MDB Roles Global Cluster Configuration Alerts Alert Configurations Maintenance Window LDAP Configuration* Continuous Backup Snapshots Continuous Backup Snapshots Schedule Continuous Backup Restore Job Cloud Provider & On Demand Snapshots Cloud Provider Snapshot Restore Job Cloud Provider Snapshot Bkup Policy Checkpoints VPC Programmatic API Keys Monitoring and Logs^ Performance Advisor^ Auditing Encryption at Rest* Root Atlas Users Events CRU R ✓ ✓ ✓ ✓ ✓ ✓ RU R ✓ ✓ ✓ RU R R RUDRU CR R RU ✓ ✓ ✓ CRD CRD Teams ✓✓ https://docs.atlas.mongodb.com/api/
- 11. MongoDB Atlas – Exploring the API Two key things to know the location of before you get started … • Project ID • Organization ID
- 12. MongoDB Atlas – Exploring the API Organization ID
- 13. MongoDB Atlas – Exploring the API Project ID (aka Group ID)
- 14. MongoDB Atlas – Exploring the API Create a Programmatic API Key either at the Organization or Project level: • Org level: all the available permissions are for the org level. • Project level: all the available permissions are for the project level • BUT all project keys can be seen in the org level as an Org Member.
- 15. MongoDB Atlas – Exploring the API @ Org level go to Access
- 16. MongoDB Atlas – Exploring the API @ Project level go to Access Management
- 17. Create a Cluster: --request POST "https://cloud.mongodb.com/api/atlas/v1.0/groups/5ce4d39d79358 e873fc12c1f/clusters?pretty=true" --data ' { "name" : "MDBW19-Class-Cluster2-API", "diskSizeGB" : 100, "autoScaling" : { "diskGBEnabled" : false }, "clusterType":"REPLICASET", "providerBackupEnabled" : true,
- 18. Create a Cluster: "providerSettings" : { "providerName" : "AWS", "diskIOPS": 340, "instanceSizeName" : "M30", "volumeType" : "PROVISIONED” },
- 19. Create a Cluster: "replicationSpecs" : [ { "numShards" : 1, "regionsConfig" : { "US_WEST_1" : { "analyticsNodes" : 1, "electableNodes" : 3, "readOnlyNodes" : 0, "priority" : 7},
- 20. Create a Cluster: "US_EAST_1" : { "analyticsNodes" : 0, "readOnlyNodes" : 1, "electableNodes" : 2, "priority" : 6 } } }] }'
- 21. Create a MongoDB Database User in the Project: -X POST "https://cloud.mongodb.com/api/atlas/v1.0/groups/5ce4d39d79358 e873fc12c1f/databaseUsers?pretty=true" --data ' { "databaseName" : "admin", "roles" : [ { "databaseName" : "admin", "roleName" : "readWriteAnyDatabase" } ], "username" : "spock2", "password" : "badpass123” }'
- 22. Add an IP to the Project IP Whitelist: --request POST "https://cloud.mongodb.com/api/atlas/v1.0/groups/5ce4d39d79358 e873fc12c1f/whitelist?pretty=true" --data ' [ { "ipAddress" : "192.0.0.15", "comment" : "IP address added via API" } ]'
- 23. Create a Quick Cluster via API
- 24. MongoDB Atlas – Exploring the API API is great but… • Requires us to really know the API: e.g. updating via PATCH, or delete via DELETE, etc. • Going to possibly take a lot of time and money. • What if we could just describe what we want and change that when we need to modify it?
- 25. Emergency Transmission Top Secret From: Starfleet Command To: Starfleet Engineering MongoDB Cluster needed STAT. Project Genesis is at risk. Deploy at once! Khan seen in sector. Red Alert recommended. . . . .
- 26. Emergency Transmission . . . .
- 27. HashiCorp’s Terraform Terraform is an open source, declarative Infrastructure as Code (IaC) tool. • Describe desired infrastructure in a configuration file • Terraform takes that file and creates a plan to implement it • One can review the plan, and if all good, apply to create • We can create, change and destroy infrastructure easily!
- 28. HashiCorp’s Terraform We can change and destroy infrastructure easily??? • Terraform keeps a state file of what it has done • Change your configuration it only implements the diff • Can import a resource and then apply changes, e.g. import in a cluster and then set it to a bigger size. • Terraform destroy really destroys!
- 29. HashiCorp’s Terraform Terraform is a plugin based system: Base diagram from hashicorp.com *.tf 1 2 3 4 5 *.tfstate The Provider understands both Terraform and the Upstream API Terraform Core doesn’t “know” any infrastructure, that’s the Plugins purpose. You simply need to define what you want in the .tf file, do not need to know the API deeply CRUD
- 30. variable "mongodb_atlas_api_pub_key" { default = ”PUB-API-KEY"} variable "mongodb_atlas_api_pri_key" { default = ”PRI-API-KEY" } variable "database_username" { default = "spock3" } variable "database_user_password" { default = "blahblah1236" } variable "mongodb_atlas_whitelistip” { default = "192.155.1.1" } variable "mongodb_atlas_org_id" { default = "599eed269f78f769464d1bc3"} variable "mongodb_atlas_project_id" { default = "5ce4d39d79358e873fc12c1f" } Main.tf - Variables Keys and passwords !? Not a great practice and angers Admiral Kirk as Khan could steal these L But we’ll address this soon.
- 31. # Configure the MongoDB Atlas Provider provider "mongodbatlas" { username = "${var.mongodb_atlas_api_pub_key}" api_key = "${var.mongodb_atlas_api_pri_key}" } Main.tf – Configure the Provider
- 32. resource "mongodbatlas_cluster" "cluster" { name = "MDBW19-Class-Cluster3-TF" group = "${var.mongodb_atlas_project_id}" mongodb_major_version = "4.0" provider_name = "AWS" region = "" size = "M30" disk_size_gb = 100 backup = false provider_backup = true disk_gb_enabled = false replication_factor = 0 Missing? -Disk IOPS -Anything related to replicationSpecs - this uses the deprecated replicationSpec option. -numShards (but it has a default!) Main.tf - Cluster A few issues
- 33. replication_spec { region = "US_WEST_1" priority = 7 read_only_nodes = 0 analytics_nodes = 1 electable_nodes = 3 } Main.tf – Cluster Continued replication_spec { region = "US_EAST_1" priority = 6 read_only_nodes = 1 analytics_nodes = 0 electable_nodes = 2 } }
- 34. # Create a Database User resource "mongodbatlas_database_user" "test" { username = "${var.database_username}" password = "${var.database_user_password}" database = "admin" group = "${var.mongodb_atlas_project_id}" roles { name = "readWriteAnyDatabase" database = "admin" } } Main.tf – Database User
- 35. # Create an IP Whitelist resource "mongodbatlas_ip_whitelist" "test" { group = "${var.mongodb_atlas_project_id}" ip_address = "${var.mongodb_atlas_whitelistip}” comment = "Added with Terraform" } Main.tf – IP Whitelist
- 36. HashiCorp’s Terraform Let’s configure a cluster, MongoDB user and Project IP whitelist with Terraform • Providers can be verified/official (both Hashicorp & Community created) OR community created but not verified • MongoDB Atlas Community Provider created by Akshay Karle: https://github.com/akshaykarle/terraform-provider- mongodbatlas/ (Thank you Akshay!)
- 37. Create a Quick Cluster via Terraform
- 38. Cadets Save Day and Defeat Khan!
- 39. Security of the Mission We have to ensure we handle data securely. Security layers we have utilized thus far: • API communication over HTTPS • RBAC to limit what the API key can do • IP Whitelist But the more secure layers the better so what else can we do?
- 40. HashiCorp’s Vault • Manage secrets & protect sensitive data • Focus here - managing secrets, especially dynamic secrets • Secret Engine extends Vault to systems, e.g. physical like a HSM or a service in the Cloud like Atlas. Vault is another HashiCorp product designed for security in the Cloud:
- 41. HashiCorp’s Vault • Database Secrets Engine: generate database credentials dynamically based on configured roles, including for MDB. • Atlas creates database users differently (via API) so need a Secrets Engine for that. • Ta da! There’s a community Secrets Engine at https://github.com/mealal/vault-atlas-plugin written by the amazing Alexey Menshikov, a Consulting Engineers @MDB!
- 42. HashiCorp’s Vault • Atlas community Secrets Engine examples: $ vault write database/roles/readonly db_name=atlas creation_statements=‘{ "db": "admin", "roles": [{ "role": "readAnyDatabase" }] }' default_ttl="1h" max_ttl="24h" $ vault write database/config/atlas plugin_name=atlas allowed_roles="readonly" apiID="public API key" apiKey="private API key" groupID=”project id" $ vault read database/creds/readonly
- 43. HashiCorp’s Vault • What about a Provider for Vault? Yes! https://www.terraform.io/docs/providers/vault/index.html • Read the cautions on using the Provider for Vault and follow the best practices to be as secure as possible!
- 44. Cadets Save Day and Defeat Khan!
- 46. MongoDB + Hashicorp – Better together! Later in 2019 MongoDB will have an official: Ø MongoDB Atlas Secrets Engine for Vault Ø MongoDB Atlas Terraform Provider
- 47. Be the First in the Know! • Go to https://bit.ly/2WS4WMJ • Fill out the brief survey, 6 required q’s, 3 not! < 2m • You will be notified on the progress for which ever mission you are interested in (Vault/Terraform/both)
- 48. Questions? Where: Atlas Booth in the Partner Pavilion (A1) When: 1:40 - 5:05 PM today! or Melissa.Plunkett@mongodb.com anytime! Github: themantissa
- 49. Please provide Session Feedback 1. MDB World App -> Menu -> “Rate a Session” -> Regent OR 1. Go to slido.com 2. Enter event code #MDBW19 3. Click on Regent Feedback poll will remain open for 10 minutes after the talk ends YOU ARE IN REGENT
- 50. Thank you!