How HashiCorp platform tools can make the difference in development and deployment
- 1. How HashiCorp platform tools can make the difference in development and deployment Dmytro Mykhailov
- 2. Dmytro Mykhailov Solution Architect • 3,5 years in EPAM • >12 years in IT • Microsoft Stack: • .NET • Azure skype: dmitry.mikhaylov email: dmytro_mykhailov@epam.com blog: https://devraccoon.com/
- 3. Problems in applications • Holy Server • Messy configuration • Slow environments setup • Works-on-my-machine syndrome • Expensive resources that do nothing • Hard non-consistent integration and end-to-end test environments • Where is your disaster recovery? • Are your secrets safe?
- 4. Holy Server • Thing that can be treated as benefits: - Easy to setup - No need of delivery infrastructure upfront - No need of tools knowledge • Drawbacks: - Hard to maintain - Service failure is a catastrophe - Custom knowledge of how to configure similar server
- 5. Immutable Infrastructure for Rescue Immutable infrastructure is an approach to managing services and software deployments on IT resources wherein components are replaced rather than changed. An application or services is effectively redeployed each time any change occurs. © WhatIs.com
- 6. Immutable Infrastructure for Rescue Image of WEB v32.2 Public portal v4.3 Admin portal v1.2 Database v2.1 New feature CI ToolNotify Build Developer DBA DevOps
- 7. Immutable Infrastructure for Rescue WEB v32.1 WEB v32.1 Backend Service A v12.34 API GW LB Backend Service B v3.2 Database A Database B WEB v32.1
- 8. Messy Configuration – Reactive Configuration Database Config Config Configuration Data Dev QA Prod
- 9. Messy Configuration – Reactive Configuration Database Config Config Dev Config DB Config DB Config DB QA Prod Common Config Seed
- 10. Messy Configuration – Reactive Configuration Database ConfigDB Master ConfigDB Agent ConfigDB Agent ConfigDB Agent
- 11. Environments Setup, Disaster Recovery - Infrastructure-as-a-Code
- 12. Tests, Works-on-my-machine syndrome – Containers and Images Local Dev Machine MacOS CI/CD Pipeline Build Run Test QA Server Linux Build Deploy Run
- 13. Tests, Works-on-my-machine syndrome – Containers and Images Local Dev Machine MacOS CI/CD Pipeline Dev Virtual Server Linux Build Deploy Run
- 14. Expensive not-used resources – Elastic cluster setup 17% CPU 10% CPU 55% CPU
- 15. Expensive not-used resources – Elastic cluster setup Cluster Scheduler CPU 55% 72% CPU 40% Requests for allocation and deploy
- 16. Secrets management Configuration Data Dev QA Prod Config Config Audit? Manage? Revoke? Where? How?
- 17. Secrets management - Vault Configuration Data Dev QA Prod Config Config Audit Manage Revoke Vault
- 18. Security Management - Vaults ConfigDB Master Vault Audit Manage Revoke
- 19. Autopilot Pattern less or none application modification for containers Application Autopilot Process Service Discovery & Health Check On Start Post Stop Health Health On Change Pre Stop
- 20. Tao of HashiCorp • Workflows, Not Technologies • Simple, Modular, Composable • Communicating Sequential Processes • Immutability • Versioning through Codification • Automation through Codification • Resilient Systems • Pragmatism
- 21. Packer for Immutable Infrastructure https://www.packer.io/ Packer is an open source tool for creating identical machine images for multiple platforms from a single source configuration.
- 22. Packer for Immutable Infrastructure https://www.packer.io/ Build Automated Machine Images
- 23. Terraform for Infrastructure-as-a-Code https://www.terraform.io/ Write, Plan, and Create Infrastructure as Code Terraform enables you to safely and predictably create, change, and improve production infrastructure. It is an open source tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.
- 24. Consul for Service Discovery, Health Checking https://www.consul.io/ Service Discovery and Configuration Made Easy
- 25. Nomad for cluster scheduling https://www.nomadproject.io/ Easily deploy applications at any scale
- 26. Vagrant against works-on-my-machine syndrome https://www.vagrantup.com/ Development Environments Made Easy
- 27. Vault for sensitive data https://www.vaultproject.io/ A Tool for Managing Secrets
- 28. HashiCorp Tools Ecosystem – Small Scale Local Dev Machine </> CI Pipeline Work Environment VM Image Repository CD Pipeline
- 29. HashiCorp Tools Ecosystem – Large Scale Local Dev Machine </> CI Pipeline Server CD Pipeline Server Nomad Cluster VM Images
- 31. Thank you!
Editor's Notes
- #2: https://www.hashicorp.com/ Target and goal of HashiCorp tools is to give sample of the best practices that I’m going to describe.
- #20: http://autopilotpattern.io/
- #21: https://www.hashicorp.com/tao-of-hashicorp/
- #22: Alternatives: Manual configuration scripts Chef, Puppet, Annsible, etc.
- #24: Alternatives: - Manual configuration scripts - Cloud-provider’s tool
- #25: http://txt.fliglio.com/2015/07/12-factor-infrastructure-with-consul-and-vault/ Alternatives: Eureka Etcd Zookeper
- #26: Alternatives: Azure Service Fabric Mezos Yarn Kubernetes Docker Swarm DCOS 1000 redises or nginx, 3 data centers across atlantic. <1s schedule 1s to first start 6s to 95% 8s to 99%
- #27: Alternatives: Docker Compose
- #28: Alternatives: Azure Key Vault Amazon KMS (Key Management Service) HSMs (Hardware Security Module): AWS CloudHSM Keywhiz by Square