Introduction to DevOps
- 1. Introduction to DevOps KEY CONCEPTS, TECHNOLOGY AND PATHS FORWARD YAZID HAMDI H T T P S : / / W W W . L I N K E D I N . C O M / I N / Y A Z I D H A M D I / H T T P S : / / Y A Z I D . X Y Z This Photo by Unknown Author is licensed under CC BY-SA
- 2. Outline 2 Key Concepts The “Why?” The “What?” The “How?” The Cultural Phenomenon Who To Follow? Technology Virtualization/Networking Build/Test Automation (CI) Infrastructure Provisioning Deployment Automation (CD?) Monitoring/Logging Paths Forward For Developers/Data Scientists For Architects For Managers
- 3. About Me •Your friendly neighborhood software architect •AWS Certified Solutions Architect – Associate (Professional in preparation) •Experience developing for and deploying on AWS •Experience with large scale data intensive systems (Nomad Centre of Excellence, automotive industry) •Was teaching assistant in Carnegie Mellon University’s Master of Software Engineering Programs for the “DevOps: Modern Deployment” course •Named reviewer of the book: “Deployment and Operations for Software Engineers” 3
- 4. Key Concepts
- 5. The “Why?” (1): How it always was 5 This Photo by Unknown Author is licensed under CC BY This Photo by Unknown Author is licensed under CC BY-SA Code/binaries, configs, “deploy this” Feature development Code QA This Photo by Unknown Author is licensed under CC BY-SA Deployment Operations/Security/Compliance Support? “Code it this way” “Deploy it this way”
- 6. The “Why?” (2): It was actually bad 6 Long development/deployment cycles Long time to market Variable deployment quality Slow incident response Unstable non-secure deployments/infrastructure Frustration on both sides of the wall
- 7. The “Why?” (3): Responsibilities… 7 This Photo by Unknown Author is licensed under CC BY-SA-NC
- 8. The “Why?” (4): How it could be 8 This Photo by Unknown Author is licensed under CC BY This Photo by Unknown Author is licensed under CC BY-SA Architecture Quality Assurance Deployment Monitoring Incident Management This Photo by Unknown Author is licensed under CC BY-SA
- 9. The “What?” 9 As defined by Len Bass et al. (“DevOps: A Software Architect’s Perspective”), the five DevOps practices: Promote Ops in requirements Give Dev more responsibility Enforce a deployment process Use Continuous Deployment Develop infrastructure code in addition to application code
- 10. The “How?” (1): Lifecycle Processes + Tools •Close collaboration •Blurring the boundaries between Dev and Ops roles •Responsibility sharing and early coordination •Automation at every turn (to enforce processes) 10 This Photo by Unknown Author is licensed under CC BY-SA-NC
- 11. The “How?” (2): It’s All About The Ops 11 DevOps DataOps SecOps/DevSecOps NoOps?
- 12. The “How?” (3): What Ops? 12 This Photo by Unknown Author is licensed under CC BY-NC
- 13. The “How?” (4): Typical Implementation Plans 13 Source: How to Implement DevOps to Automate the Software Delivery Process?
- 14. The “How?” (4): Typical Implementation Plans 14 Source: The Five Steps to DevOps Adoption
- 15. The “How?” (4): Typical Implementation Plans 15 Source: DevOps: Consulting with Foresight
- 16. The “How?” (4): Typical Implementation Plans 16 This Photo by Unknown Author is licensed under CC BY-SA Isn’t it all familiar?
- 17. The Cultural Phenomenon (1): The Hype Curve 17 This Photo by Unknown Author is licensed under CC BY-NC-ND
- 18. The Cultural Phenomenon (2): The Job Description 18 •DevOps Engineer •Software Architect/DevOps Specialist •Infrastructure Engineer •Platform Engineer •Site Reliability Engineer (SRE) •… (Senior) DevOps Engineer, Infrastructure
- 19. The Cultural Phenomenon (2): The Job Description 19 This Photo by Unknown Author is licensed under CC BY-SA
- 20. The Cultural Phenomenon (3): The Inevitable Death 20
- 21. The Cultural Phenomenon (3): The Inevitable Death? 21
- 22. Who To Follow? 22 “Authorities”: “All Day DevOps: World’s Largest DevOps Conference” “DevOps Institute” Blogs (there is no shortage) Publications: SEI, ACM, IEEE Books (see Paths Forward section of this presentation) Certifications are a good indication: “10 Best DevOps Certifications in 2020 Ranked” Cloud Providers (AWS, GCP, Azure, Redhat, Oracle…) Tool/Platform Providers (Docker, Kubernetes, Jenkins, etc.)
- 23. Technology
- 24. Virtualization/Networking 24 Virtualization: Break down the monolith Scale out instead of scaling up “Throwaway” units VMs/Containers/Lambda Functions Services instead of servers Networking Traffic routing Traffic load balancing Dependencies/Service hierarchies
- 25. Build/Test Automation (CI) (1) 25 This Photo by Unknown Author is licensed under CC BY-SA
- 26. Build/Test Automation (CI) (2) 26 Google Cloud Build GoCD Buildbot
- 28. Infrastructure Provisioning (2) • Build compute resources: • Vagrant/Packer: • build/configure virtual machines • Docker/Kubernetes: • build containers/groups of containers • Chef/Ansible/Puppet: • build/orchestrate different compute units • Build entire stacks as code (Infrastructure as Code – IaC): • Terraform/Pulumi/Cloudify: • describe infrastructure, platform-agnostic • AWS Cloudformation/Azure Resource Manager/GCP Deployment Manager: • platform-specific 28 This Photo by Unknown Author is licensed under CC BY-SA
- 29. Deployment Automation (CD?) = CI + INFRASTRUCTURE PROVISIONING 29 This Photo by Unknown Author is licensed under CC BY-SA
- 30. Monitoring/Logging 30 Monitor: •Failures •Automatic operations •Network-level traffic •OS-level metrics •Application-level metrics •Cloud metrics •Usage statistics and patterns (for business analysis feedback) Source: “Coding Defines Monitoring”
- 31. Paths Forward
- 32. Disclaimer The following are subjective recommendations from my own experience 32 This Photo by Unknown Author is licensed under CC BY-NC
- 33. For Developers/Data Scientists •For your use case: • Find opportunities for automation • Be careful about ROI! • Learn about domain-specific techniques/tools •Cloud knowledge is key: • Modern DevOps is heavily reliant on Cloud technology • In parallel to your domain knowledge, acquire Cloud architecture/deployment experience and knowledge •Book recommendation: “Deployment and Operations for Software Engineers” by Len Bass and John Klein 33
- 34. For Architects • System quality implications of adopting DevOps practice(s): • Performance? • Reliability? • Recoverability? • Testability? • Modifiability?... • Special care for designing and maintaining the code around the code: • Infrastructure templates • Configuration management • Scripts • Secrets management… • Book recommendations: • “DevOps: A Software Architect’s Perspective” by Len Bass, Ingo Weber and Liming Zhu • “Production-Ready Microservices” by Susan J.Fowler 34
- 35. For Managers •Consider business tradeoffs, especially Agility/speed vs extra costs • Long-term benefits vs immediate cost savings • Does the scale/lifecycle require this? •Listen to experts before making an investment •Keep an overview of the couplings between requirements engineering/planning/tracking/risk management and the adoption of DevOps •Book recommendations: • “DevOps Handbook” by Gene Kim, Jez Humble, Patrick Debois and John Willis 35