Harman deepak v - agile on steriod - dev ops led transformation
1 like153 views
The document discusses an enterprise agile transformation led by dev-ops practices to improve collaboration between software development and IT operations. It highlights the benefits of implementing a culture focused on automated software delivery and the shift from monolithic architectures to microservices. Key strategies include adopting a devops cadence, built-in quality mechanisms, and effective tooling to enhance deployment speed and system reliability.
![HARMAN International. Confidential. Copyright 2015. 8
[AGILE + DEV-OPS] IS THE NEW ENTERPRISE AGILITY
Agile is about building quality products with shorter and smoother development cycles
Dev Ops is about shorter and smoother delivery cycles to take agile to production
Source - DevOps for Dummies – Sanjeev Sharma](https://image.slidesharecdn.com/harman-deepakv-agileonsteriod-devopsledtransformation-170407085851/85/Harman-deepak-v-agile-on-steriod-dev-ops-led-transformation-8-320.jpg)
![HARMAN International. Confidential. Copyright 2015.
TOOLING – BEST PRACTICES
• Choose tools where they can deliver end-to-end solution (Wherever possible) Ex:
TFS
[Depends on technology, no. of teams, product architecture, complexity of solution,
no. of components]
• Choose always minimum tool set or otherwise, providing stable cadence is challenge
(DevOps cadence is end-to-end delivery, so, breakage between any tool set would
break the entire chain and hence, failure)
• Bring redundancy in tool chain (especially on Web - Azure, AWS, Verizon). Even a
minor interruption in network can cause build/deployment failure. So, manual
intervention is required. For ex. If build failed to get a NuGet package, re-trigger.
Unable to push package to Octopus (Deployment tool) due to network error, re-
push..
• Avoid linked chains as much as possible (if chain breaks, whole CD fails)- instead,
identify the parallelism and trigger (which saves time and rework)
• Enable mail, SMS and various other forms of feedback](https://image.slidesharecdn.com/harman-deepakv-agileonsteriod-devopsledtransformation-170407085851/85/Harman-deepak-v-agile-on-steriod-dev-ops-led-transformation-19-320.jpg)
Harman deepak v - agile on steriod - dev ops led transformation
- 1. HARMAN International. Confidential. Copyright 2015. 1 “AGILE ON STEROIDS” DEV-OPS LED ENTERPRISE AGILE TRANSFORMATION A CASE STUDY DEEPAK VIJAYARAGAVAN HEAD OF QUALITY, PROCESS ENGINEERING AND AGILE PRACTICE FOR XEBIA AGILE MEET UP @ PUNE 17 MARCH 2017
- 2. HARMAN International. Confidential. Copyright 2015. 2 WHO AM I • Developer, Tester • Project Manager or Delivery Head • Agile Practitioner and Evangelist • Change agent • Voice of the Customer • Sales person • Leader
- 3. HARMAN International. Confidential. Copyright 2015. 3 SILOS STARTED BUILDING ACROSS… HOW IT STARTED… http://www.mindtheproduct.com/2016/02/what-the-hell-are-ci-cd-and-devops-a-cheatsheet-for-the-rest-of-
- 4. HARMAN International. Confidential. Copyright 2015. 4 THEN AGILE HAPPENNED… - Focusing on faster development cycles packed with features… - Documentation to working software each iteration - Waterfall releases to Incremental high value feature releases - Dev + Test – one agile team with cross functional skills BUT SOME SILOS CONTINUED… DEV+ TEST OPS
- 5. HARMAN International. Confidential. Copyright 2015. TILL ALL HELL BREAK LOSE WHEN…
- 6. HARMAN International. Confidential. Copyright 2015. 6 • Too costly defects found in the production • Delay in Speed time to market (production) • Dev and ops with different mindsets • Traceability and feedback becomes difficult • Blame game starts due to Human errors leading to havoc in the system - automate • Fail to adapt to changing requirements Ultimately.. Dissatisfied customers LEADING TO…
- 7. HARMAN International. Confidential. Copyright 2015. 7 DevOps (a clipped compound of "software DEVelopment" and "information technology OPerationS") is a term used to refer to a set of practices that emphasize the collaboration and communication of both software developers and information technology (IT) professionals while automating the process of software delivery and infrastructure changes. It aims at establishing a culture and environment where building, testing, and releasing software can happen rapidly, frequently, and more reliably. “DevOps” - Wikipedia DEV-OPS ;)
- 8. HARMAN International. Confidential. Copyright 2015. 8 [AGILE + DEV-OPS] IS THE NEW ENTERPRISE AGILITY Agile is about building quality products with shorter and smoother development cycles Dev Ops is about shorter and smoother delivery cycles to take agile to production Source - DevOps for Dummies – Sanjeev Sharma
- 9. HARMAN International. Confidential. Copyright 2015. …WITH COMMON INTERESTS • Shift left development and operation concerns • Speed time to market ---> speed time to production • continuously dev, test -- Continuously dev, test, build, and release • Independent testable and deployable unit • Fast Feedback (including visual) • Consumer-centric and Value driven • Optimizing on speed and quality
- 10. HARMAN International. Confidential. Copyright 2015. 10 CASE IN POINT Client: a leading Industrial automation player • Re-platform & Integrate over 20 different products developed across 6 different geographies from a Monolith Architecture to a Cloud First service driven platform – Cross sell / Up sell features across products • Reduce release cycle times from 6 –8 months to 2 – 3 weeks – Stay current in the market • Scale Operations – Increase global foot print , expand customer base into Commercial , Residential , Enterprise BUSINESS PROBLEM
- 11. HARMAN International. Confidential. Copyright 2015. 11 CHALLENGES • Monolithic code base posing challenges to build, test and deploy for every changes • 25 + Services with manual Deployment, Manual Testing, 500 odd config changes ---errors leading to unstable build • Turnaround time to handover a refreshed build for QA = >1 week • Environment creation was manual and tedious ---lack of automation • No Built in quality like Unit tests, Static code checkers etc • Dev doing the ops with very limited knowledge of tools, release management, dependencies, config. Changes • Tools – Just Bamboo and Manual Deployment • Limited Safety Nets no API and UI functional tests
- 12. HARMAN International. Confidential. Copyright 2015. DEVOPS STRATEGY Tool and Automation Culture and Mindset Built in QualityMicro services
- 13. HARMAN International. Confidential. Copyright 2015. CULTURE AND MINDSET - Train the team and make them aware about the impact of their mistakes - Build the dev-ops team with App experts and tool experts - Integrate code daily (Continuous Integration) - Run Continuous Delivery (CD- complimenting CI) - Promote builds on various environments - Set up DevOps Cadence (Sample) - Avoid name and shame but educate the team on implications Do you know? In certain organizations, there is a penalty or made them wear a cap for breaking the builds etc. should be avoided) - Honor and respect the Master branch and always maintain the sanctity of it
- 14. HARMAN International. Confidential. Copyright 2015. Sample DevOps Cadence Feature branch: 1. Create a Task for every Story for every Service. If a Story cuts through 4 Services, create 4 branches. A branch per Service 2. Create Feature branch from Master Branch – Tech lead 3. Feature branch must be created at Team level where all the Developers code within Team will be merged to this Branch 4. Code committer ensures successful CI Builds (Auto build, SonrQube, UT Suite) mandatorily. If Code committer feels, API Suite and Functional Smoke Suite needs to be run, it has to be run manually. 5. MSI changes must be done at Feature branch level for each Service that is changed and tested successfully 6. If new APIs are added, ensure the API team adds their tests to Feature branch. 7. Ensure Code committers & Service Owners reviews the changes been made at Team level before you merge to Master 8. Inherit the latest code base from Master before you merge to Master (Along with, you will inherit latest UT, SonarQube, API) 9. Generate Pull request and make Code Committer as approver 10. After approval from Code committer, merge the code to Master 11. Code committer is responsible to ensure the sanctity of the build and he will be responsible for any build failures/ build breakers Local branch: 1. Create the Branch from Feature branch and for every Developer, there will be associated Local Branch 2. Ensure your CI (Auto Build, UT ) build runs on your local branch (SonrLint will help you guide writing a better code @ Desktop) 3. Ensure Peer Review is done before you merge to Feature branch Master branch 1. Ensure UT, API suite, Functional Smoke suite are running successfully at Master branch – Service Owners 2. Ensure MSI changes are happening @ Master branch for every ticket raised in JIRA – DevOps 3. Ensure all services are up and running & allocate to the respective Code committers the responsibility of bringing up the service incase of failure- Service Owners
- 15. HARMAN International. Confidential. Copyright 2015. MICRO-SERVICES - Monolithic to Micro-services - Windows Services - Easily deployable for changes
- 16. HARMAN International. Confidential. Copyright 2015. BUILT-IN QUALITY • Design & develop quality gates (Ex. UT >=90%, No Blocker, Critical static code errors, Code Complexity <=15, Code duplication <=5%) and strictly adhere to it or otherwise, fail the build • Separate CI with CD – Run overnight API, Integration Test suite or Functional Test suite and ensure fail fast approach. • UI, API, UT should be integral part of Sprint so that automation is part and parcel of your Delivery else Tech Debt raises • Infuse code checks (like SonarQube, PCLint, KlockWorks, Coverity, Fortify etc) • For Legacy, you have no choice except accept the truth on Technical Debt. • Various ways to address the issues • Develop a Quality Improvement Plan based on customer feedback, internal defects, change burst prediction models (based on no.of times, you change a project, you can predict which one is more buggy- contextual) • Form a separate team to work on • Address the technical debt incrementally (first remove blockers, followed by Critical etc.)
- 17. HARMAN International. Confidential. Copyright 2015. TOOLING SNAPSHOT Environment MSI Octopus Source Control Compile Nuget package UT Sonar Report Test Results Packaging Releasing Source Control CI Build MSI Build Nightly Build (CD)
- 18. HARMAN International. Confidential. Copyright 2015. BUILD PLAN - 12:00am – CI Build CI Build Success/Failure Report - 3.00- 5:00am – MSI generation - 6:00am – Push to Octopus Push Success/Failure Report - 6:00 -7:00am – Octopus deployment - Deployment Success/Failure Report - 7:00am – Environment Ready - 7:15am – Sanity Check Sanity Check Success/Failure Report
- 19. HARMAN International. Confidential. Copyright 2015. TOOLING – BEST PRACTICES • Choose tools where they can deliver end-to-end solution (Wherever possible) Ex: TFS [Depends on technology, no. of teams, product architecture, complexity of solution, no. of components] • Choose always minimum tool set or otherwise, providing stable cadence is challenge (DevOps cadence is end-to-end delivery, so, breakage between any tool set would break the entire chain and hence, failure) • Bring redundancy in tool chain (especially on Web - Azure, AWS, Verizon). Even a minor interruption in network can cause build/deployment failure. So, manual intervention is required. For ex. If build failed to get a NuGet package, re-trigger. Unable to push package to Octopus (Deployment tool) due to network error, re- push.. • Avoid linked chains as much as possible (if chain breaks, whole CD fails)- instead, identify the parallelism and trigger (which saves time and rework) • Enable mail, SMS and various other forms of feedback
- 20. HARMAN International. Confidential. Copyright 2015. AUTO ENVIRONMENT - Any new environment creation should be a mouse click ( Well, its not that easy in real world- purely contextual based on tool set you choose) - CI/CD should be enabled for any branch (Not only limited to Master branch) - Divest the CD to geographically distributed environments (ensure a common time zone for CD to trigger) - Ensure seamless integration across tool set
- 21. HARMAN International. Confidential. Copyright 2015. BENEFITS - Sanity of master build = Dev CI after every commit - Scalable deployment model = Micro-services - DevOps Cadence – Disciplined DevOps Delivery (DDD) - Automated and more frequent deployments = weeks to hours - Daily CI and CD Build – Better stability - Faster recovery from failures – Daily visibility - Built in quality and safety nets – Shift left quality - More (environments) with less with latest code base = Automation - No weekend or late night shifts during release times = Work life balance
- 22. HARMAN International. Confidential. Copyright 2015. • Not a Separate team • Not Just tooling • Not one size fit all • Not automation or CD • Certification or set of prescriptive processes • Not simply combining Development & Operations teams Concluding.. Culture and Mindset Relationship and Collaboration between all teams who build, run and release features Tooling and Automation Structure • Dev team gets involved in features + performance + reliability of the system • Ops don’t come in the end but throughout Agility = Being Agile + DevOps
- 23. HARMAN International. Confidential. Copyright 2015. 23 Questions ? DEEPAK.VIJAYARAGAVAN@HARMAN.COM