Dev ops
- 1. Emerging Trends in DevOps The prescription for accelerated speed-to-market, effective patient care and greater collaboration in Healthcare IT January 2022 Prepared & Presented by: Megha Sinha
- 2. 2 1Current State of DevOps in the Healthcare Industry DevOps Maturity Assessment and Adoption Curve 2 The future of DevOps – Revolutionizing with ML & AI 4 AGENDA 3 Impact of Standardization and Metrics in DevOps Maturity
- 3. 3 Current State of DevOps in the Healthcare Sector - Where do we stand vis-à-vis other evolved sectors
- 4. 4 Study of the state of DevOps adoption revealed healthy growth during the period 2017 – 2021 BUSINESS PERFORMANCE 43% of organizations, have seen their business perform above or in line with expectations. SPEED-TO-MARKET 51% of Healthcare IT teams deploy one or more database changes compared to 47% in other sectors ADOPTION 73% of those in Healthcare have now adopted DevOps CLOUD ADOPTION Healthcare IT has witnessed a rise in cloud adoption from 42% in 2019 to 55% in 2021 20 18 15 13 11 33 30 27 18 15 35 39 43 50 48 12 13 15 18 26 0 20 40 60 80 100 120 2017 2018 2019 2020 2021 Growth of DevOps in Healthcare IT Not Adopted Plan to Adopt Adopted on Some Projects Adopted on All Projects Source:https://www.red-gate.com/solutions/database-devops/entrypage/state-database-devops-healthcare-2021,
- 5. 5 The healthy growth of DevOps led to positive implications on the state of healthcare despite a pandemic-struck world AI/ML driven critical insights Regulatory compliance and security Data-led improved patient care Detect and analyze patterns across data in real-time to identify health conditions, potential treatment methods, and their outcomes-and take life-saving decisions at speed DevOps’ core principle, “infrastructure as code,” vulnerability analysis, implementation with principle of “least privilege” allows healthcare providers to integrate security fundamentals inside the application module itself. Self-check-in systems, electronic prescription filling, and automatic appointments built with CI/CD pipelines address patients’ needs for convenient and quality healthcare. Disruptive Innovation in Healthcare A wristband that can read your mind, Cancer-diagnosing artificial intelligence, Rehab in virtual reality, Drone-delivered medical supplies have all been possible due to the integration of cutting- edge technology, speed and scale of operations. DevOps as a practice plays a critical role in this integration Improved Infrastructure Efficiency Event-driven serverless architecture allows developers to leverage only the resources they need for the app or microservice being developed. The utility consumption of resources employed when leveraging a healthcare cloud provider (AWS or Azure) on their HIPAA-compliant architecture offers ways to manage resources efficiently. Reduced Operational Expenses (OPEX) A multinational US-based conglomerate with more than 80,000 employees and $30B in annual revenue designed a DevOps-as-a-Service offering. The proposed solution cut their OPEX by 45% while ensuring high availability of the products, full regulatory compliance of operations, integration of new products and features, as well as ML and Big Data analytics services.
- 6. 6 Which leads us to the most burning question – ‘How to define a successful DevOps Strategy?’ DevOps is not just Automation DevOps is not the Cloud DevOps requires team identities and clear interaction paradigms Presence of Platform team is key to achieving DevOps at Scale Risk taking, fast flow optimization and efficient feedback loops are required CALMS Framework
- 7. 7 So, What has Worked Well and What Needs to Change in the Healthcare IT DevOps Strategy? C - Culture L - Lean IT A - Automation S - Sharing M - Measurement 58% 47% 29% 29% 74% 55% 55% 51% 22% 22% 73% 56% 0% 10% 20% 30% 40% 50% 60% 70% 80% Adoption of Cloud Infrastructure Frequency of deployments Automation of Builds, Testing, and Deployment Version Control Culture of Adoption Teamwork and Collaboration Healthcare Other Sectors • DevOps adoption rates have increased • 80 % of respondent companies stuck in the middle level of DevOps Maturity journeys has remained stagnant. • There is a wide-gap between the have’s and have-not (team) in the same organization. Source: Puppet State of DevOps Report 2021
- 8. 8 8 DevOps Maturity Assessment and Adoption Curve - Know where you stand and plan your strategy for the way forward
- 9. 9 Let us take a case-based approach to delve further into the conundrums of requisite moves to emerge successful in streamlining IT development and delivery Background Challenges What should they do? A leading US healthcare payer with a large IT structure adopted agile and DevOps practices to gain true business value through faster feature releases, better service quality and cost optimization. Teams achieved initial success with their Pilots on CI/CD pipeline automation strategy, monitoring, alerting and logging. 1. Teams that have had success on pilot projects are being inundated with requests for new projects, applications, and infrastructure. 2. The successful team are fully utilized on day-to-day projects and business requests with little capacity for strategic planning, architecture improvement, process maturity and technical debt efforts. 3. Attempts to guide the adoption of DevOps solutions across all teams have not been fruitful. 4. The pilot projects are also looking for better recipes for a Future-Ready DevOps Solution
- 10. 10 So, what should the organization do? • With only so many hours in a day, existing staff is not likely to be able to take on more work; the only option is to expand that capability, but how? • With good understanding of the DevOps Culture, how should the organization design a Roadmap for Pilot-to-Scale? • What Arsenals do the leadership teams need to achieve success?
- 11. 11 Industry leading DevOps bodies prescribe Maturity Assessment Models to map organizational Current-state and Future-State for success-at-scale 1. Leverage DASA* DevOps Maturity Assessment Framework to evaluate capabilities against current practices, DevOps goals, and business model. 2. Document current state analysis and identified gaps within IT infrastructure and operations, application development and service desk. 3. Implement solutions to plug the identified gaps. DASA Maturity Model Source: https://drive.google.com/file/d/0B2mJR_0L0FroWnc5WU9VdTljSkk/view?resourcekey=0-LE-_-HmJjMpMGOYBuh1eSg SOLUTION 0 1 2 3 4 5 Courage Teambuilding DevOps Leadership Continuous Improvement Infrastructure Engineering Security, Risk, Compliance Continuous Delivery Programming Test Specification Business Analysis Business Value Optimization Architecture and Design Maturity Scores (Maximum Score - 5) Results Journey Mapping * * * * * * * * * * * * Knowledge Area * Skill Area * Level 1 Novice Level 2 Competent Level 3 Proficient Level 4 Expert Level 5 Master DevOps maturity Score – 3.1 Current State Phase 1 Phase 2 The detailed model here explains what will the final-state look like.
- 12. 12 Further Analysis revealed that success limited to few teams was not a failure of DevOps Strategy but the emergence of new or advanced hurdles • Organizations stuck in the Mid-level of DevOps Maturity and Adoption • DevOps as an approach is followed in various teams but lacks centralized standardization • Customer experience is inconsistent • Every technical team implements security and risk regulations but there is no centralized monitoring platform available to track such implementations • Not all teams speak the same language in DevOps Cause and Effect Analysis of the DevOps stagnation in healthcare enterprises Culture Automation Lean IT Measurement Sharing • Absence of platform team model • Lack of defined Interaction paradigms • Excessive cognitive load * on delivery teams *Cognitive load examples - good practices, automation, and support that negatively impact performance metrics • DevOps approaches over focussed on the hyper care and feeding of CI/CD Pipelines • Most of the Repetitive tasks are automated, but no real optimization for the team • Absence of self-service capabilities and shortage of skills • Teams have defined their own standards for CI/CD pipeline and automation metrics • Absence of standardized usage of tools, frameworks and processes • Absence of metrics to measure effectiveness of ‘teams of teams’ working model • Infrastructure topologies are not designed for promoting fast flow of change • Absence of the culture to map customer value stream to CI/CD pipelines designed • Failure to create cultures of knowledge sharing and best practices • Unclear responsibilities and aversion to risk • DevOps is promoted passively.
- 13. 13 What do Metrics and Standardization bring to the table? - Why are some organizations able to evolve, while the vast majority are stuck in the middle?.
- 14. 14 Shifting focus on the Solution, Let us explore ‘How’ should we move ahead in the Journey? Stop talking about culture, start doing stuff 01 Scale DevOps practices with internal platform teams 02 Focus on Integrating security fully into the software delivery process 03 • Build a ‘Team of Teams’ • Devise clear team definition and interaction paradigms • Encourage employee involvement in the change management process • Enable self-service offerings for developers • Provide self-service capabilities for security and compliance validation • Improve abilities to quickly remediate critical vulnerabilities • Reduce complexity and bring standardization in the technology stack • Define organization-wide coherent measurement criteria
- 15. 15 CALMS - Culture & Change Management Launched DevOps Center of Excellence as an enterprise entity to lead and govern DevOps 1 CoE topology was defined 2 Clear Interaction paradigms were defined and enforced 3 Centralized Platform was designed and launched as a deliverable from Step 3 for DevOps, Measurement dashboard with uniform metrics, tools and architecture patterns were introduced by respective teams 4 Regular reviews, organization-wide Kx and team report-outs were formally launched, tracked and improved to ensure success of the Platform. 5 DevOps CoE Stream Aligned Team Enabling Team Complicated Sub-system Team Platform Team 2 Members each from different business stream/function Representation from IT experts for each business stream/function A cross- functional team of DevOps SMEs - Team to configure, develop, and support a centralized DevOps Platform/application Collaboration Facilitation X-as-a-Service Working together for a defined period to discover new things (APIs, practices, technologies, etc.). One team helps and mentors another team. One team provides, one team consumes something “as a Service.”
- 16. 16 CALMS - Automation and Lean IT The platform has a collection of DevOps toolchain that is managed as a product to support delivery team customers 1 The toolchain is co-created and continually reviewed with stream aligned team and complicated sub-system to ensure value and relevance 2 The product manager of the platform teams actively markets the toolchain to drive adoption across the organization 3 Suggestive Continually monitor the Platform toolchain adoption by measuring the following metrics Net Promoter Score (i.e., the willingness of delivery teams to recommend the product to others). Percentage of delivery teams using the product. Percentage decrease in feature cycle time. • 28-day rolling and/or seven-day rolling uptime • Incident frequency • Mean investigation time per incident (e.g., how long it takes to find out what went wrong) • Percentage of services or capabilities with vulnerability-patching SLAs • Cost per work unit • Developer throughput rate • Deployment rate • Rollback rate • Conformance metrics (how close a service is to using “paved roads,” i.e., standards provided by the platform team)
- 17. 17 CALMS – Measurement The DevOps CoE defined organization-wide coherent measurement criteria to measure the effectiveness of toolchain in the platform DEV/CI QA DEPLOY RELEASE OPERATE Development Lead Time Rework required by defects, build breakage, downtime Idle time Work-in-progress technical debt Cycle time Idle Time Defects discovered/escaped , impact of defects MTTD Deployment Lead Time Deployment frequency, duration Change Success rate MTTR Release frequency, % automated Time/cost per release Predictability planned vs Actual MTTR Cost/ frequency of outage On-call after business hours Performance/ utilization --Cycle Time-- --Visibility-- --Scale--
- 18. 18 CALMS – Sharing The DevOps CoE Set up Communities of Practice, centralized Knowledge exchange repositories and drove democratization initiatives Role Responsibilities Community Leader(s) Community Facilitator(s) Community Advocate(s) • Manage the ongoing health and direction of the CoP. • Track and encourage member participation. • Communicate CoP results and success stories to the DevOps CoE Lead • Plan and facilitate CoP events. • Collect member feedback and support the community leader in setting the CoP agenda and direction. • Create formal and informal connections between CoP members. • Socialize and promote the CoP • Monthly Learning Sessions • Weekly mailers to share automation patterns - deployment automation, test automation, anything another team might find useful. • Actively market the toolchain by making the tools easy to find and implement, • Mailers highlighting how the toolchain enables delivery teams to achieve their objectives more quickly and easily. • Quarterly Release of Technology Radar representing new DevOps tools added both in the trial phase, established phase and tools being phased out. Activities carried out in Phase 1
- 19. 19 A collective summary of the benefits and progress made along the DevOps Maturity Curve Benefits Realized 0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% Before After 0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% Before After % of self-service interfaces % of improvement in DevOps processes
- 20. 20 The future of DevOps – Revolutionizing with ML & AI - Why are some organizations able to evolve, while the vast majority are stuck in the middle?.
- 21. 21 AIOps technologies will enable intelligent automation, reduce toil in IT operations and increase adoption of AI-augmented DevOps practices. By 2025, leaders must shift their focus from building automated systems to instead deliver autonomous systems Descriptive and Diagnostic Predictive and Prescriptive 2020 2025 Deploy Support Monitor AI-Enabled Systems • Scale support with prescriptive analytics • Wrap incident alerts with contextual insights • Triage and deduplicate alerts • Train AI models to learn from human interventions to automate future root cause analysis • Assess risk score of builds • Instrument apps to gather telemetry to train AI models • Optimize cloud workloads for cost and performance • Automate “measure-interpret-act” functions • Go from exception-based monitoring to continuous monitoring • Correlate insights from multiple data sources
- 22. 22 How should organizations prepare themselves to embrace the future? By 2025, leaders must shift their focus from building automated systems to instead deliver autonomous systems Step 1: Identify the Current Challenges with the evolution of Digital systems and apps Step 2: Assess if these challenges are caused by overwhelming amounts of data generation and can be solved by AI Step 3: Take Steps to Identify how to make your DevOps AI-driven Adopting Advanced APIs Identifying Related Models Parallel Pipeline Pre-trained Model Public Data Due Identity Broaden Horizons What these mean?
- 23. 23 Where should we utilize AI and ML to augment DevOps A stage-wise use case and impact of leveraging AI and ML in DevOps Development Integration Testing Monitoring Feedback Deployment Operations Addressing Emergencies - ML can deal with sudden alerts by training systems continuously on identifying repeating patterns and inadequate warnings, thus filtering the process of sudden alerts. Business Assessment - DevOps pays high regard to understand of code release for achieving business goals, ML tools deal that with its pattern-based functionality by analyzing user metrics and alert the concerned business teams and coders in case of any issue. Early Detection - ML can create key patterns that can analyze and also predict user behavior, such as analyzing configuration to meet the expected performance levels, response rate and detect an issue at an early stage Triage Analytics - ML tools can help you identify issues in general processing and also manage release logs to create coordination with new deployments. Dealing with Production Cycles - ML helps to understand and analyze resource utilization, among other patterns to detect possibilities of abnormal patterns such as memory leaks Quality Check - ML performs efficient review Quality Assessment results and builds test pattern library based on discovery Application Progress - Application of ML on DevOps tools helps to addresses the irregularities around code volumes, long build time, delays in code check-ins, slow-release rate, improper resourcing and process slowdown, among others
- 24. 24 End-to-End Functional View of Intelligent DevOps Collect data Source: https://www.wipro.com/infrastructure/the-power-of-ai-in-devops/
- 25. 25 Looking forward ➢ The disruption in the consumption and subscription of services across all domains has led to intelligent ways of enabling various platforms to get connected with the end users. ➢ The momentum in DevOps will be led by how historical and transactional data is intelligently used for faster product releases ➢ The change, on one hand, and stability, on the other will help the industry to adapt to other technological disruptions in near future ➢ Managing DevOps platforms with a product mindset and platform team approach brings benefits
- 26. Thank you.
- 27. Appendix
- 28. 28 A Typical DevOps Pipeline Source: https://www.xenonstack.com/blog/devops-tools
- 29. 29 Open-Source DevOps tools Source: https://www.xenonstack.com/blog/devops-tools
- 30. 30 •Adopting Advanced APIs: Moving development teams to gain hands-on experience in working with canned APIs like Azure, AWS and GCP that allow the deployment of robust AI/ML capabilities into their software without having to create self-developed models. Further, they can focus on integrating add-ons such as voice-to-text and other advanced patterns. •Identifying Related Models: The next step after the above would be identifying similar AI/ML APIs. Doing this, development becomes easier with successful ML/AI model deployments and individual teams can work on further enhancements and apply the same to additional use cases. •Parallel Pipeline: Given the fact that AI and ML are at the experimentation stage, it will be important to also consider running parallel pipelines so that things won’t go bad in case of any failure or sudden halts. The better way to deal with would be adding ML/AI capabilities in a step-wise manner, gradually in line with the projects’ progress avoiding significant delays. •Pre-trained Model: A well-documented, pre-trained model can drastically cut down the threshold for adoption of ML and AI capabilities. A pre- trained model can be helpful in recognizing user behavior or inputs in a specific search. If it can at least match the basic aspects of the user search pattern, further add-ons to it can yield better results that can fully match with the user behavior pattern. So, having a pre-trained model is key to AI/ML adoption at an initial phase. •Public Data: Finding the initial training data is a key challenge in adopting AI/ML. None would actually feed this data. So, where will you drive information from? That’s where you require public data sets. It may not exactly meet your full requirements but can at least fill the gaps to enhance project viability. •Due Identity: One starts witnessing the true potential of ML/AI only after the software runs and shows completion at a high rate, quality and performance compared to the traditional approach. So, it will be important for any organization to identify and forward the success stories that they see out of AI/ML adoption to further teams keeping them updated. •Broaden Horizons: Developers should ideally be in continuous quest of knowing new and staying updated. This applies more to AI/ML use cases. For this, organizations should encourage teams by easing their ways to access MI/AL sandboxes and general-purpose APIs without additional formalities that come as part of the corporate procurement process. AI Readiness of DevOps Systems