Pepperdine itsm presentation 10.11
1 like504 views
The document summarizes the operational transformation of an IT organization facing financial distress over 5 phases from 2009 to 2011. Key steps included ITSM training, capacity and availability reporting, resource planning, reorganizing around ITSM principles, launching an integrated ITSM suite, and establishing a network operations center. Results included moving to the top 10% on benchmarks, increasing customer satisfaction by 16%, and reducing SLA exceptions by 77%.
Pepperdine itsm presentation 10.11
- 1. Accelerating IT Mojo in the Face of Financial Distress Joel A. Manfredo Chief Technology Officer October 12, 2011 1
- 2. Agenda Background Timeline Phases 1 - 5 Results Next Steps Conclusions and Lessons Learned
- 3. County of Orange $5.6 Billion Budget 7 Programs 32 Agencies / Departments
- 4. CEO IT Services Services are delivered through an Internal Service Fund “Running a Service Delivery Business”
- 5. Pricing a Service What does internet access cost? Hardware: •Redundant Core Switches Maintenance Contract HW and SW) •Redundant Firewalls •Content Filtering Maintenance Contract •Redundant Routers •County ARIN IP and BGP AS Services •Redundant Core Switches •IPS / IDS Hardware Depreciation: •Monitoring (Orion) Servers •Firewalls - Hardware and software Depreciation associated •Content Filtering Appliances •Routers - Hardware and software Depreciation associated •Content Filtering Report Servers •Core Switches - Hardware and software Depreciation associated •DNS Servers •Monitoring - Hardware and software Depreciation associated •Content Filtering - Hardware and software Depreciation associated Software: •Firewall Security Software licensing Other CEOIT Services: •Content Filtering Software licensing & Report Services •Contracts & Procurement Management •Orion Monitoring Software licensing •Resource Management •DNS Software (Unix and Wintel) •Financial and Cost, Budget Analysis •Billing Processing Data Center Facilities Labor: Contracts: •Network Engineering Labor •Redundant Internet Service Providers: •Security Engineering Labor 1.TW Telecom •Availability Management Analysis 2.VPLS (this is a fail-over service) •Capacity Management Analysis •IPS / IDS monitoring Service •Incident Management (Service Desk) •Redundant WAN Circuit Contracts •Change Management •Redundant Firewall Maintenance Contract (HW and SW) •Management •Redundant Routers Maintenance Contract HW and SW)
- 6. The Story… Operational Transformation in the Face of Financial Distress Environment: How: Organization: Combination: 1. A culture built on Heroics 1. Leadership 2. Siloed organization 2. Soft skills introduction 3. Assigning blame 3. Sustained “drumbeat” 4. Resistance to change 4. The persuader 5. No priorities 5. ITSM principles Financial: Method: 1. Plan 1. 27% budget reduction (2 years) 2. Implement 2. Expiring staff augmentation contract 3. Absorb 4. Measure Results: 1. Moved to top 10% against benchmark 2. Increased customer satisfaction 16% 3. Reduced SLA exceptions by 77% “Be transparent, or be gone”
- 7. Phases of the Journey Method - Plan, Implement, Absorb, Measure Phase 1: ITSM Training April 2009 DISC (Soft Skills) Training May 2009 Capacity Management May 2009 Availability Management June 2009 Resource Planning June 2009 Project Prioritization June 2009 Phase 2: Network & Platform Services Reorganization October 2009 Service Design Package November 2009 Monthly MAP meetings March 2010 Phase 3: Marketing Brochure (Mini Service Catalog) October 2010 Service Desk re-implementation December 2010 Phase 4: Network Operations Center January 2011 ITSM Full Suite (BMC – “AMP”) March 2011 Phase 5: Re-benchmark AIMM July 2011 Today Service Catalog October 2011 ITSM Reorganization Future Service Level Management Future Further Automate Capacity & Availability Future
- 8. Phase 1 Driving Continuous Improvements Phase 3 2009 2010 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec NPS Reorganized: Phase 2 -Plan & Design Customer Survey (Jul – Dec 09) -Build & Transition Training -Operate & Maintain Incident Mgt Process ITSM Integrated Suite Service Desk SDP Process Customer Survey (Jan – Jun 10) Capacity Reporting “A journey of a Chaos CTO Portal thousand miles begins “One Team” with a single step” Certifications: 95 IT Service Management Path to Maturity Marketing Brochure “Agency Success is Availability Reporting AIMM (HP) Report 46 Our Mission” -Technology & Architecture AIMM (HP) Report 34 ACS Program -Mgt Tools & Process Maturity -Culture & IT Staff Maturity Executive hired Ticket Aging Process -Demand, Supply & IT Governance Communication Team Formed Resource Planning Performance Audit Report County Connection: OCDC Project Prioritization Service request turnaround improvements (100%) Incidents – from over 5 days to 1 Data Center Tour (DISC) Not a single failed non-standard RFC Tech Fest Not a single Service Level Breach
- 9. Phase 4 Driving Continuous Improvements Phase 5 2011 Method Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec ITSM based Reorganization ITSM Integrated Suite Launch – FULL SUITE (CMDB, Change, Release…) MSB Planning Service Catalog Plan Success Customer Survey (Jul – Dec 10) Implement IT Service Management Path to Maturity Absorb AIMM (HP) Report 10 Service Level Management Network Operations Center Measure Launched Application Monitoring Virtual Server Farm Replacement
- 10. Phase 1
- 11. DISC Assessment Phase 1: Dominance, Influence, Steadiness, Compliance
- 12. Management Action Plan Results through Present 400 350 300 250 200 150 100 50 0 Nov Mar April May June Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jul Aug Sep 2010 2011 Cumulative Monthly Accomplishments
- 13. Capacity & Availability Reporting Phase 1 Capacity Report Capacity Report Availability Report Table of Contents Executive Summary............................................................................................................ 4 Dashboard Summary........................................................................................................... 4 Risks.................................................................................................................................... 4 Warnings ............................................................................................................................. 4 Scope................................................................................................................................... 5 Capacity Measurements ...................................................................................................... 5 Network .......................................................................................................................... 5 County Wide Area Network ....................................................................................... 5 Time Warner Internet.................................................................................................. 6 Security ........................................................................................................................... 7 Storage ............................................................................................................................ 8 Tier-1........................................................................................................................... 8 Tier-2........................................................................................................................... 8 E-Mail ........................................................................................................................... 10 Virtual Environment ..................................................................................................... 13 Analysis Additional Capacity ................................................................................... 14 Conclusion ................................................................................................................ 14 Recommendation ...................................................................................................... 14 Power5 Systems............................................................................................................ 15 Mainframe..................................................................................................................... 18 Power ............................................................................................................................ 19 Data Center Space......................................................................................................... 22 Floor Space ............................................................................................................... 22 Office Space.............................................................................................................. 24 Telephone Services ....................................................................................................... 25 APPENDIX – A – Report Data Source and Contributors ................................................ 26 APPENDIX – B – Virtual Environment Cluster Information .......................................... 27
- 14. Capacity & Availability Reporting Phase 1
- 15. Resource Planning Key Metrics 3 Week Summary (by organization unit): Values Row Labels ETC Hours 3 Week FTE Enterprise Infrastructure Srvcs 13.92 0.12 Networks and Platform Services 523.87 4.37 Security & Bus Continuity Oper 15.50 0.13 Grand Total 553.28 4.61
- 16. Resource Planning Key Metrics 3 Week Summary (by project): Row Labels Sum of ETC Disaster Recovery Implementation (FY11-12) 336.00 Enterprise Secure File Transfer Migration 33.20 ePages Replacement 22.00 Exchange 2010 Production Build out 112.00 HR & FS Access Definition Project Schedule 1.50 Secure FTP Infrastructure Upgrade 48.58 Grand Total 553.28 Projects with late tasks: Sum of ETC Row Labels Grand Total ePages Replacement 10.00 Grand Total 10.00
- 17. Phase 2
- 18. Driving Continuous Improvements 2009 2010 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec NPS Reorganized: Phase 2 -Plan & Design Customer Survey (Jul – Dec 09) -Build & Transition Training -Operate & Maintain Incident Mgt Process ITSM Integrated Suite Service Desk SDP Process Customer Survey (Jan – Jun 10) Capacity Reporting “A journey of a Chaos CTO Portal thousand miles begins “One Team” with a single step” Certifications: 95 IT Service Management Path to Maturity Marketing Brochure “Agency Success is Availability Reporting AIMM (HP) Report 46 Our Mission” -Technology & Architecture AIMM (HP) Report 34 ACS Program -Mgt Tools & Process Maturity -Culture & IT Staff Maturity Executive hired Ticket Aging Process -Demand, Supply & IT Governance Communication Team Formed Resource Planning Performance Audit Report County Connection: OCDC Project Prioritization Service request turnaround improvements (100%) Incidents – from over 5 days to 1 Data Center Tour (DISC) Not a single failed non-standard RFC Tech Fest Not a single Service Level Breach
- 19. ITSM Organizational Design Phase 2: Reorganization of NPS ASG IRM Process Security NPS OPS TSD PMO BA Plan & Build & Operate & Design Transition Maintain
- 20. ITSM Organizational Design Plan and Design Operate and Maintain Build and Transition Overall Architecture and Technologies / Standards* Design Validation / Review and System acceptance Acceptance Capacity Requirements Registering application in ops Procurements Availability Requirements (contact information, service levels, Configuration and Installation monitoring and backup information, Training Requirements new or changed procedures, Access Update and Audit of Asset POC requirements Lists/CMDB levels) Design Diagrams and Requirements RFC Execution and Update procedures MOUs/SLRs Coordination/Communication System lockdown Success Criteria Rollout Plan QA and Validation of Monitoring, TCO Calculations Backout Plan Backups and Access Alternatives Risk Assessment for RFC Assist in Negotiating SLRs/SLAs Backup requirements Ops Training and Handoff Freeze and Warranty Period Management Process / Procedure requirements QA and Validation of Systems Monitoring requirements Feedback to Design Team Access Requirements Approval of Rollout from Ops Standard Requests Approval of Rollout from customer Engineering and Ops review and Approval of Rollout from Design acceptance group Continuous improvement or Support and monitoring during cool scalability plans down/warranty period Approval of the Design from Peer Groups Approval of the design from Chief Architect Approval of the design from Customer
- 21. SDR SDP – Issues identified through the process
- 22. Phase 3
- 23. Driving Continuous Improvements Phase 3 2009 2010 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec NPS Reorganized: -Plan & Design Customer Survey (Jul – Dec 09) -Build & Transition Training -Operate & Maintain Incident Mgt Process ITSM Integrated Suite – Service Desk SDP Process Customer Survey (Jan – Jun 10) Capacity Reporting “A journey of a Chaos CTO Portal thousand miles begins “One Team” with a single step” Certifications: 95 IT Service Management Path to Maturity Marketing Brochure “Agency Success is Availability Reporting AIMM (HP) Report 46 Our Mission” -Technology & Architecture AIMM (HP) Report 34 ACS Program -Mgt Tools & Process Maturity -Culture & IT Staff Maturity Executive hired Ticket Aging Process -Demand, Supply & IT Governance Communication Team Formed Resource Planning Performance Audit Report County Connection: OCDC Project Prioritization Service request turnaround improvements (100%) Incidents – from over 5 days to 1 Data Center Tour (DISC) Not a single failed non-standard RFC Tech Fest Not a single Service Level Breach
- 24. It’s working... Phase 3: “Mojo” November 2009 Not a single Service Level Breech - Marshall Goldsmith Incidents – from over 5 days to 1 Not a single failed non-standard RFC Service request turnaround improvements (100%)
- 25. Marketing Brochure Phase 3: “Mini Service Catalog”
- 26. Marketing Brochure Phase 3: “Mini Service Catalog”
- 27. Phase 4
- 28. Phase 4 Driving Continuous Improvements 2011 Method Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec ITSM based Reorganization ITSM Integrated Suite Launch – FULL SUITE (CMDB, Change, Release…) MSB Planning Service Catalog Plan Success Customer Survey (Jul – Dec 10) Implement IT Service Management Path to Maturity Absorb AIMM (HP) Report 10 Service Level Management Network Operations Center Measure Launched Application Monitoring Virtual Server Farm Replacement
- 29. Network Operations Center Phase 4 Network Monitoring Server Monitoring Handoff/Review Logs Handoff/Review Logs Orion Network Dashboard Orion Server Dashboard Daily Walkthrough AM/PM VM Console IP Monitor Int/Ext/DR IP Monitor Probation/Int INMON HP SIM
- 30. Integrated ITSM Suite Phase 4: BMC Suite through the Cloud Results Custom Written
- 31. More Results
- 32. 100 200 300 400 500 600 700 800 0 100% 50% 70% 80% 90% 60% Jan-09 Feb-09 Mar-09 Results Apr-09 May-09 Jun-09 Jul-09 Aug-09 Sep-09 Oct-09 Nov-09 Dec-09 Jan-10 Feb-10 Mar-10 Apr-10 May-10 Jun-10 Incident Tickets Percent Resolved Jul-10 Aug-10 Sep-10 Oct-10 Nov-10 Evidence of Improvements Dec-10 Jan-11 Feb-11 Mar-11 Apr-11 May-11 Jun-11 Jul-11 Aug-11 Sep-11
- 33. Evidence of Improvements Results SLA Exceptions Reduced by 77% 500 400 300 200 100 0 CY2009 CY2010
- 34. Evidence of Improvements Results Average Days to Resolve Tickets Reduced 75% 14 12 10 8 6 4 2 0 CY2009 CY2010
- 35. Evidence of Improvements Results: HP’s Converged Infrastructure Maturity Model
- 36. Evidence of Improvements Results: HP’s Converged Infrastructure Maturity Model
- 37. Evidence of Improvements Results: HP’s Converged Infrastructure Maturity Model Orange County versus US Public Sector Benchmark July 2011 4.00 3.50 3.00 Orange County US Public Sector 2.50 2.00 1.50 1.00 Technology & Tools & Processes Culture & IT Staff Demand, Supply & IT Architecture Maturity Governance
- 38. Evidence of Improvements Results: HP’s Converged Infrastructure Maturity Model Orange County Percent of Characteristics "At or Better" July 2011 Head 100 count Top 10% 90 80 70 - 27% Better 60 Equal 50 40 30 20 10 0 Dec-09 Jul-10 Jul-11
- 39. Evidence of Improvements Customer Survey Scale: 1 = worst; 4 = best
- 40. Phase 5
- 41. Driving Continuous Improvements Phase 5 2011 Method Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec ITSM based Reorganization ITSM Integrated Suite Launch – FULL SUITE (CMDB, Change, Release…) MSB Planning Service Catalog Plan Success Customer Survey (Jul – Dec 10) Implement IT Service Management Path to Maturity Absorb AIMM (HP) Report 10 Service Level Management Network Operations Center Measure Launched Application Monitoring Virtual Server Farm Replacement
- 42. Service Catalog Internet Service Provider SERVICE DESCRIPTION CEO/IT is the ISP for the County and its agencies. This service includes reasonable and equitable Internet bandwidth and access HOW WE CHARGE capability to and from the County backbone to the Internet. This service is included in the County Network Services Rate. CEO/IT utilizes the total email address count from WHAT IS INCLUDED each agency’s Active Directory (AD) as a divisor to calculate This includes the installation, configuration, ongoing the cost distribution for CEO/IT Security Services, maintenance and security of the core routers that are outward Enterprise Shared Services, and County Network facing to the Internet Connectivity. Provides redundant Internet providers configured in failover configuration 64 Class C Networks that are available and managed by CEO/IT SERVICE TIPS for existing outward facing applications Limit Internet usage to business related activities Availability of an Enterprise Class, Tier 2 Data Center Network Operations Center (NOC) (staffed hours: Mon-Fri, 8am-5pm) Monitoring of Enterprise Components and Hosts on a 24x7x365 ADDITIONAL INFORMATION basis For those who wish to filter their Internet browsing, we Command Center staffed on a 24x7x365 basis offer a secure web filtering solution. See Secure Web Enterprise Software Licensing and Contracts Management Filtering Services. associated with this service Enterprise Services
- 43. Extend ITSM Based Organization Move the Plan & Design Group and the Build & Transition Group “up”
- 44. Conclusions and Lessons Learned
- 45. Conclusions & Lessons Learned 1. Financial distress clouds have their silver linings in IT environments. a. Working together in the face of a single galvanizing event b. Speak with “one voice” 2. It Takes an IT mojo-driven strategy to implement sustained continued improvements in financial distress. a. Courage and consistency; buy-in b. Incremental changes (process, organizational), planning and soft skills provided results 3. New leadership can ignite IT Mojo. a. Common goals b. Common messages 4. Establish a drumbeat to pace the changes and measure visibly and consistently. a. Messaging consistency b. Allow the organization to digest changes 5. Establish key measurements of the categories shown in the customer surveys. a. Report on them b. Make sure everyone knows what’s being measured 6. Establish a consistent program of accountability . a. Report on them b. Make sure everyone knows what’s being measured 7. Benchmark the organization before commencing the journey.
- 46. Driving Continuous Improvements 2009 2010 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec NPS Reorganized: -Plan & Design Customer Survey (Jul – Dec 09) -Build & Transition Training -Operate & Maintain Incident Mgt Process ITSM Integrated Suite Service Desk SDP Process Customer Survey (Jan – Jun 10) Capacity Reporting “A journey of a Chaos CTO Portal thousand miles begins “One Team” with a single step” Certifications: 95 IT Service Management Path to Maturity Marketing Brochure “Agency Success is Availability Reporting AIMM (HP) Report 46 Our Mission” -Technology & Architecture AIMM (HP) Report 34 ACS Program -Mgt Tools & Process Maturity -Culture & IT Staff Maturity Executive hired Ticket Aging Process -Demand, Supply & IT Governance Communication Team Formed Resource Planning Performance Audit Report County Connection: OCDC Project Prioritization Service request turnaround improvements (100%) Incidents – from over 5 days to 1 Data Center Tour (DISC) Not a single failed non-standard RFC Tech Fest Not a single Service Level Breach
- 47. Driving Continuous Improvements 2011 Method Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec ITSM based Reorganization ITSM Integrated Suite Launch – FULL SUITE (CMDB, Change, Release…) MSB Planning Service Catalog Plan Success Customer Survey (Jul – Dec 10) Implement IT Service Management Path to Maturity Absorb AIMM (HP) Report 10 Service Level Management Network Operations Center Measure Launched Application Monitoring Virtual Server Farm Replacement