Business continuity at_northrop_grumman
1 like1,193 views
The document summarizes Northrop Grumman's experience with and recovery from Hurricane Katrina in 2005. Key points: - Katrina caused severe damage to Northrop Grumman's shipbuilding facilities and IT infrastructure in Mississippi and Louisiana. - The company's disaster preparedness helped minimize data loss and enabled relocation of IT workers and restoration of key applications within weeks. - Lessons learned included keeping data centers out of harm's way, planning for public infrastructure failures, and leveraging suppliers in recovery plans.
Business continuity at_northrop_grumman
- 1. BUSINESS CONTINUITY AT NORTHROP GRUMMAN ANSHUMAN DEEPTI PARTH SOUMYA (GROUP 7)
- 2. Agenda Brief History about the company About ship systems sector (Katrina strikes and the damages) Starting of Recovery Restoring the IT infra Planning recovery and learning from disaster Lessons Learnt from Northrop Grumman
- 3. Brief About the company A $33 billion USD company with 120,000 employees worldwide Provides innovative systems, products and services in aerospace, electronics, IS, shipbuilding and technical services Customers include US government and commercial enterprises worldwide Shipbuilding is the highest revenue earner. US’s sole designer and builder of nuclear powered aircraft carrier Ship systems employees:12900 in Pascagoula & Gulfport, 7100 in New Orleans Shipbuilding facilities in Pascagoula and New port News, Virginia
- 4. Hurricane Katrina A Category 5 hurricane with 175 miles wind speed with 200 miles gust Landfall on 29 th August in Mississippi 30ft surge,238 causalities in MS New Orleans went underwater with levees broken and pump failure 1577 deaths in Louisiana and massive property damages
- 6. Damages
- 7. Damages to Northrop Grumman Severe damage to the under construction ships, the shipyard Power and network outage 1,500 PCs,200 servers, 300 printers, 600 data input devices, and hundreds of two-way radios damaged Primary data centre and network equipments were gone and link with outside was cut In NO, servers shut down due to air conditioner failure Some means of communication still alive in NO Estimated loss of Northrop Grumman was $1 billion USD
- 8. Starting of recovery Preparedness proved worth with less damage in data Restoring human resources was primary concern Information centre at Dallas was into immediate action Supplies to affected area in corporate jets, helicopters and cargo planes IT workers became productive after relocation Arrangements with Wal-Mart and Western Union
- 9. Restoring the IT infrastructure Both the data center (Pascagoula and Gulfport) damaged Unavailability Public communication & other infrastructural facilities Switching stations, cellular antennas and other essential elements damaged Satellite communication equipment used for initial damage assessment
- 10. Reconnecting the yards Restoring communication Satellite phone was initially ; line-of-sight access to satellites Satellite dishes installed to provide bandwidth for voice and data VoIP Pascagoula data center could not be rebuilt ; reopening of the yards Walkie Talkies replaced the two way radios previously used for within yard communication
- 11. Dallas : New Datacente r Procurement Challenge : Replacing 1500 desktop , 200 servers and communication infrastructure Ongoing consolidation some applications were migrated to Dallas Space available ; personnel and equipment Employee from Pascagoula relocated to Dallas within 2 days. Restoration challenges Hardware : Hitches such as relocation of server required taking disrupting business. Incompatibility between hardware and software environment. Inaccessibility of SAD
- 12. Restoring Data and Applications No data lost due to the IT disaster preparedness. Data backups @ Iron Mountain & Dallas Challenge : Prioritizing application recovery due to lack of properly documented business continuity plan Change in application criticality e.g. e-mail & communications infrastructure DDG 1000’s : Migrated to Dallas & brought up in 6 days.
- 13. Planning, Recovering and Learning from the Disaster Recovery (within 2 weeks) New Data Center Both Shipyards had connectivity All employee’s and their families found Agility, teamwork and Leadership were the key
- 14. Disaster Preparedness “ The best form of crisis management is preparedness” Most Organizations Prepare for specific domains Having greatest impact on their operations E.g. Financial institutions preparing for IT failure, hospitals for pandemics Alternative Broad Spectrum – Economic, Information, Physical, Human resources Portfolio Approach – One in each portfolio
- 15. COBIT vs. ITIL
- 16. Advantages : IT Governance COBIT (Control Objectives for Information and Related Technology) and ITIL (IT Infrastructure Library) include sections on disaster preparedness and business continuity planning. Understood to both IT and business managers. Supports Audit Process. Generic objectives and measurements Excellent guidelines for establishing IT disaster preparedness
- 17. Dis-advantages of Frameworks Limited Guidance to operational level Emphasis on IT leaves out planning parameters outside technical realm Lot of assumptions Less direction on recovery strategies
- 18. Learning from Disaster : Case for Post Disaster Analysis Prepare for unanticipated Crisis Nature of disaster shifts over time No post recovery analysis COBIT or ITIL do not offer guidance
- 19. Lessons Learned Keep Data and Data Centers Out of Harm’s Way Don’t Assume the Public Infrastructure Will Be Available As NGC adapted to non availability of infra on the fly rather than reference plans. Its leadership in crisis is an inspiration to future planners. Today, its disaster preparedness plans include the vulnerability of the public infrastructure. As a result, extra satellite phones are now in secure locations, and satellite dishes can be safely sheltered and then redeployed after disruptive events.
- 20. Lessons Learned Plan for Civil Unrest Management decided to bring in personnel from outside the area to secure the facility Disaster preparedness can include contracting for such services and planning how security personnel will be notified, transported, fed, and supported during the period of threat. Assume Some People Will Not Be Available Fortunately, NGC had employees who were called upon to start recovery operations Employees with little computing background were also deployed Post Katrina, NGC has established backup role takers for all key personnel.
- 21. Lessons Learned Leverage Your Suppliers as Critical Team Members Implementing a contingency plan is closely linked to suppliers’ capabilities. Assessing damage quickly and getting orders to suppliers can reduce bottlenecks in supply chain, particularly if other firms are also clamoring for replacement equipment. Think of it had Katrina hit Los Angeles? Expect the Unexpected Build a “crisis portfolio” by preparing for at least one disaster in each crisis category of disasters. No economic sense to plan for all type of disasters Larger the firm - > More Risk prone - > More risks come outside the scope of BCP
- 22. Lessons Learned Get Prepared Plan processes to deal with unplanned events (Use COBIT or ITIL) Identify crisis teams in multiple locations, define roles, and identify likely role holders and backups. Train and test scenarios. List supplies most likely to be needed, and identify suppliers Agree beforehand on exceptions to procurement policies during states of emergency. Include in the contact list the telephone numbers, e-mail addresses, and other contact information of people outside the potential disaster area who might assist in locating missing personnel. Inform employees how to call in, before a disaster hits IT recovery document should be simple and consistent help backup teams get in place quickly. Consider simulating a disaster, evaluating the responses, and modifying your plans accordingly
- 23. Lessons Learned Establish a Strong Leadership Position Reacting to the situation at hand than executing a pre-defined plan. Use the disaster recovery plan as a model. But be flexible in allowing leadership skills to transcend an inadequate plan. Act quickly (at “hurricane speed”) and decisively. There may be no time for a committee discussion. Decision makers may have to fall back on their instincts and leadership experience Sharing facts stimulates the flow of feedback from disparate, disaster-affected key constituents, which, in turn, increases the quality of information for decision making.
- 24. Lessons Learned Empower Decision Makers on the Team Often, pre-established decision hierarchies dissolve in crises. In addition, leadership becomes more centralized and more decentralized, simultaneously. With first-hand knowledge and the necessity to execute, people at the disaster scene often must exceed their prior authorities and operational scope to move the recovery effort forward. Exploit Fresh Start Opportunities Opportunity to upgrade ? Katrina accelerated consolidation of DCs Resistance to Upgrades evanporated New technologies were brought in…
- 25. Thank You