Hale wayne
- 1. PROJECT MANAGEMENT CHALLENGE 2007 SHARED VOYAGES: LESSONS LEARNED FROM THE EXTERNAL TANK How I Found Out That We Are Not Nearly As Smart As We Thought We Were N. Wayne Hale 1 Manager, Space Shuttle Program
- 2. “We don’t know a millionth of one percent about anything.” Thomas A. Edison “Progress comes from the intelligent use of experience.” Elbert Hubbard “Experience teaches the teachable.” Aldous Huxley ‘It ain’t what ya don’t know that will get ya, it’s what you think ya know that ain’t so.” Yogi Berra “The ET is just a big, dumb drop tank.” anonymous Shuttle Commander in the early years 2
- 3. Quotes from Chapter 6: Engineering Culture …the messy interior of engineering practice, which after the accident investigation looks like “an accident waiting to happen” is nothing more than “normal technology. Normal technology…is unruly. …experts are operating with far greater levels of ambiguity, needing to make uncertain judgments in less than clearly structured situations. Practices do not follow rules, rather, rules follow evolving practices. In the implementation and operation of complex technological systems, new rules and relationships are continually being invented and negotiated. Information generated by anomaly, by discrepancy between expected and actual outcomes becomes the means by which fallible rule sets are corrected and moved toward solution sets. This general tendency is profoundly realized in engineering work. Learning proceeds through iteration. 3
- 4. Quotes from Chapter 6: Engineering Culture Absolute certainty can never be attained for many reasons, one of them being that even without limits on time and other resources, engineers can never be sure they have foreseen all possible contingencies, asked and answered every question, played out every scenario. Many technologies…cannot be tested in laboratory conditions. Tests are conducted on models, which can only approximate the complex systemic forces of nature and technical environment. This situation creates risk: the world outside the laboratory becomes the setting for experiments. Judgments are always made under conditions of imperfect knowledge. The essence of engineering as a craft is to convert uncertainty to certainty, figuring probabilities and predictions for technologies that seldom stay the same…in the workplace, engineers formulate the rules as they go along, attempting to capture the unruly technology with numbers, experienced based theories, and practical rules. Even in closure there is ambiguity. 4
- 5. 5
- 6. History of the External Tank STS-1: December 29, 1980 STS-2: Columbia is mated to its ET/SRB stack Repair operations to holes caused by woodpeckers on ET for STS-70 Only part of the Space Shuttle Vehicle not returned for reuse and evaluation! 6
- 7. 7
- 8. ET Production History 121 Units Delivered to Date Delivered Flown Three Versions: Standard Weight Tank 6 6 Al 2219 (Al=Aluminum) (1981 – 83) Dry Wt. 77,099 lbs. (actual ET1) Lightweight Tank 87 86 Al 2219 (1983 – 98, 2002, 2003) Dry Wt. 65,767 lbs (actual ET71) Super Lightweight Tank 28 21 Al 2195 (Al-Li = Aluminum Lithium) (1998 – Present) Dry Wt. 58,319 lbs. (actual ET96) Substantially Completed Tanks 4 8
- 9. External Tank Foam pre-STS-107 Prior to STS-107, foam loss was regarded as a vehicle processing issue, not a safety of flight issue. 9
- 10. ET CRYOINSULATION: General Properties Foam / Property (HCFC) NCFI 24-124 (HCFC) NCFI 24-57 (HCFC) PDL 1034 (HCFC) BX265 (HCFC) SS 1171 (CFC) CPR 488 (CFC) NCFI 22-65 (CFC) PDL 4034 (CFC) BX 250 (% of total foam) (77%) (7%) (1%) (14%) Application LH2,L02,I/T sidewall LH2 aft dome Closeouts and repairs LH2 forward dome, L02 aft dome, closeouts Process Spray Spray Pour/Mold Spray Description Isocyanurate Isocyanurate Urethane Urethane Requirements Spec Typ Flt Spec Typ Flt Spec Typ Flt Spec Typ Flt Req Prop Pred Req Prop Pred Req Prop Pred Rez Prop Pred Density PCF 2.0-2.5 2.28 Lighter6 2.6-3.1 2.97 Heavier6 2.3-3.1** 3.3** same6 1.8-2.6 2.4 2.1-2.6 2.4 2.6-3.1 2.90 2.3-3.1 2.6 1.8-2.6 2.4 same6 1.8-2.6 2.4 Tensile RT (psi) 30min 44 19 40min 66 19 60 113 19 35min 80 35min 54 40min 71 60 104 35min 53 19 35min 75 Tensile -423° F (psi) N/A1 34 19 N/A 49 19 N/A 50 19 N/A 74 19 41 47 49 62 53 Tensile +300° F (psi) N/A 32 19 N/A 36 19 N/A 712 19 N/A 53 19 37 45 53 355 47 Compression (psi) 25min 33 20 35min 49 20 30 61 20 24min 43 20 24min 40 35min 51 30 42 24min 30 24min 42 Recession Rate @ 7 BTU/ft N/A 0.0094 lower6 N/A 0.00997 same6 N/A 0.0303 higher6 0.031 sq sec (in/sec) 0.0168 0.00997 0.0235 N/A 0.0173 lower6 0.024 Thermal Cond @ R/T BTU/hr 0.025 0.017 same6 0.0225 0.0180 higher6 0.016 0.015 higher6 0.015 0.015 ft °F) 0.017 0.0158 0.0156 0.016 0.012 0.015 0.013 higher6 0.011 Cryostrain (ksi) 61@-423 65@-423 pass 58@-423 65@-423 pass N/A 60@-320 pass N/A 65@-423 pass 65@-423 65@-423 60@-320 65@-423 N/A- Not Applicable 1 2+200ºF Values 3@ 4 BTU/ft sq sec 4Max density 3.0 in dome area allowed 10 @ + 200ºF 5 6Means new vs. old 7Radiant heating 82.4 – 2.8 PCF for thick/thin **Spec Req-cup pour; Typ Prop-dissection Reference: ET Project – Design Values for Non-Metallic Materials, LM 809-9600 Rev C, May 2006.
- 11. ET CRYOINSULATION: Key Material Engineering Aspects Levels of Structure Polymeric Structure Cellular Structure Knitline Geometry Subtrate Geometry 11
- 12. Columbia (STS-107) • In the early morning on Saturday, February 1, 2003, the Space Shuttle Columbia broke up during entry. All seven crew members were killed. • 81 seconds after launch, foam insulation on the External Tank broke off and struck the Shuttle’s wing at Mach 2.46, creating a hole roughly the size of a pizza box. • When Columbia reentered the atmosphere to land, highly heated plasma entered the breached wing, and burned or melted away the wing’s internal structure. The structural failure of the wing led to Crew of the Space Shuttle Columbia the loss of vehicle control and the vehicle broke apart as it descended toward Earth. CAIB: “Foam Did It!” but WHY did the foam do it? 12
- 13. ET Foam Certification Testing 13
- 14. Loss of ET Thermal Protection System Acceptance Rationale 14 from the STS-121 FRR
- 15. How Air-Divots Are Formed 15 from the Physics Models for Foam Debris presentation on May 4, 2005 by Dr. Peter B. Pollock
- 16. Codes for Predicting Foam Debris 16 from the Physics Models for Foam Debris presentation on May 4, 2005 by Dr. Peter B. Pollock
- 17. Thermal Protection Verification (TPS) / Validation Issues 17
- 18. 18
- 19. ET Major Design Changes 19
- 20. Return to Flight (RTF) RTF concentrated on improved foam application processes to minimize defects (voids) • Much tighter controls on workmanship • More oversight and review • Continuing practice • Routine destructive evaluation of foam applied to near flight fixtures 20
- 21. Return to Flight (RTF) • RTF included redesign of Bipods to eliminate the “ramp” and greatly reduce foam in the area • Serious review of PAL ramp, which is the largest manually applied foam structure on the ET, showed no significant defects, no improvement in safety by removing and using new processes to reapply STS-107 Bipod Ramp 21 STS-114
- 22. Return to Flight (RTF) Return to Flight was 2.5 years in the making It was noted in passing that the defect/void divot debris generation theory could not explain the STS-107 bipod loss. 22
- 23. STS-114 RTF Results • Major foam loss from Bipod wedge • Major foam loss from Protuberance Air Load (PAL) ramp • Significant foam losses around the Ice/Frost Ramps • Near misses of Orbiter from all three areas could have been catastrophic 23
- 24. What Went Wrong? How Could That Happen? 24
- 25. • Bipod “wedge” was lost because we introduced a new failure mechanism: Cryopumping • Wires were not sealed which allowed air to liquefy and become the motive force to blow off significant foam Classic case of a new design having an undesirable side effect! 25
- 26. What About The Other Losses? • ST-120 underwent 2 tanking cycles at KSC and then was shipped back to the factory • Evaluation of the tank delayed until Michoud Assembly Facility (MAF) operations resumed following Hurricane Katrina • Immediate observation: cracks in and under the PAL ramp and in and under the Ice Frost Ramp (IFR) Not Seen in Previous Testing! 26
- 27. 1 2 B is the second crack reported from visual inspection; A is crack found during backscatter inspection; found in Zone 6 inspection One of the first 2 cracks reported; crack that appears to be closed at the surface; found in zone 5 inspection 3 5 This is the 4th crack detected during a backscatter inspection; this is the first crack detected in BX250; found in zone 8 4 Second crack detected during inspection on 11/3/05; found in zone 14 Crack detected during inspection on 11/3/05; appears to run between plugs and under or into the larger plug; found in zone 13 27
- 28. Relearning the Lesson Turns out that the full size (test) article shows there is significant differences in thermal expansion for foam on foam application, which leads to cracks, primarily on the hydrogen tank The Space Shuttle Program immediately directed the removal of the PAL ramp This caused a huge engineering recertification effort of the protuberances and their associated load capability! 28
- 29. But the time of loss during flight was not understood, so Probability Risk Assessment (PRA) analysis was based on the assumption that foam losses, as seen in ET separation photography (end state), were evenly dispersed during the ascent or assumed to all happen at the most critical times This lead to extremely high probabilities of catastrophic failure prior to the second return to flight, STS-121 29
- 30. During STS-121, a very good image with new camera views showed a divot coming off the Ice Frost Ramp at a significant time Detailed review of other video sources built up a record of when the Coefficient of Thermal Expansion (CTE) failure causes pieces to come off Most losses will occur after the aero region where there is not motive force to cause damage, ergo, no hazard 30
- 31. ET Debris Table Sensitivity (Foam on Tile Void DeltaP Risk Assessment) 31
- 32. Debris Overview 32
- 33. Review of pre STS-107 imagery shows several flights where bipod losses occurred also had losses in foam adjacent to the bipod loss Several more flights showed just losses in the adjacent acreage, which tends to confirm the Bipod foam loss of STS-107 was associated with CTE mismatch, not void defect divot 33
- 34. MORAL(S) of the Story: 1. You are never as smart as you think you are 2. If the hypothesis does not explain reality, the hypothesis is not right 3. Flight test is the only REAL test 4. Continually question your fundamental assumptions 5. Don’t expect certainty 34
- 35. The Universal Abstract “We have not succeeded in answering all of our problems. Indeed, we have not completely answered any of them. The answers we have found have only served to raise a whole new set of questions. In some ways, we feel as confused as ever, but we think we are confused on a much higher level about more important things.” 35