Design margin analysis & prediction 2005
2 likes1,257 views
The document discusses design margin analysis and prediction using Six Sigma methods. It provides examples of how Raytheon has used design margin analysis on production test data and probabilistic modeling to identify circuits and components with low design margins. Monte Carlo simulation was used to predict line of sight variation for a military video sight, determining that mechanical alignment was not needed, saving $300k. The analysis tools discussed allow incorporating unit-to-unit variability into design margin quantification to better predict performance.
Design margin analysis & prediction 2005
- 1. Raytheon Design for Six Sigma Design Margin Analysis & Prediction Richard W. Johnson DFSS Deployment Lead Raytheon Company January 11, 2005
- 2. Raytheon Design for Six Sigma Definition of “Margin” USL Target Nominal Value x How do we apply “Margin” in our Designs? Design Margin = ? Design Margin = ? USLLSL xy Target Nominal Value • An amount allowed beyond what is needed (e.g. a small margin of safety)* * The American Heritage Dictionary, Fourth Edition x USL Design Margin = x LSL Design Margin = y What if we determine that the predicted unit-to-unit variability looks like this....how does this change our predicted design margin? What if we determine that the predicted unit-to-unit variability looks like this....how does this change our predicted design margin? Design Margin = ? USLLSL Mean Value σ So.....How can we quantify design margin such that the effects of expected unit-to-unit variation are comprehended?
- 3. Raytheon Design for Six Sigma What is Design Margin in the DFSS Paradigm? Design Margin can be quantified using the Mid-Term Capability Index (Cpk) MIN (USL - µ, µ - LSL) 3σ Design Margin (DM) = • Assumes that µ is between USL and LSL = Cpk USLLSL Mean (µ) Desired DM: DM = 2.0σ 6σ DM = 6σ/3σ = 2.0 Min Acceptable DM: DM = 1.5 DM = 4.5σ/3σ = 1.5 6σ4.5σ σ
- 4. Raytheon Design for Six Sigma A Common Dilemma USL Estimated Response You work hard to get your estimated response below the USL....... • Pushing the response estimate lower will • Take more design time (more design iterations) • Require higher-cost components & materials • Drive tighter manufacturing tolerances Negative impact on Program Schedule Negative impact on Product Cost Your design meets theYour design meets the requirement..... Right?requirement..... Right? 0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1.0 Should you change yourShould you change your design to push the responsedesign to push the response lower? ..... Do you havelower? ..... Do you have enough design margin?enough design margin?
- 5. Raytheon Design for Six Sigma A Common Dilemma USL Estimated Response Mean If the actual response distribution looks like this....this placement of the mean should result in acceptable yields at test Defective Units If the actual response distribution looks like this....this placement of the mean will drive a high rate of defective units at test 0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1.0 Placement of the estimated response mean with respect toPlacement of the estimated response mean with respect to the specified limit is athe specified limit is a ““guessing gameguessing game”” if the expectedif the expected response variation is not knownresponse variation is not known
- 6. Raytheon Design for Six Sigma Design Margin Analysis ExampleDesign Margin Analysis Example Military Radio Production • Experiencing poor first pass yields at several ambient gain tests • Design margin analysis recommended • Mean value of many test measurements are close to limit • Poor design margin suspected to be the problem
- 7. Raytheon Design for Six Sigma Program ExampleProgram Example –– Military Radio ProductionMilitary Radio Production Design Margin AnalysisDesign Margin Analysis Approach • Select tests to be analyzed • Download historical test data to statistical data analysis tool • Agilent ADS (Advanced Design System) • Analyze the Data • Calculate Design Margin • Verify correlation of DM analysis with probabilistic predictions
- 8. Raytheon Design for Six Sigma Std Dev = .055 dB Gain Distribution 0 50 100 150 200 250 300 -11 -10.6 -10.2 -9.8 -9.4 -9 -8.6 -8.2 -7.8 -7.4 -7 -6.6 -6.2 -5.8 -5.4 -5 -4.6 -4.2 Gain (dB) NumberofUnits Ant SW: Gain Margin 03-07320 Test #60 : Gain J10 to P5 Lower Spec Limit Original Mean Temp Modified Limit Lower Spec Limit Original MeanTemp Modified Limit Program ExampleProgram Example –– Military Radio ProductionMilitary Radio Production Design Margin AnalysisDesign Margin Analysis Goal Mean 6 Sigma Probabilistic modeling of performance output verified that inadequate design margin could have been predicted, identified input variables driving most of response variation, and ensured success of new design. 2 Sigma µ - LSL 3σ DM = = - 0.83 – (- 0.94) 3 (.055) Design Margin CalculationDesign Margin Calculation DM = 0 .67
- 9. Raytheon Design for Six Sigma Upper Spec Limit Original Mean Temp Modified Limit Coupler T/R: Insertion Loss Margin 61-41680 Test #18 : RX Insertion Loss Program ExampleProgram Example –– Military Radio ProductionMilitary Radio Production Design Margin AnalysisDesign Margin Analysis Insertion Loss Distribution 0 20 40 60 80 100 120 140 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1 Insertion Loss (dB) NumberofUnits Original Mean Upper Spec Limit Temp Mod Limit Std Dev = .1022 dB .9 Sigma USL - µ 3σ DM = = 0.70 - .608 3 (.1022) Design Margin CalculationDesign Margin Calculation DM = .30 Goal Mean 6 Sigma Again, probabilistic modeling showed that inadequate design margin could have been predicted and identified the components driving most of the response variation.
- 10. Raytheon Design for Six Sigma Program ExampleProgram Example –– Military Radio ProductionMilitary Radio Production Design Margin AnalysisDesign Margin Analysis Results • 19 of 37 analog circuits were identified with low design margin (Cpk<.72) • Probabilistic modeling using design specifications verified that the low design margins could have been predicted • Design margin analysis on production test data coupled with probabilistic modeling & simulation of the circuit designs provided clear visibility to which design variables could be adjusted to achieve desired margins
- 11. Raytheon Design for Six Sigma Design Margin Prediction ExampleDesign Margin Prediction Example Lightweight Video Sight (LVS) • LVS mounted on grenade machine gun • Developed for military combat applications • Implemented design improvements to include optical path
- 12. Raytheon Design for Six Sigma Design Margin Prediction ExampleDesign Margin Prediction Example Objectives • Determine the Line of Sight (LOS) variation for the LVS sensors • Image-Intensified Night Vision Camera (I2TV) • Day Television Camera (DTV) • Laser Rangefinder (LRF)
- 13. Raytheon Design for Six Sigma Design Margin Prediction ExampleDesign Margin Prediction Example Approach • Identify error sources • Create model (transfer function) • Establish relationships • Develop equations • Run simulation • Monte Carlo • Analyze results
- 14. Raytheon Design for Six Sigma Identify Error SourcesIdentify Error Sources Design Margin Prediction ExampleDesign Margin Prediction Example • Error sources identified which affect line of sight • Optics and housings • Detectors • Fabrication and Assembly • Tilts and De-centering • Az and EL separated Examples of LVS Boresight Error Sources Error # Assy Part Feature #1 Feature #2 Type Direction 1 I/F SEL/Sight Mount SEL mtg holes assy az 2 Sight mount Plate SEL mtg hole Plate edges fab az 3 Sight mount Plate SEL mtg surf Housing mtg surf. - flat fab el 4 Sight mount Plate SEL mtg surf Housing mtg surf. - ang. fab az 5 Sight mount Plate Housing mtg surf. - ang. Plate edge fab az 6 I/F Sight mount/Sight Sight mount (?) Sight (?) assy 7 Housing Housing Sight mount- angular Plate mtg flange fab az 8 Housing Housing Sight mount- flat Plate mtg flange fab el 9 Housing Housing Sight mount- flat Plate mtg pins fab 10 Housing Housing Sight mount- angular Plate mtg pins fab
- 15. Raytheon Design for Six Sigma LENS #1 TILT CELL LENS #1 TILT LENS #2 TILT CELL LENS #2 TILT LENS #3 TILT CELL LENS #3 TILT LENS #1 DE-CENTER CELL LENS #1 DE-CENTER LENS #2 DE-CENTER CELL LENS #2 DE-CENTER LENS #3 DE-CENTER CELL LENS #3 DE-CENTER FOCUS CELL TILT LENS HSG. TILT LENS HSG. DE-CENTER LENS #5 DE-CENTER CELL LENS #5 DE-CENTER LENS #6 DE-CENTER CELL LENS #6 DE-CENTER LENS #7 DE-CENTER CELL LENS #7 DE-CENTER ASSY LENS #5 DE-CENTER ASSY LENS #6 DE-CENTER ASSY LENS #7 DE-CENTER LENS #4 DE-CENTER CELL LENS #4 DE-CENTER LENS #4 TILT CELL LENS #4 TILT LENS #5 TILT CELL LENS #5 TILT LENS #6 TILT CELL LENS #6 TILT LENS #7 TILT CELL LENS #7 TILT FLAT MIRROR TILT ANNULAR MIRROR TILT LOS ERROR OPTICAL BENCH DTV ASSY I2 ASSY DE-CENTER ANNULAR MIRROR LOCATION FLAT MIRROR LOCATION FAB TOLERANCE SUMMATION CALCULATION ASSY TOLERANCE LENS #1 TILT LENS #2 TILT LENS #3 TILT LENS #1 DE-CENTER LENS #2 DE-CENTER LENS #3 DE-CENTER LOS ERROR LENS ASSY LENSES LOS ERROR LENS ASSY ASSY LENS #1 DE-CENTER ASSY LENS #2 DE-CENTER ASSY LENS #3 DE-CENTER LENS #5 DE-CENTER LENS #6 DE-CENTER LENS #7 DE-CENTER LENS #4 DE-CENTER ASSY LENS #4 DE-CENTER LENS #4 TILT LENS #5 TILT LENS #6 TILT LENS #7 TILT LOS ERROR OPTICAL BENCH I2 ASSY DTV DE-CENTER ASSY DTV DE-CENTER ASSY I2 ASSY DE-CENTER ASSY LENS #3 TILT DTV DE-CENTER I2 ASSY DE-CENTER Create Response ModelCreate Response Model Design Margin Prediction ExampleDesign Margin Prediction Example
- 16. Raytheon Design for Six Sigma Design Margin Prediction ExampleDesign Margin Prediction Example Create Response ModelCreate Response Model Input Variables • Six Sigma tolerances used for all fabrication errors • Obtained tolerances from Raytheon Internal Process Capability Database • 6σ tolerances derived from actual measured part data • Methods and Tooling group consulted for distribution fit Six Sigma Tolerances for Some of the Machined Features Involved +6σ-6σ Feature #n +6σ-6σ Feature #5 +6σ-6σ Feature #4 +6σ-6σ Feature #3 +6σ-6σ Feature #2 +6σ-6σ Feature #1 •FAB TOLERANCES, LENS ASSY • TILT, LENS SEATS - ⊕ .00076 (N/C Lathe) • DE-CENTER, LENS BORES - ⊕ .00076 (N/C Lathe) • FAB TOLERANCES, OPTICAL BENCH ASSY • TILT, LENS SEATS - // .003, ⊥ .0036 (N/C Mill) • DE-CENTER, LENS BORES - ⊕ .00174 (N/C Mill) • TILT, MIRRORS - ∩ .006 (N/C Mill) • LOCATION, MIRRORS - ∩ .006 (N/C Mill)
- 17. Raytheon Design for Six Sigma Create the Response ModelCreate the Response Model Design Margin Prediction ExampleDesign Margin Prediction Example Using Decisioneering’s Crystal Ball ® – Monte Carlo Simulation Error Sources (Input Variables) Random Values Transfer Function Generate Distribution for Response LOS calculation USLLSL USLLSL USLLSL USLLSL USLLSL USLLSL USLLSL USLLSL USLLSL User defined: • Means • Tolerances • Distributions • Sigma levels Re-iterate “X” number of trials
- 18. Raytheon Design for Six Sigma Run the Simulation and Analyze the ResultsRun the Simulation and Analyze the Results Frequency Chart .000 .006 .012 .019 .025 0 61.75 123.5 185.2 247 -0.44 -0.22 -0.00 0.21 0.43 10,000 Trials 163 Outliers Forecast: LOS Error, DTV to LRF, Elevation Frequency Chart .000 .006 .012 .018 .025 0 61.5 123 184.5 246 -0.46 -0.23 -0.01 0.22 0.44 10,000 Trials 130 Outliers Forecast: LOS Error, DTV to LRF, Azimuth Forecast: LOS Error, DTV to LRF, Elevation Statistic Value Trials 10,000 Mean 0.00 Median -0.00 Mode --- Standard Deviation 0.17 (6σ = 1.02 mrad) Variance 0.03 Skewness -0.03 Kurtosis 3.61 Mean Std. Error 0.00 Forecast: LOS Error, DTV to LRF, Azimuth Statistic Value Trials 10,000 Mean 0.00 Median 0.00 Mode --- Standard Deviation 0.17 (6σ = 1.02 mrad) Variance 0.03 Skewness -0.02 Kurtosis 3.51 Mean Std. Error 0.00 Spec = 2.08 mrad Max Using 6 Sigma Manufacturing Tolerances! Design Margin Prediction ExampleDesign Margin Prediction Example USL - µ 3σ DM = = 2.08 - 0.00 3 (0.17) 4.08 Design Margin CalculationDesign Margin Calculation DM =
- 19. Raytheon Design for Six Sigma Design Margin Prediction ExampleDesign Margin Prediction Example Results • System level mechanical boresight alignment not required (~$300K Avoidance). • Optical alignment of the I2 and DTV camera components will be required. • Software alignment of the aiming reticle to the LRF transmitter beam at the system level will be required.
- 20. Raytheon Design for Six Sigma Analysis ToolsAnalysis Tools Data Analysis • Statistical Analysis and Acceptance Test Software (STAATS) • Advanced Design System – Agilent Technologies • Minitab • Microsoft Excel • Raytheon Analysis of Variability Engine (RAVE) • Crystal Ball - Decisioneering • Advanced Design System (ADS) – Agilent Technologies • Statistical Design Institute Tools Probabilistic Performance Modeling
- 21. Raytheon Design for Six Sigma Conclusions • Design Margin is a familiar concept in engineering and manufacturing environments, but has been under-utilized because classical methods of quantifying design margin in product performance do not comprehend unit-to-unit variability • Classical design methods recognized the existence of unit- to-unit variability, but in the absence of available/efficient methods and tools to model variability, adopted the use of safety factors and worst-case design (infinite margin) • More design iterations • Higher-cost materials/components • Tighter tolerances • Using Cpk as a design margin model provides way to: • Communicate how much variability is occurring or tolerable • Communicate how much risk is present or tolerable