NG BB 23 Measurement System Analysis - Introduction
8 likes4,654 views
This document discusses measurement system analysis (MSA) and evaluating measurement systems to ensure reliable data collection for process improvement efforts. It introduces the eight-step CPI roadmap and explains why accurate measurements are important. Sources of variation are described, including repeatability, reproducibility, and bias. Conducting a gage R&R study to quantify measurement system variation is recommended so that variation due to the measurement process can be distinguished from natural process variation. The goal is to eliminate measurement system variation and rely on data to make good decisions about process performance and improvement opportunities.
NG BB 23 Measurement System Analysis - Introduction
- 1. UNCLASSIFIED / FOUO UNCLASSIFIED / FOUO National Guard Black Belt Training Module 23 Measurement System Analysis (MSA) Introduction UNCLASSIFIED / FOUO UNCLASSIFIED / FOUO
- 2. UNCLASSIFIED / FOUO CPI Roadmap – Measure 8-STEP PROCESS 6. See 1.Validate 2. Identify 3. Set 4. Determine 5. Develop 7. Confirm 8. Standardize Counter- the Performance Improvement Root Counter- Results Successful Measures Problem Gaps Targets Cause Measures & Process Processes Through Define Measure Analyze Improve Control TOOLS •Process Mapping ACTIVITIES • Map Current Process / Go & See •Process Cycle Efficiency/TOC • Identify Key Input, Process, Output Metrics •Little’s Law • Develop Operational Definitions •Operational Definitions • Develop Data Collection Plan •Data Collection Plan • Validate Measurement System •Statistical Sampling • Collect Baseline Data •Measurement System Analysis • Identify Performance Gaps •TPM • Estimate Financial/Operational Benefits •Generic Pull • Determine Process Stability/Capability •Setup Reduction • Complete Measure Tollgate •Control Charts •Histograms •Constraint Identification •Process Capability Note: Activities and tools vary by project. Lists provided here are not necessarily all-inclusive. UNCLASSIFIED / FOUO 2
- 3. UNCLASSIFIED / FOUO Learning Objectives Understand the importance of good measurements Understand the language of measurement Understand the types of variation in measurement systems UNCLASSIFIED / FOUO 3
- 4. UNCLASSIFIED / FOUO Exercise: The Three Rs UNCLASSIFIED / FOUO 4
- 5. UNCLASSIFIED / FOUO Examples The Hale Koa Hotel manager wants to reduce customer check-in time The VA wants to reduce VA Home Loan Guarantee Program processing errors The Army Community Service organization wants to improve its customer service performance A VA Hospital is interested in finding ways to improve in-patient and out-patient care UNCLASSIFIED / FOUO 5
- 6. UNCLASSIFIED / FOUO Why Is MSA Important? Our ability to assess the performance of a process we wish to improve is only as good as our ability to measure it The measurement system is our “eyes” for our process We need to be able to see the performance of our process clearly in order to improve it Sometimes, improving the ability to measure our process results in immediate process improvements Can you trust your measurements to tell you the truth? Measurement System Analysis UNCLASSIFIED / FOUO 6
- 7. UNCLASSIFIED / FOUO Sources Of Observed Process Variation Observed Variation Observed Variation Actual Process Variation Actual Process Variation Measurement Variation Measurement Variation - Long-term Process Variation - Short-term Process Variation Variance Variance Variance Variance Due to Instrument Due to Instrument Due to Operators Due to Operators - Repeatability - Reproducibility - Calibration - Stability - Linearity The variation due to the measurement system must be identified first, then separated from actual process variation UNCLASSIFIED / FOUO 7
- 8. UNCLASSIFIED / FOUO Variation Is Additive Measured values Actual values s2 Observed = s2 Measurement + s2 Part + s2 Error s2 Measurement = s2 Observed – s2 Part – s2 Error s2 Measurement = s2 Repeatability + s2 Reproducibility + s2 Error UNCLASSIFIED / FOUO 8
- 9. UNCLASSIFIED / FOUO Why Worry About Measurement Variation? Consider the reasons why we measure: Assist in Verify process How might measurement continuous conformity to variation affect these decisions? improvement specifications activities What if the amount of Process Process measurement variation is unknown Measurement Measurement ? Measurement variation can make our process capabilities appear worse than they are. UNCLASSIFIED / FOUO 9
- 10. UNCLASSIFIED / FOUO Measurement Variation Measurement Variation is broken down into two components: (The two Rs of Gage R&R) Reproducibility (Equipment or Gage or Operator Variability) Different individuals get different measurements for the same thing Repeatability (Equipment or Gage or Operator Variability) A given individual gets different measurements for the same thing when measured multiple times The tool we use to determine the magnitude of these two sources of measurement system variation is called Gage R&R UNCLASSIFIED / FOUO 10
- 11. UNCLASSIFIED / FOUO Reproducibility (Operators’ Precision) Reproducibility is the variation in the average of the measurements made by different operators using the same measuring instrument when measuring the identical characteristic on the same part Inspector A s s s 2 m 2 g 2 o Inspector B Inspector C UNCLASSIFIED / FOUO 11
- 12. UNCLASSIFIED / FOUO Repeatability (Gage Precision) Repeatability is the variation between successive measurements of the same part, same characteristic, by the same person using the same equipment (gage). Also known as test /re-test error, used as an estimate of short-term variation. Ideal Process Target s s s 2 m 2 g 2 o UNCLASSIFIED / FOUO 12
- 13. UNCLASSIFIED / FOUO Measurement Error Gage R & R variation is the percentage that Generally recognized criteria for measurement variation (repeatability and gage acceptability is when reproducibility) represents of the variation observed Gage R & R variability to process in the process variability is : Under 10%: Acceptable gage 10% to 30%: Might be Observed Measurements acceptable Over 30%: Gage is unacceptable and should be corrected or replaced True Values Measurement Error Bias Gage R&R Stability Discrimination Linearity Repeatability Reproducibility Operator Operator * Part UNCLASSIFIED / FOUO 13
- 14. UNCLASSIFIED / FOUO Bias (Instrument Accuracy) Bias is the difference between the observed average value of measurements and the master value. The master value is determined by precise measurement typically by calibration tools linked to an accepted, traceable reference standard. Master Value (Reference Standard) Average Value UNCLASSIFIED / FOUO 14
- 15. UNCLASSIFIED / FOUO Stability Stability = If measurements do not change or drift over time, the instrument is considered to be stable Time One Time Two UNCLASSIFIED / FOUO 15
- 16. UNCLASSIFIED / FOUO Discrimination Discrimination is the capability of detecting small changes in the characteristic being measured The instrument may not be appropriate to identify process variation or quantify individual part characteristic values if the discrimination is unacceptable If an instrument does not allow differentiation between common variation in the process and special cause variation, it is unsatisfactory .28 .28 Ruler .28 .28 .279 .282 Caliper .282 .279 .2794 .2822 Micrometer .2819 .2791 UNCLASSIFIED / FOUO 16
- 17. UNCLASSIFIED / FOUO Linearity A measure of the difference in bias (or offset) over the range of the sample characteristic the instrument is expected to see determines linearity. If the bias is constant over the range of measurements, then linearity is good. Over what range of values for a given characteristic can the device be used? When the measurement equipment is used to measure a wide range of values, linearity is a concern. Measurement Variation Low High End Measurement Scale End UNCLASSIFIED / FOUO 17
- 18. UNCLASSIFIED / FOUO Name That Problem! Master Value Instrument 1 Instrument 2 Average Value Master Value (Reference Standard) Time One Time Two .28 1. Discrimination .279 2. Bias/Accuracy .2791 3. Repeatability 4. Reproducibility Inspector A 5. Instrument Bias Inspector B Inspector C 6. Stability UNCLASSIFIED / FOUO 18
- 19. UNCLASSIFIED / FOUO Measurement Systems Analysis Template The Measurement System used to collect data has been calibrated and is considered to have no potential for significant errors. The data collection tool is reliable, can be counted on, has good resolution, shows no signs of bias and is stable. Type of Measurement Description Considerations to this Project Error The ability of the measurement Work hours can be measured to <.25 Discrimination system to divide measurements into hours. Radar usage measure to +- 2 (resolution) “data categories” minute. The difference between an observed No bias - Work hours and radar start- Bias average measurement result and a stop times consistent through reference value population. No bias of work hours and radar Stability The change in bias over time usage data. Not an issue. Labor and radar usage Repeatability The extent variability is consistent is historical and felt to be accurate enough for insight and analysis. - Example - Remarks in usage data deemed not Different appraisers produce reproducible, therefore were not Reproducibility consistent results considered in determining which radars were used in each op Variation The difference between parts Required Deliverable process. N/a to this UNCLASSIFIED / FOUO 19
- 20. UNCLASSIFIED / FOUO Reported by : Gage name: Tolerance: Measurement Systems Analysis Template Date of study : Misc: Gage R&R (ANOVA) for Response Gage R&R Components of Variation Response by Part %Contribution Source VarComp (of VarComp) 100 % Contribution % Study Var 10.00 Total Gage R&R 0.0015896 3.70 Percent Repeatability 0.0005567 1.29 9.75 Reproducibility 0.0010330 2.40 50 Operator 0.0003418 0.79 9.50 Operator*Part 0.0006912 1.61 0 Part-To-Part 0.0414247 96.30 Gage R&R Repeat Reprod Part-to-Part 1 2 3 4 5 6 7 8 9 10 Total Variation 0.0430143 100.00 Part R Chart by Operator Study Var %Study Var Response by Operator 1 2 3 Source StdDev (SD) (6 * SD) (%SV) UCL=0.1073 0.10 10.00 Total Gage R&R 0.039870 0.23922 19.22 Sample Range Repeatability 0.023594 0.14156 11.38 Reproducibility 0.032140 0.19284 15.50 _ 9.75 0.05 Operator 0.018488 0.11093 8.91 R=0.0417 Operator*Part 0.026290 0.15774 12.68 9.50 Part-To-Part 0.203531 1.22118 98.13 0.00 LCL=0 1 2 3 Total Variation 0.207399 1.24439 100.00 Operator Xbar Chart by Operator Number of Distinct Categories = 7 1 2 3 Operator * Part Interaction 10.00 10.00 Operator Sample Mean UCL=9.8422 _ 1 The Measurement _ Average 2 X=9.7996 9.75 9.75 LCL=9.7569 3 System is acceptable 9.50 with the Total Gage 9.50 1 2 3 4 5 6 7 8 9 10 R&R % Contribution Part <10% - Example - Optional BB DeliverableUNCLASSIFIED / FOUO 20
- 21. UNCLASSIFIED / FOUO Takeaways It is important to be able to rely on the accuracy and precision of the measurement system to make good decisions Understand the various types of measurement system variation Eliminate as much of the variation in the measurement system as possible to focus on and improve the true cause of variation in process performance UNCLASSIFIED / FOUO 21
- 22. UNCLASSIFIED / FOUO What other comments or questions do you have? UNCLASSIFIED / FOUO 22