3. Monitoring and updating multiple of medians (MoM) values within the laboratory
- 1. Monitoring and updating MoM values Updating MoM values • When to make changes • Factor updates • Discussion CUSUM charts • Practical session Confidence intervals • Practical session
- 2. CUSUM Charts
- 3. Practical session: Constructing CUSUM charts • Quick demonstration in XL • Some illustrative examples with no random variation to illustrate the methodology • Discussion of features of the CUSUM charts
- 4. Practical session: Constructing CUSUM charts For each series of MoM values 1. Compute the log10MoM values 2. Produce the CUSUM 3. Plot the CUSUM and interpret Observation Series 1 Series 2 Series 3 Series 4 Series 5 1 1.1 0.9 1.0 0.9 1.00 2 1.1 0.9 1.0 0.92 1.00 3 1.1 0.9 1.0 0.94 1.00 4 1.1 0.9 1.0 0.96 1.00 5 1.1 0.9 1.0 0.98 1.00 6 1.1 0.9 1.2 1.00 0.99 7 1.1 0.9 1.2 1.02 0.98 8 1.1 0.9 1.2 1.04 0.97 9 1.1 0.9 1.2 1.06 0.96 10 1.1 0.9 1.2 1.08 0.95 11 1.1 0.9 1.2 1.10 0.94 12 1.1 0.9 1.2 1.12 0.93 13 1.1 0.9 1.2 1.14 0.92 14 1.1 0.9 1.2 1.16 0.91 15 1.1 0.9 1.2 1.18 0.9 16 1.1 0.9 1.0 1.2 0.89 17 1.1 0.9 1.0 1.22 0.88 18 1.1 0.9 1.0 1.24 0.87 19 1.1 0.9 1.0 1.26 0.86 20 1.1 0.9 1.0 1.28 0.85
- 5. 0 5000 10000 15000 20000 25000 -600 -400 -200 0 Sample log10MoM JanFeb M ar Apr M ay Jun Jul Aug 1 MoM 0.9 MoM 1.1 MoM 0.95 MoM 1.05 MoM Log MoM CUSUM: Perfect The horizontal trajectory reflects the situation where there is no bias in MoM values.
- 6. 0 1000 2000 3000 4000 5000 6000 7000 -150 -100 -50 0 50 100 Sample log10MoM Dec JanFeb M ar Apr M ay Jun Jul Aug 1 MoM 0.9 MoM 1.1 MoM 0.95 MoM 1.05 MoM Log MoM CUSUM with bias Positive bias apparent from May.
- 7. Interpretation of CUSUM charts • CUSUM charts are a powerful way of monitoring MoM values over time • Interpretation of CUSUM charts can be far from straightforward • There is always a danger of over interpretation • Confidence intervals can be used to account for the uncertainty in estimation of median MoM values
- 8. Confidence Intervals for Median MoM values
- 9. Confidence Intervals Given a series of n MoM values, the process of obtaining a confidence interval is as follows 1. First ensure that no updates to medians have taken place in the series 2. Second check that the CUSUM scatters about a straight line 3. Compute the log10 MoM values 4. Compute the mean (i.e. average) and standard deviation 5. Compute the standard error (SE = standard deviation/sqrt(n)) 6. The lower and upper limits are given by mean ± 1.96×SE
- 10. Practical session: Constructing Confidence Intervals • Quick demonstration in XL • Hands on Confidence intervals for MoM values - practical.xls
- 12. Dealing with changes • Detecting changes • When to make changes • What changes to make 12
- 13. • The aim is to keep median MoM levels close to the target by making changes to remove biases that are going to continue • We need to avoid ‘chasing noise’ which will make things worse 13 Deciding when to make changes
- 14. Given an apparent shift in a temporal plot, try to identify the cause. Decide whether that cause is likely to persist. If so, consider making a change. Information on a ‘special cause’ such as a lot change would provide further evidence that the change is likely to persist. Deciding when to make changes log(MoM) 14 Lot change
- 15. Factor Updates • If the median MoM is running at a value different from 1 and this situation is going to persist into the future, then make a factor update. • The methodology is dependent on the software.
- 16. ViewPoint Changes to ViewPoint involve completing a form specifying the median MoM.
- 17. ViewPoint Changes to ViewPoint involve completing a form specifying the median MoM.
- 18. LifeCycle • If a log10 polynomial regression is used for gestational age, the intercept parameter A in the regression is updated by adding log10(Median MoM). • New A = Current A + log10(Median MoM) • For example if the median MoM is running consistently at 0.95 log10(0.95) = -0.022276 is added to A. • New A = Current A - 0.022276
- 19. Testing • It is important to test all updates. • If the current median MoM is 0.95 and a factor update is applied, then the effect should be to divide by 0.95 so that, for example, a MoM that was 0.95 before the change should become 0.95/0.95 = 1. • A simple way of testing is to look take samples before the change and verify that after the change the factor has been applied correctly.
- 20. Updating the full equations • Anything other than a factor updates requires specialist input • This requires • new coefficients • test data • Diagnostics based on the data provided 20
- 21. 21
- 22. Discussion • What processes do you have in your laboratory for applying factor updates? • How do you document these updates? • Does the software supplier provide any advice on updating and testing the software configuration? • Are there any suggestions for improvement?
- 24. Rationale • Need for sufficient precision to estimate performance (standardised SPR) • To enable monitoring of laboratory medians and early detection of change points • For proficiency/expertise
- 25. log(MoM) -50 -40 -30 -20 -10 0 10 20 log(MoM) -50 -40 -30 -20 -10 0 10 20 log(MoM) -50 -40 -30 -20 -10 0 10 20 log(MoM) -50 -40 -30 -20 -10 0 10 20 Sample log(MoM) -50 -40 -30 -20 -10 0 10 20 0 200 400 600 800 1000 Sample log(MoM) -250 -200 -150 -100 -50 0 50 0 2500 5000 5 labs 1,000 samples 1 lab5,000 samples