Proficiency Testing (PT) – a tool to improve laboratory performance - Brian Brookman
Download as PPTX, PDF4 likes3,149 views
Proficiency testing (PT) is a quality tool used by laboratories to monitor and improve their measurement capabilities, providing objective evidence of competence and areas needing improvement. Various types of PT schemes exist, focusing on multiple aspects including performance evaluation, technical competence, and method validation. An effective PT participation strategy considers factors such as frequency, risk, and the suitability of selected schemes to ensure optimal benefits and enhance laboratory performance.
Proficiency Testing (PT) – a tool to improve laboratory performance - Brian Brookman
- 1. 1 Proficiency Testing (PT) – a tool to improve laboratory performance Science for a safer world 3rd GCC Laboratory Proficiency Conference Dubai, 24-25 March 2015
- 2. Overview PT as a quality tool Purpose, types and benefits of PT PT participation strategy Selecting fit for purpose PT schemes Interpretation of PT Results
- 3. 3 Proficiency Testing - as a quality tool
- 5. Decision on measurement Report on measurement The measurement cycle Client issue Client Define issue Client Sampling Analysis Analyst Interface Decision on result Evaluation Data evaluation AnalystValidated procedures
- 6. Method Validation Calibration Training IQC Quality Assurance System ? Measurement System Input Output RMs PT
- 7. Fit-for-purpose analysis requires: Careful development and validation of methods Accurate calibration of instruments and appropriate equipment Qualified and competent staff Comparability with measurements made in other laboratories Well defined QC and QA procedures Independent evidence of performance CRMs Traceable Standards RMs & PT RMs RMs & QC PT
- 8. 8 Proficiency Testing - purpose, types and benefits
- 9. Purpose of PT The primary aim of proficiency testing is: “To provide the infrastructure for a laboratory to monitor and improve the quality of its routine analytical measurements” • PT provides an objective evidence of the competence of the participant which can be used to – Improve the performance of the participant – Give confidence in the participant’s ability to perform a specific measurement • PT provides information on other aspects of the management system – Reception/treatment of the sample, treatment of the data, result reporting etc
- 10. Types of PT • Various types of PT available – based on one or more elements of four different categories: Qualitative Quantitative Interpretive 1 Descriptive – nominal or ordinal scale Numeric – interval or ratio scale No measurement involved
- 11. Types of PT • Various types of PT available – based on one or more elements of four different categories: Qualitative Quantitative Interpretive Pre-measurement Measurement Post-measurement 1 2
- 12. Types of PT • Various types of PT available – based on one or more elements of four different categories: Qualitative Quantitative Interpretive Sequential Simultaneous Pre-measurement Measurement Post-measurement 1 32
- 13. Monitor Performance Report Sequential participation scheme structure PT Provider Test Item Test Result Test Result Test Result Monitor Performance Monitor Performance Lab 1 Lab 2 Lab 3 PT P r o v i d e r Evaluation of Results Lab 1 Lab 2 Lab 3 NMi
- 15. Types of PT • Various types of PT available – based on one or more elements of four different categories: Qualitative Quantitative Interpretive Single Continuous Sequential Simultaneous Pre-measurement Measurement Post-measurement 1 32 4
- 16. Use of PT by laboratories • Basic use of PT for a laboratory is: – to assess its performance for the conduct of specific measurements or calibrations • Results and information received from participation will provide laboratories with: – Confirmation that the laboratory’s performance is satisfactory – An indication that there are potential problems and that corrections should be made • However, the use of PT should be much wider – Many additional benefits from PT participation
- 17. Benefits of PT to laboratories Benefits of PT Third Party Demonstration Method/ Procedure Evaluation Performance Comparison Monitoring Trends Independent Check Training and Education
- 18. Maximising the benefits of PT • To obtain the optimum benefit from PT participation, laboratories should: – treat PT scheme test items as routine samples (i.e. do not give them any special treatment) – vary the analyst testing the test items – vary the instrumentation used for the test items • To learn from PT participation, laboratories should: – use the scheme results to give feedback to laboratory procedures and staff; both positive and negative – use the scheme results constructively – be honest; the scheme is there to help – remember that PT is education not regulation
- 19. 19 Proficiency Testing - Participation Strategy
- 20. Strategy of PT participation • All laboratories need to develop an adequate PT participation strategy • A laboratory should decide on an appropriate level and frequency of participation – Level: number of specific proficiency tests in which to participate – Frequency: How often the laboratory will participate in each of the specific test identified • This will be specific to the laboratory’s circumstances • Guidance provided by the European co-operation for Accreditation (EA) EA-4/18: 2010 – Referenced in ILAC-P9: 2010
- 21. Level of PT Participation • Consider areas of technical competence based on: • Measurement technique e.g. ICP-MS, GC-MS • Property to be measured e.g. PAHs, Fat, hardness • Products to be tested e.g. soil, milk, human serum • An area of technical competence may encompass several products, properties and/or measurement techniques • The laboratory must be able to demonstrate equivalence within each area of technical competence 21
- 22. Level and frequency of PT participation • Important to consider a number of different aspects • Other QA measures implemented: - Use of CRMS - Comparisons by independent techniques - Participation in validation or characterisation studies - Use of internal QC measures - Other laboratory comparisons • Level of risk: - No measurements undertaken - Turnover of technical staff - Staff experience/knowledge - Known stability/instability of measurement technique - Significance and final use of data •Different types of PTs can be used • Difficulty of PT participation e.g. technical characteristics of the measurement, low number of laboratories etc • Legislative requirements for frequency of type of participation
- 23. 23 Proficiency Testing - selecting fit for purpose schemes
- 24. Fit for purpose PT schemes • Need to decide which scheme is fit for purpose – Often a scheme will not have a perfect fit, so need to find the best fitness for purpose • A number of factors to consider – Test Items – Distribution – Participants – Results – Reports – PT Provider
- 25. Selection factors • Test Item – Matrix (real/simulated) – Parameters – Concentrations • Distribution – Dates available – Frequency – Flexibility • Participants – National/International – Language – Methods • Results – Deadlines – Reporting mechanism – Statistical approach • Reports – Speed – Information/Language – Format • PT Provider – Experience – Scope – Competence
- 26. 26 Proficiency Testing - Interpretation of results
- 27. Performance evaluation • Limited value unless performance is evaluated • Evaluate after each round and also over time • Interpretation of PT performance concerns all management levels • PT is not about “passing” or “failing” – it is about learning from the results
- 28. Review of single PT round performance • Results of each PT round should be evaluated regardless of performance – A satisfactory result may not mean good performance – Unsatisfactory performance in one round does not indicate that the laboratory is not competent • Important to consider a number of factors: – The overall results in the round – Test method performance – Test item factors – Bi-modal result distribution – PT Scheme factors
- 29. Trends in PT performance • Trends are valuable in preventive and corrective actions, and root cause analysis – Performance over time – Common groups of determinands – Performance of methods – Performance of analysts
- 30. Need for an investigation • All laboratories will occasionally have unsatisfactory or questionable PT results • A laboratory should establish its own criteria for when an investigation is necessary • The depth of investigation will depend on a number of factors: – Criticality of the method – Frequency of unsatisfactory results – Evidence of a bias
- 31. Causes of unsatisfactory performance • Can be analytical and non-analytical • Both are equally serious – the result reported is the important factor for a laboratory • Analytical Errors – Calibration – Instrument problems – Extraction/clean-up – Interferences – Method performance – Analyst • Non-Analytical Errors – Calculation – Transcription – Units – Reporting format
- 32. 32 Proficiency Testing - in conclusion
- 33. PT – powerful quality tool PT is an essential tool to all laboratories making measurements PT participation brings many benefits PT enables measurements to be monitored and improved PT is a very powerful quality assurance tool underpinning laboratory accreditation
- 34. PT - drives laboratory quality PT Reliable Measurements Reliable Decisions