Data Quality and Integrity.pdf
- 1. Data Quality and Integrity Investigation in Laboratories (Analytical) Dr. Ademola O. Daramola, DHSc., MPH Assistant Country Director – International Relations Specialist (Drug) US FDA Office of International Programs; India Office
- 2. Disclaimer The views and opinions expressed in this presentation are those of the author and do not necessarily represent the official policy or position of the U.S Food and Drug Administration www.fda.gov
- 3. Data Quality & Data Integrity • Data Quality: The completeness, accuracy, timeliness and consistency of stored information • Data Integrity: The completeness, accuracy, and consistency of data • Complete, consistent, and accurate data should be attributable, legible, contemporaneously recorded, original or a true copy, and accurate (ALCOA) • Data quality drives data integrity Data Integrity and Compliance With CGMP www.fda.gov
- 4. FDA Data Integrity Requirements Include… • 211.68 (“backup data are exact and complete,” and “secure from alteration, inadvertent erasures, or loss”); • 212.110(b) (data is “stored to prevent deterioration or loss”); • 211.100 and 211.160 (activities be “documented at the time of performance” and laboratory controls be “scientifically sound”); • 211.180 (records be retained as “original records,” “true copies,” or other “accurate reproductions of the original records”); • 211.188, 211.194, and 212.60(g) (“complete information,” “complete data derived from all tests,” “complete record of all data,” and “complete records of all tests performed”). www.fda.gov
- 5. Why Data Integrity Issues are Mostly Found in the Lab • Data integrity problems do not apply only to QC, but the QC lab is where many symptoms of data integrity problems and GMP/Quality problems will be seen. • E.g., if a company is having problems in sourcing good quality starting materials and producing good quality products, this is likely to be visible in the QC laboratory's test results. www.fda.gov
- 6. The "Spectrum" of Data Integrity Issues www.fda.gov
- 7. If The Laboratory is a Gold Mine! www.fda.gov
- 8. Electronic software and OOS Investigations Are Treasure Chests! www.fda.gov
- 9. Electronic Systems Common data integrity issues encountered during review of electronic laboratory data (HPLC/GC/UV/FT-IR/Karl Fischer/Particle Counts) 1. Trial Sample Analysis 2. Deletion/Overwrite of Data 3. Testing Into Compliance 4. Back-door Manipulation 5. Administrator Foul Play 6. Physical manipulation 7. Extraneous peaks not processed 8. Manual reintegration www.fda.gov
- 10. Trial Sample Analysis Prior to testing the ‘official’ samples, trial samples are pre-tested to determine if they will meet specifications • Results are not documented and investigated according to written procedures • Results often differ from the subsequent official analysis (e.g. fail specifications) • Suggests that the analyst is choosing only those sample solutions found to be meeting specifications • They are vaguely identified e.g. test, trial, SS www.fda.gov
- 11. Trial Sample: Sample or Standard? Trial Injection Official Standard Injection Official Sample Injection – 4 hour Acid www.fda.gov
- 12. Trial Sample Analysis • 4 – hour acid dissolution specification = NLT 60% Name Vial Position Peak Area Area Similarity % Dissolution Meets Specification Disso 1 101 111,312 Acid (4 hours) 65% Yes Disso 2 102 120,561 Acid (4 hours) 70% Yes Disso 3 103 115,984 Acid (4 hours) 68% Yes Disso 4 104 60,837 Acid (4 hours) 30% No Disso 5 105 110,512 Acid (4 hours) 64% Yes Disso 6 106 65,943 Acid (4 hours) 33% No Name Vial Position Peak Area % Dissolution Meets Specification B0654 Acid 4 hours – 1 101 111,453 65% Yes B0654 Acid 4 hours – 2 102 120,123 70% Yes B0654 Acid 4 hours – 3 103 115,894 68% Yes B0654 Acid 4 hours – 4 104 115,548 68% Yes B0654 Acid 4 hours – 5 105 110,185 64% Yes B0654 Acid 4 hours – 6 106 110,631 64% Yes Trial Official www.fda.gov
- 13. Deletion of Data • Individual files or folders could be deleted within the analytical software. - Software that allows the user to view the results, but does not require saving of data. • Source data is deletable from the hard drive of the associated computer. www.fda.gov
- 14. Overwriting of Data Steve 1/29/2013 15:19 Sequence - Acquire and Analyze Run 4 - D:DATA2013Finish ProductIbuprofen29012013004.dat Steve 1/29/2013 14:43 Sequence - Acquire and Analyze Run 3 - D:DATA2013Finish ProductIbuprofen29012013003.dat Steve 1/29/2013 14:07 Sequence - Acquire and Analyze Run 2 - D:DATA2013Finish ProductIbuprofen29012013002.dat Steve 1/29/2013 13:05 Sequence - Acquire and Analyze Run 1 - D:DATA2013Finish ProductIbuprofen29012013001.dat Steve 1/29/2013 19:08 Sequence - Acquire and Analyze Run 4 - D:DATA2013Finish ProductIbuprofen29012013004.dat Steve 1/29/2013 18:33 Sequence - Acquire and Analyze Run 3 - D:DATA2013Finish ProductIbuprofen29012013003.dat Steve 1/29/2013 17:57 Sequence - Acquire and Analyze Run 2 - D:DATA2013Finish ProductIbuprofen29012013002.dat Steve 1/29/2013 17:01 Sequence - Acquire and Analyze Run 1 - D:DATA2013Finish ProductIbuprofen29012013001.dat www.fda.gov
- 15. Deletion of Data Injection is listed in the audit trail, but it is missing when we try to open it. Back-ended deletion done through Windows. www.fda.gov
- 16. Testing into Compliance • When undesirable results are encountered, samples are retested until acceptable results are achieved – No Laboratory OOS Investigation is initiated – Raw data may be destroyed – Electronic data may be deleted www.fda.gov
- 17. Testing into Compliance • The 1st injection was 10/30/15 at 11:23 • Specification for Benzophenone is <200ppm • Injection fails at 238 ppm www.fda.gov
- 18. Testing into Compliance • The second injection was 11/3/15 at 21:32 • This injection passes at 117ppm • This is the only reported data www.fda.gov
- 19. Electronic Data Reprocessing • Reprocessing should not be regularly needed if analytical methods are capable and stable. • If chromatography is reprocessed, written procedures must be established and followed and each result (original + reprocessed) retained for review . • It is NOT acceptable to only save the final results from reprocessed laboratory chromatography www.fda.gov
- 20. Is it Permissible to Exclude CGMP Data From Decision Making? • To exclude data from the release criteria decision-making process, there must be a valid, documented, scientific justification for its exclusion. • Record retention and review requirements are the same for paper-based and electronic data • All records required under CGMP are subject to FDA inspection. www.fda.gov
- 21. Back-door Manipulation • Changing the sample weight – Altering the sample weight for an Assay analysis to increase or decrease potency as desired. www.fda.gov
- 22. Back-door Manipulation Integrating into compliance • Increasing or decreasing peak cut-off points to achieve passing results. • Using integration parameters to suppress valid peaks • etc www.fda.gov
- 23. Administrator Foul Play • Using Administrator privileges to turn off/on audit trails - hide trial analyses or data manipulation • Using Administrator Privileges to set the controlling PC clock back in time - repeat failing runs www.fda.gov
- 24. Physical Manipulation Equipment physically manipulated: • Forcing the equipment to fail to provide a reason for invalidation of already generated data. • Preventing the equipment from transferring or saving the data…cable disconnect. www.fda.gov
- 25. Honorable Mention • Sharing user names and passwords • Backdating of analyses, such as stability tests, in order to meet the required commitments • Reuse of old data, passing it as new data, to avoid performing supplementary analyses (saving time and money…?) • Failure to record activities at the time they are performed • Creation of false records during an inspection • Leaving out systems that are labeled “R&D systems" from the inspection. www.fda.gov
- 26. OOS Investigations What is a meaningful OOS investigation? • Thorough • Timely • Unbiased • Well-documented • Scientifically sound www.fda.gov
- 27. OOS Investigations and Data Integrity 1. Invalidating out-of-specification test results Disregarding OOS results without scientific justification 2. Testing Into Compliance Repeated testing until a passing result is obtained 3. Inserting failed system suitability standards in a sequence 4. Finding a flaw in the analysis after the fact 5. Failure to extend investigation to other batches 6. Averaging of failed replicates 7. Sample weight manipulation www.fda.gov
- 28. Testing Into Compliance • Testing Into Compliance • Repeated testing until a passing result is obtained • Disregarding OOS results without scientific justification • Retesting • Maximum number of retests NOT specified in SOP • May vary based on variability of method • IS adjusted during OOS Investigation • Testing NEVER seems to end and batch NOT evaluated www.fda.gov
- 29. Averaging of OOS Results • Averaging • Inappropriate Use • Blend Uniformity • Content Uniformity • Averaging OOS results with in-spec results to obtain a passing result • Averaging in-spec testing results obtained during an OOS investigation with the original OOS results to obtain passing result www.fda.gov
- 30. Averaging of OOS Results • What if there is no valid reason to invalidate the original OOS results? • What should the firm do? • Keep all the results • Do NOT average ANY of the results • Evaluate all the individual results against the written specifications www.fda.gov
- 31. Averaging of OOS Results • Specification equals 90.0 – 110.0 % • Example 1: • 90.0, 89.9, 90.1, 90.3, 90.0, 90.2, 90.0, 90.0 • Example 2: • 89.9, 99.2, 98.7, 99.3, 99.1, 99.4, 98.9, 99.0 www.fda.gov
- 32. Failure to Extend Investigation • So the OOS result is confirmed and the root cause identified. The firm closes the investigation and rejects the product. Time to move on. Right? • Not so fast! • A failure investigation that extends to other batches or products that may have been associated with the specific failure must be completed. • (21 CFR § 211.192) • Why is this a common violation? www.fda.gov
- 33. More Reasons For 21 CFR 211.192 Violations • Difficult and time-consuming to investigate related batches • Shutdown Production • What about the batches on the market? • It becomes a wrestling match between • Quality Control Unit & Pharma Mgmt www.fda.gov
- 34. And Even More Reasons For 21 CFR 211.192 Violations • OOS results may indicate a flaw in product/process: • A lack of robustness in product formulation • Inadequate raw material characterization or control • If so, what now??? • Redesign of the product/process • FDA Approval Process again • How many firms want to do that? www.fda.gov
- 35. Concluding The OOS Investigation: Caution • Individual results of a test should be expected to produce a result that meets specification • If assay is low, but within specification, it may suggest that batch was not formulated properly. • 21 CFR § 211.101 (a) – Firm Investigate? • 21 CFR § 211.194 - Records must be kept of complete data derived from all tests performed to ensure compliance with established specifications and standards. www.fda.gov
- 36. Data Quality and Integrity Investigations - Triggers www.fda.gov • List of OOS Investigations • Firm’s SOP for OOS vs. FDA Guidance • Analyst error ??? How Often? • Instrument Error • Unknown? What now? • Averaging OOS – NEVER! • Testing Into Compliance • Everyone Else Does It! • Too good to be true • Internal audit • Audit trails • Reprocessing - How often?? • Missing HPLC/GC vials ?? • Expired samples – deliberate delays??? • Failed system suit…how often? • Failed bracketing standard spec??
- 37. Addressing Lab Data Integrity • Determine the scope of the problem • Demonstrate effective remediation of problems Third party auditor • Implement a corrective action plan (globally), • Remove at all levels individuals responsible for problems from CGMP positions. www.fda.gov
- 38. Scope of DQ & I Investigations • Suspected or known falsification or alteration of records required under parts 211 must be fully investigated under the CGMP quality system to determine the effect of the event on patient safety, product quality, and data reliability; to determine the root cause; and to ensure the necessary corrective actions are taken. www.fda.gov
- 39. Review of Audit Trails • Audit trail review is critical both to the detection and correction of electronic data integrity problems. • Audit trails must be reviewed with each record and before final approval of the record. • Review should include, but are not limited to: the change history of finished product test results, changes to sample run sequences, changes to sample identification, and changes to critical process parameters. • Must be reviewed and approved by the quality unit www.fda.gov
- 40. Prevent & Deter, The Best Policy • Prevent: Personnel MUST BE trained in detecting data integrity issues as part of a routine CGMP training program • Deter: A comprehensive ethics program that describes standards for employees and procedures for educating employees about data integrity implications and for enforcing a zero tolerance policy within quality control laboratories • Independent System Administrator, not QC Manager! www.fda.gov
- 41. www.fda.gov
- 42. References • Guidance for Industry Draft Guidance: Data integrity and compliance with cGMP (April 2016) • Guidance for Industry: Investigating Out-of- Specification (OOS) Test Results for Pharmaceutical Production (October 2006) • Boyd, J. (2017): Data integrity in quality control: Inspector perspective • Panagiotis, S. (2014): OOS Investigations • Croft, S. (2013) Quickest ways to find data integrity issues during inspections • www.fda.gov