Data integrity & microbiological data: Reviewing the essentials

Data integrity is applied to maintaining and assuring the accuracy and consistency of data over its entire life-cycle. In this sense, data integrity is the opposite to data loss and data corruption. It applies to computerized systems, paper documentation, and other means of capturing data (down to the level of reading a microbiological environmental monitoring sampled). Thus, data integrity is relatively broad in scope and it can have widely different meanings in different situations (1).

While a key concern is with computerized systems, much of the scope of many microbiology laboratories is with the reading of samples (such as broth culture and agar plates). The information recorded may then be placed onto a report form or into a computerized system.

Microbiology laboratories handle a lot of data, although unlike analytical laboratories the data is more often an even mix of quantitative and qualitative data. Examples of data include the result of a settle plate exposed during the environmental monitoring of a cleanroom; the endotoxin content in a sample of water; the weight result from the weighing of a raw material for use in the pharmacopoeial Microbial Limits Test and so on.

Despite the current high profile, the subject of data integrity in the pharmaceutical microbiology laboratory has not been afforded very much attention in terms of regulatory guidance or in terms of active discussion through articles penned by those working in the pharmaceuticals or healthcare sectors. One aspect of this is with culture media and this article looks at some of the data integrity concerns within the microbiology laboratory that affect culture media and considers some of the steps that can be taken to address identified weaknesses. 

Definitions of data

Data refers to is a set of values of qualitative or quantitative variables; that is pieces of data are individual pieces of information. Data is something which is measured, collected and reported, and analyzed. With data integrity there are generally agreed definitions of data:

• Original record: Data as the file or format in which it was originally generated, preserving the integrity (accuracy, completeness, content and meaning) of the record, for example,  original paper record of manual observation, or electronic raw data file from a computerized system.

• True Copy: An exact verified copy of an original record. Data may be static (for example, a ‘fixed’ record such as paper or portable document format) or dynamic (such as an electronic record which the user can interact with). Importantly, hard copy (flat data – printed, portable document format (pdf), photocopy) is no longer considered to be acceptable by regulatory authorities as this data is not complete and not original.

With microbiology, data is the information that can be obtained from raw data. This could be, for example, a reported result from a bioburden test; a particle count from an aseptic process; or the result of an autoclave cycle, in relation to the temperature hold time. Data varies in its complexity. Enumerating the result from a settle plate, such as counting 6 colonies, is relatively simple as is taking a pH reading. In contrast, the output from a phenotypic microbial identification system where the predicted result is based on the reaction outcomes of sixty odd tests, is far more complex.

With raw data, this is the original record and documentation retained in the format in which it was originally generated (this can be paper or electronic); or it may refer to a document retained as a true copy. Taking the particle count from an aseptic process, the raw data is the data captured by the particle counter (which could be copied onto a memory stick or transferred to a computer server). With a microbial count on a culture media plate, because the plate cannot be retained indefinitely, the documentation used to record the count (or computerized system into which the number of colony forming units is entered) becomes the repository of the raw data.

Metadata is ‘data about data’ and it gives data context or meaning. This may be a date or a unit of measurement. For example, the number ‘6’ in itself is meaningless, whereas 6 CFU (to represent 6 colony forming units) gives the data context. As a second example, writing ‘yellow’ in relation to a description of a colony on an agar plate means little because different organisms can produce differently colored pigments on different types of agar. Thus a yellow colony on tryptone soya agar could signify any number of commonly found bacteria, whereas a yellow colony on Mannitol salt agar (a medium that contains a high concentration of between 7.5%-10% of salt) is significant. The bacterium Staphylococcus aureus produces yellow colonies with yellow zones, whereas other coagulase-negative staphylococci produce small pink or red colonies with no color change to the medium (2).

Returning to the matter of complexity, and its relation to raw data and metadata, the complexity of the data informs as to the likelihood that a computer print-out (or hand written result) represents original data. A result of 6 CFU recorded into a Laboratory Information Management System (LIMS) or onto a report form is more likely to represent original data than is the result from an identification system.

Importance of microbiological data

As with other types of laboratory testing, some microbiological data is more important than others, and some if more vulnerable to data loss or corruption. The loss of a result of one sample from the environmental monitoring a lower grade cleanroom, where several samples are taken each week from the room, is less important, say, than the mislaying of a result from a sterility test.

In terms of data vulnerability, data held in computerized systems is more vulnerable to corruption; and all data is, if not covered by suitable procedures, vulnerable to unauthorized changes or manipulation. Good control of computerized systems, with appropriate password access and audit trails, can prevent the intended or unintended manipulation of data.

Regulatory findings

Recent letters issued by regulatory agencies like the FDA highlight an increased regulatory focus on data integrity issues. Areas of inspectorate concern in relation to microbiology and culture media include:

• Plate counting, where colony forming units are miscounted;

• Missing samples, such as environmental monitoring samples not being taken or dropped on transit to an incubator;

• Failure to take exit suit plates when a staff member leaves an aseptic processing area;

• Paperwork and samples not being reconciled;

• A worksheet being ripped up and the results re-written onto a second sheet;

• Lack of reconciliation of printed forms;

• Concerns with computer systems e.g. lack of passwords;

• Out-of-limits results from growth promotion testing not adequately investigated;

• Organism not matching the colonial description;

• Evidence of microbial identifications being re-run in order to obtain a better result;

• Temperature of incubator being out-of-specification with no risk assessment undertaken for the samples contained within the incubator;

• Samples retested with no reason recorded.

The above represents a small selection from FDA warning letters, the point being to highlight the importance of data integrity in microbiology.

As to why such problems arise, one reason is a lack of understanding of what data integrity entails or, where there is a base understanding, either incorrectly assuming that data integrity applies to computerized systems only or not fully realizing how data integrity relates to microbiology. Other, more general, reasons include:

• Ineffective supervision;

• Poorly designed controls;

• Lack of checking;

• Management competency;

• Not having audit trails (paper or electronic in place);

• Accuracy being too often assumed but not verified;

• Poor system security for electronic devices;

• No back-up or archiving of electronic devices;

• Cultural issues, such as seeking to avoid deviations.

Ensuring people are aware of data integrity falls within the overall scope of good data governance. This can partly be addressed through each facility having a data integrity policy and procedure. This is, however, insufficient without an appropriate culture and with having a quality risk management system that helps prevent, detect and control potential data integrity risks.

Variability and microbiological data

Variability affects a number of laboratory operations. To a degree variability can be controlled with validated analytical methods (through introducing categories like repeatability and robustness); the same degree of control extends to many rapid microbiological methods. However, the question of laboratory variability remains a central concern within microbiology. Microbiologists work every day with variability in the detection, recovery and growth of microbiological species, particularly when using culture based methods (3).

Variability can be divided between two categories (4, 5):

• “Avoidable” variability (variability due to poor practice);

• Inherently unavoidable variability (variability due to limitations of the methods and the vagaries of dealing with biological samples.

In the context of this article, avoidable variability connects with the subject of data integrity. It should be the goal of “best practices” to minimize “avoidable” microbiological error

Key factors for a robust data integrity system

There are several factors that make for a good data integrity system. Here it is important to have procedures are in place for all samples tested in the laboratory. Such procedures should describe the way to measure the quality, or acceptability, of the data. It is also important that all data is reviewed by the technician at the time of testing or reading. With this role, out of limits results need to clearly communicated.

There also needs to be a system for data checking. Personnel tasked with approving final reports should verify the data, check for typographical errors, ensure that corrections have been performed correctly, and review any calculations performed (such as dilution adjustments).

Further important factors can be drawn from the common acronym used for data integrity is ALOCA, which represents ‘Attributable, Legible, Contemporaneous, Original, and Accurate.’

For the person tasked with data review it is important that they understand what they are looking for and what they are approving. This is more difficult with an electronic record since knowing where and how changes are captured is less clear. With both paper systems and electronic systems it is important to avoid only reviewing ‘by exception’ (that is just reviewing out-of-limits results), since low count data may also have been recorded inaccurately.

A further good practice system to have in place is with the segregation of responsibilities. The person who records or enters data, for example, must not be the person who reviews and approves data (and roles should not be swapped either between tests or on different days). In addition, with computerized systems the administrator (the person who can change parameters) should, ideally, be independent of the department.

Assessing computerized systems

With computerized systems there are various guidance documents available.  Guidance has also been produced by the FDA (2016) (6) and the U.K. MHRA (2015) (7), together with other regulatory agencies. These documents provide useful advice, for example, stating that systems should be designed in a way that encourages compliance with the principles of data integrity. Some examples of this could include:

• Access to clocks for recording timed events.

• Accessibility of batch records at locations where activities take place so that ad hoc data recording and later transcription to official records is not necessary.

• Control over blank paper templates for data recording.

• User access rights which prevent (or audit trail) data amendments.

• Automated data capture or printers attached to equipment such as balances.

• Proximity of printers to relevant activities.

• Access to sampling points (e.g. for water systems).

• Access to raw data for staff performing data checking activities.

Regulatory agencies place an importance, in terms of computerized systems, on access and password control. Full use should be made of access controls to ensure that people have access only to functionality that is appropriate for their job role, and that actions are attributable to a specific individual. Furthermore, systems should be in place to ensure data back-up.

When carrying out internal audits, especially with respect to computerized systems, five key data integrity questions are:

• Is electronic data available?

• Is electronic data reviewed?

• Is meta data (audit trails) reviewed regularly?

• Are there clear segregation of duties?

• Has the system been validated for its intended use?

Common regulatory inspection areas include (8):

• Data processing and review, accuracy checks

• Potential for data manipulation and deletion

• Repeat testing / replicate data

• Date / time stamp manipulation

• Criteria used to invalidate data

• Data transfer to systems - Checks that data are not altered in value and/or meaning

• Level of checking should be statistically sound

• Security of the system and user access levels – appropriate segregation of duties

• Electronic signatures – use of individual and generic passwords

• Regular back-ups of all relevant data should be done. Integrity and accuracy of backup data and the ability to restore the data should be checked during validation and monitored periodically.

• Archived data should be checked for accessibility, readability and integrity.

• Audit trails.

• Change management, including changes to a part of the system may pose a risk due to interdependencies.

There are additional, system specific areas. However, covering these basic points provides the foundation for conducting a data integrity review.

Summary

Data integrity is an important subject and a regulatory ‘hot topic’. This subject matter can directly inform an auditor or inspector about the organization, for data integrity issues occur and are identified by auditors as a direct result of poor quality culture within the organization.

This article has considered the topic of data integrity and has used a microbiology laboratory as an example. While there are a few texts about data integrity, and several regulatory guidance documents, these do not focus on microbiology laboratories specifically. Given the variables with microbiological data and the fact that many items of ‘data’ are captured from culture media, a degree of interpretation is needed.

References

1. Schmitt, S. (2014a) Data Integrity, Pharmaceutical Technology Europe, 38 (7). Online edition: http://www.pharmtech.com/data-integrity

2. Albon, K., Davis, D., and Brooks, J.L. (2015) A Risk-Based Approach to Data Integrity, Pharmaceutical Technology 39 (7): 16-20

3. Sandle, T. (2012). ‘Environmental Monitoring: a practical approach’ In Moldenhauer, J. Environmental Monitoring: a comprehensive handbook, Volume 6, PDA/DHI: River Grove, USA, pp29-54

4. Singer, D. and Sutton, S. (2011) Microbiological Best Laboratory Practices, USP <1117> Value and Recent Changes to a Guidance of Quality Laboratory Practices, American Pharmaceutical Review, 14 (4): 41-47

5. Jarvis, B. 1989. Statistical aspects of the microbiological examination of foods. In Progress in Indust. Microbiol. 21. Elsevier Scientific Publishers B.V. Amsterdam

6. MHRA (2015) MHRA GMP Data Integrity Definitions and Guidance for Industry March 2015, Medicines Healthcare products and Regulatory Agency, London, UK

7. FDA (2016) Data Integrity and Compliance With CGMP, Draft Guidance for Industry, April 2016, U.S. Department of Health and Human Services, Food and Drug Administration, Washington

8. Schmitt, S. (2014b) Data Integrity - FDA and Global Regulatory Guidance, Journal of Validation Compliance, 20 (3). At: http://www.ivtnetwork.com/article/data-integrity-fda-and-global-regulatory-guidance