SlideShare a Scribd company logo
Drug Information Journal, Vol. 45, pp. 495–502, 2011  •  0092-8615/2011
Printed in the USA. All rights reserved. Copyright © 2011 Drug Information Association, Inc.
Submitted for publication: August 17, 2010
Accepted for publication: January 10, 2011
Key Words
Electronic data capture;
Electronic data review;
Real-time data access;
Clinical trials
Correspondence Address
Paula Brown Stafford,
Quintiles, Durham, NC
(email: paulabrown
.stafford@quintiles.com).
Paula Brown Stafford, BS,
MPH
President, Clinical
Development, Quintiles,
Durham, North Carolina
Andrew Garrett, PhD
Vice President, Global
Biostatistics and Medical
Writing, Quintiles,
Reading, United Kingdom
The new health landscape requires biopharma-
ceutical companies to conduct smarter clinical
trials and produce better outcomes faster. Elec-
tronic data capture and electronic data review
tools enable drug developers to review data
shortly after it is captured, creating opportuni-
ties for improving the process and outcomes of
clinical trials, and proactively managing quali-
ty, patient safety, and risk. We examine the
extraordinary opportunity for the biopharma-
ceutical industry to use real-time data to drive
better decisions faster. We identify the advan-
tages of access to real-time data during a clini-
cal trial, while recommending guidelines for
mitigating associated risks. Topics discussed
include the following:
•	 Technologies that make access to real-time clinical
trial data widely available
•	 Regulatory implications of reviewing real-time data
throughout clinical trials
•	 Clinical trial roles and the rationale for granting
real-time data access
•	 Guidelines and controls for use of real-time data
during clinical trials
•	 How real-time data enhance patient safety, strength-
en quality, and accelerate timelines
Using Real-time Data to
Drive Better Decisions, Faster
s t a t i s t i c s 495
I ntroduction
Advances in technology have enhanced data
capture capabilities, making more timely and
accurate information available to drug develop-
ers. Specifically, electronic data capture (EDC)
systems electronically collect patient data di-
rectly from an investigative site, and data in-
tegration tools combine electronic data from
various sources in real time. These advances,
alongside data standards initiatives brought
forward by the Clinical Data Interchange Stan-
dards Consortium (1), create new opportuni-
ties to streamline the clinical research process.
Drug developers can now access more complete
and higher-quality data faster.
Over the past several years, EDC has enabled
the use of adaptive trial designs, or multistage
trials during which drug developers review ac-
cumulating data at predetermined intervals to
decide how to modify aspects of a study after it
has started. These trials and the technology
that support them reduce the gap between ac-
cruing clinical data and decision making, offer-
ing drug developers an alternative to lengthy
and increasingly expensive traditional studies.
The same technology supporting adaptive
clinical trials makes possible ongoing review of
data throughout the course of a clinical trial. If
used appropriately, real-time data open the
door for drug developers to improve drug devel-
opment in the following ways:
1.	Enhancing patient safety
2.	Strengthening quality
3.	Accelerating timelines
With opportunity, however, comes risk. Tech-
nology has made information that once re-
quired statistical expertise to assemble avail-
able to a wider group of people. As with adaptive
clinical trials, more formal and clearly estab-
lished protocols describing how ongoing data
review will be conducted and documented and
plans for implementing an audit trail enabling
regulators to easily follow a drug developer’s
logic and method are required. With this, the
use of real-time data can help to make signifi-
cant strides in clinical research.
Real-Time Data Access:
Key Considerations
The notion of allowing drug developers open
access to data during clinical trials is somewhat
controversial, with the concern that even ag-
gregate, blinded data in the wrong hands can
be used to bias a trial. Still, with access to tech-
nology and a desire to enhance drug safety,
496 S t a t i s t i c s Brown Stafford, Garrett
there is now more pressure than ever to utilize
novel safety review tools to explore associations,
event combinations, coding disparities, safety
signals, and more by reviewing accruing data
throughout the course of a clinical trial.
The US Food and Drug Administration (FDA)
has signaled both concern and encouragement
for the use of real-time data and the novel tools
that enable it. In its guidance “Establishment
and Operation of Clinical Trial Data Monitoring
Committees,” the FDA stated, “Even aggregate
data on safety and efficacy may be informative;
these data . . . are best limited to those who can-
not otherwise carry out their trial management
responsibilities” (2).
Yet, during a presentation at the 19th annual
Euromeeting in March 2007, the FDA conclud-
ed, “The Critical Path Initiative and Opportuni-
ties list and the FDA response to the IOM (Insti-
tute of Medicine) report on safety highlight the
need for new tools and processes for signal de-
tection and clinical trial review” (3). However,
the FDA has also made clear that protecting
study blinding is “particularly important to
avoid the introduction of bias in the study con-
duct and to maintain confidence in the validity
of the study’s result” (4).
Utilizing real-time data, therefore, requires
careful management of the risks associated with
introducing bias into a study, and mitigating
them by ensuring that those individuals most
vested in whether a study is successful on safety
or efficacy criteria are most remote from the
data access.
A potential temptation to be avoided is the
case whereby a drug developer observes a high-
er incidence of adverse events in a placebo-
controlled trial at a particular investigative site.
The temptation in such an instance might be to
make an educated guess that the active group is
the one experiencing more adverse events, po-
tentially leading a drug developer to limit en-
rollment at that site, while encouraging enroll-
ment at another site with fewer adverse events.
Similarly, a graphic of aggregated data for a
quantitative efficacy outcome might reveal a bi-
modal distribution, suggestive of a difference
between the treatments. Although such shifts
are seldom observed early in a trial, since limit-
ed data are available, a small risk remains.
Therefore, although the sponsor organization
is charged to act on early safety signals—and
must be supplied with the data to do so—data
relating to efficacy could be more strictly con-
trolled to prevent data mining aimed at deter-
mining whether the study is proceeding suc-
cessfully.
The functioning of a typical data monitoring
committee (DMC) for interim analyses provides
a useful paradigm in the context of ongoing
data access. In a closed session, the DMC acts
as a third party, reviewing certain unblinded
data prepared by an independent biostatistics
center. This independence ensures that those
running the trial are not influenced by knowl-
edge of the treatment effect. However, the DMC
also operates an open session, attended by the
sponsor, where aggregated blinded data are re-
viewed, such as patient enrollment, protocol ad-
herence, descriptions of the patient population,
and descriptions of safety and perhaps efficacy.
To prohibit information from biasing a trial, the
information provided for both open and closed
sessions must be detailed in the DMC charter,
updated as the trial progresses and well docu-
mented.
By specifying in study protocols or in separate
data charters exactly how information will be
used, who will have access to what type of data
and when, and establishing appropriate fire-
walls and a clear audit trail as per FDA guidance
(5), controls over the use of real-time data can
be established and enforced, thereby minimiz-
ing risk. Additionally, sponsors can allay regula-
tor concerns about the introduction of bias and
avoid a compound being rejected for perceived
mishandling of trial data. Ongoing review of
real-time data is generally accepted by regula-
tors when study sponsors set out a plan and fol-
low it carefully.
Controlling Data A ccess
To establish the safe and effective use of real-
time data, it is important to grant access to pro-
fessionals on the basis of the role they play
within the clinical trial, and in accordance with
Real-time Data to Drive Better Decisions S t a t i s t i c s 497
Drug Information Journal
international good clinical practice guidance.
For example, while a clinical research associate
(CRA) may engage in a range of activities, the
primary responsibility of this role is to monitor
and ensure the scientific integrity of the data
being collected at investigator sites and the
well-being of trial patients. To carry out their
responsibilities, CRAs require access to the pa-
tient data being collected at investigator sites;
however, these individuals should not be grant-
ed access to aggregated on-treatment data. In-
stead, aggregated data is best placed in the
hands of the DMC to perform activities such as
sample size reestimation in an adaptive trial de-
sign.
Keeping in mind that specific job roles and
responsibilities may differ from sponsor to
sponsor, Tables 1 and 2 provide guidance for
granting access to real-time data based on pro-
fessional roles.
Care must always be taken to ensure that
real-time access to on-treatment data that may
inadvertently unblind patients and groups of
patients, such as pharmacokinetic or biomark-
er samples, is not provided. Indeed, even open-
label comparative studies can be reviewed in a
blinded manner simply by restricting access to
specific data—for instance, device informa-
tion for medical device trials. However, access
to pretreatment data across the range of data
captured in a clinical trial is broadly accept-
able since no knowledge of the treatments can
be imparted or inferred. From a system per-
spective, it is also important to balance the
risks of providing real-time data access with
the administrative burden of managing an
overly complex security protocol. In fact, in Ta-
ble 2, access levels have been deliberately sim-
plified to balance such competing demands.
Practical implementation along these lines ne-
cessitates defining role groups, distinguishing
between pretreatment and on-treatment data
by visit (or domain for event data), restricting
access according to investigator site ID, and re-
stricting reporting to one patient at a time for
some roles. However, a security protocol that is
designed to classify each variable on an indi-
vidual study basis (eg, is blood pressure a safe-
ty or efficacy variable) is likely to be overly bur-
densome.
Alongside real-time data access, data may be
summarized at certain cutoff times for specific
purposes—for instance, to enable the DMC to
perform a safety evaluation or to undertake a
dry run on the database to prepare for the final
analysis. In the case of dry runs, outputs are
produced using a dummy randomization so that
their format and layout can be finalized. Data
derivation rules are also tested and refined to
accommodate unexpected data combinations
while highlighting data issues. In both cases
there are clear advantages to having the most
up-to-date data to summarize. Yet, although
processes and documentation are most likely to
be in place for DMC dissemination, there has
been less control over the dissemination of dry-
run outputs. Although the risk of influencing
trial conduct may be viewed as remote, since
most dry runs are performed toward the end of
a trial or after the recruitment process is com-
plete, it remains good practice to plan, control,
and document which professionals have access
to what data, when they have access, and exactly
why access was granted. It is recommended that
this information is also detailed in the protocol
defining real-time data access.
Enhancing Patient S afety
Identifying patient safety concerns during clin-
ical trials is of paramount importance, and the
earlier issues associated with developmental
treatments are flagged and addressed, the bet-
ter. The ongoing review of patient data through-
out a trial plays a critical role in a drug devel-
oper’s ability to monitor patient safety events,
identify early safety signals, build a safety pro-
file, and check protocol compliance. The faster
this information is made available, the quicker
sponsors and health authorities will be empow-
ered to stop a study that is found to put patients
at risk, or to develop additional safety measures
for their protection.
The utility of real-time data with regard to
evaluating safety is illustrated with the applica-
tion of Hy’s law (6), which is used during drug
development to determine whether a drug
498 S t a t i s t i c s Brown Stafford, Garrett
TA B LE 1
Job Role Role Description Rationale for Granting Real-time Data Access
Lead CRA/CRA Ensure the well-being of patients enrolled
in trial. Ensure trial conduct is compliant
with the approved protocol amendment(s).
Monitor and ensure the scientific integrity
of study data—in terms of accuracy, com-
pleteness, and verifiability—collected at
investigator sites.
To ensure that enrolled patients meet the inclusion/exclusion
criteria.
To monitor the screening and recruitment numbers.
To check the accuracy and completeness of data entries and
that missed visits and assessments, withdrawals, and AEs are
reported within required time frames.
Clinical manager Coordinate cross-functional teams for a
specific study to ensure quality deliver-
ables on time, within budget, and in accor-
dance with SOPs, policies, and procedures.
To monitor patient enrollment and completion to ensure
cross-functional tasks are planned and timelines are met.
To investigate issues of protocol and SOP compliance—
including issues raised by others.
Global clinical lead Coordinate cross-functional teams for a
specific compound or indication to ensure
quality deliverables on time, within bud-
get, and in accordance with SOPs, policies,
and procedures.
To monitor patient enrollment and completion across a pro-
gram of studies to ensure regulatory submission tasks are
planned and timelines are met.
To investigate issues of patient eligibility, protocol compli-
ance, and patient withdrawal in relation to the drug program
as a whole.
Safety physician Undertake regular surveillance activities
directed at a specific developmental com-
pound, and prepare safety evaluation doc-
uments for review in cases where safety
signals are identified or suspected.
To identify potential safety issues as early as possible during
a trial by exploring associations, event combinations, and
temporal dependencies that may require action leading to
protocol modification or termination, and that may impact
other ongoing or planned studies.
Project statistician/
statistical programmer
Undertake statistical analysis and interpre-
tation for a specific study.
Write and test programs to perform data
manipulation and derivation, and produce
tables, listings, and figures.
To identify potential data errors, inconsistencies, patterns,
and outliers that may impact the results or analysis methods.
To implement statistical methods to identify poor practices or
potential fraud at investigator sites.
To develop programs using real data to accelerate delivery
times post-database lock.
To support the ongoing evaluation of a study by undertaking
programming tasks to facilitate safety surveillance activities
and provide information for DMCs (via an independent,
unblinded statistical center).
Program statistician/
statistical programmer
Undertake statistical planning, analysis,
and interpretation for a program of stud-
ies directed toward regulatory submission.
Design integrated databases, write and
test programs to combine and summarize
data across studies.
To develop analysis plans and programs to integrate and
report data across studies to characterize thoroughly a devel-
opmental compound and to accelerate submission delivery
times.
To support the ongoing evaluation of a developmental com-
pound by undertaking programming tasks across studies to
facilitate safety surveillance activities.
Lead data manager Ensure the accuracy, completeness, and
consistency of data through data capture,
edit checks, and data clarification for a
specific study.
To check data accuracy, completeness, and consistency.
To raise and resolve data queries.
Medical writer Create documents for a specific study that
describe research results accurately and
clearly, and that comply with regulatory
guidance in terms of content, format, and
structure.
To prepare patient narratives on an ongoing basis and high-
light potential data issues to the project team.
Description of Clinical Trial Roles and Rationale for Granting Real-time Data Access
Real-time Data to Drive Better Decisions S t a t i s t i c s 499
Drug Information Journal
could cause fatal liver injury or sufficient dam-
age to require liver transplant. Most drugs caus-
ing severe liver injury do so infrequently, and
the incidence of damage is generally not picked
up during the course of a trial. Access to accru-
ing laboratory data enables faster detection of
markers (eg, elevated serum aminotransferases
accompanied by increased serum total biliru-
bin) that cannot be explained by other causes,
therefore, playing a critical role in assessing the
potential toxicity of a drug.
It is important, therefore, in a real-time way,
to obtain data generated from patients’ med-
ical history, vital signs, laboratory, concomitant
medications and diseases, and adverse events.
Early detection relies not only on accessing
data in near real time, but on the ability to inte-
grate information that has traditionally been
housed in disparate locations so that meaning-
ful trending reports can be created. Depend-
able reports made available shortly after data
capture are critical to the flow of useful infor-
mation, and are necessary for all levels of study
management to take action when safety con-
cerns arise. Graphical displays are particularly
helpful to identify patterns and to evaluate tem-
poral relationships for individual patients. More
advanced graphics include motion-enabled
bubble plots that plot data in three dimensions
through time. Review tools depend upon real-
time data to allow drug developers to identify
possible safety concerns as they materialize,
rather than months after the fact. These inter-
active tools enhance DMC meetings by en-
abling reviews to be performed more effectively
and thoroughly, providing the capability to drill
down into specific patients and link informa-
tion efficiently—a common frustration for
committee members who have a limited window
to undertake such tasks.
TA B LE 2
Job Role Data Type/Access Levela
  Administrative data
(eg, CRF completion;
protocol/patient com-
pliance; enrollment
and visit attendance
data, etc)
Pretreatment data
(ie, any trial data col-
lected up until the first
exposure to study
drug, including lab, vi-
tal signs, pretreatment
adverse events, pre-
treatment efficacy, pre-
treatment safety, de-
mographic/patient
accounting data, etc)
On-treatment and
follow-up efficacy data
(eg, NINCDS-ADRDA
scores for an Alzheimer
study)
On-treatment and
follow-up safety data
(ie, after treatment
lab, vital signs, adverse
events, ECG, concomi-
tant medications, etc)
Lead CRA/CRA Single study access Single study access Single study access (by-
patient only)
Single study access (by-
patient only)
Clinical manager Single study access Single study access No access No access
Global clinical lead Multistudy access Multistudy access No access No access
Safety physician Multistudy access Multistudy access Multistudy access Multistudy access
Project statistician/
statistical programmer
Single study access Single study access Single study access Single study access
Program statistician/
statistical programmer
Multistudy access Multistudy access Multistudy access Multistudy access
Lead data manager Single study access Single study access Single study access Single study access
Medical writer Single study access Single study access Single study access Single study access
aBoth aggregated and by-patient data unless otherwise specified.
Guidelines for Granting Access to Blinded Real-time Trial Data
500 S t a t i s t i c s Brown Stafford, Garrett
As an additional protection, accumulating
safety information can be combined and com-
pared with safety data from previously complet-
ed studies, essentially forming a dynamic, in-
tegrated summary of safety. In this way, drug
developers are more quickly able to identify
and monitor for safety flags and adverse events,
making the most current trial more informative
with regard to a compound’s relative risk/
benefit. As information accrues, researchers are
aware of the totality of safety information avail-
able for the compound being studied by incor-
porating it with previously acquired data. Ad-
verse events in excess of previous studies might
be an indicator for an independent safety re-
viewer, such as a DMC or project safety physi-
cian, to further scrutinize the data.
Strengthening Quality
The quality and integrity of a clinical study de-
pends on the ability to maintain protocol com-
pliance from the outset. With greater regulatory
scrutiny over protocol deviators and violators,
and a record number of FDA warning letters,
drug developers must ensure that patients not
only meet inclusion and exclusion criteria, but
also adhere to the protocol throughout the
course of a clinical study. Ensuring protocol
compliance is becoming increasingly difficult
as protocols increase in complexity; however,
real-time data play a significant role in helping
drug developers quickly and easily detect proto-
col deviators and violators, thereby managing
the ultimate quality of a clinical trial. Protocol
adherence can also be strengthened by investi-
gating the capture of all scheduled data at each
visit, by monitoring visit attendance according
to protocol schedules, and by reviewing reasons
for patient withdrawal.
The quality of a study is also dependent
upon the highest level of data accuracy. For ex-
ample, if errors at any trial site go undetected,
they can compromise the quality and integrity
of the entire study. Real-time data play a criti-
cal role in helping researchers achieve data ac-
curacy from the outset, spotting missing data,
identifying data errors, or highlighting poten-
tially fraudulent data early. For example, if a
laboratory assessment occurs without the
completion of a corresponding clinical visit,
then a missing page report can be generated
in real time.
Furthermore, real-time data allow for signal
detection of quality issues long before issues
become real problems. By using data to spot
trends or early warning signs, drug developers
can proactively investigate and correct poten-
tial issues early—ensuring both corrective and
preventive actions are taken. Quality signals in-
clude edit checks firing more often than the
targeted value, prompting additional training
or coaching to the problematic investigator site.
On the other hand, edit checks firing too infre-
quently could suggest that a review of the edit
check code is required.
The use of data to detect potential fraud
is well documented (7), and statistical tech-
niques have been developed that may point to
further investigation of specific investigator
sites. Methods include: end digit preference
where the last digit of a set of assessments has
an unusual pattern; and detecting inliers (rath-
er than outliers) through the investigation of
multivariate data structures where fraudulent
data often exhibit markedly less variability rel-
ative to that observed elsewhere. Fraud re-
mains an uncommon problem, but real-time
data access enables checking programs to be
put in place that encourage proactive trial
management in this area.
The use of real-time data allows for ongoing
review of a site’s past site performance, the
number of subjects and the rate of site recruit-
ment, staff feedback on protocol compliance,
site contact and record keeping, information
received from data management, inaccurate or
repetitive data, and other variables. Real-time
access to each of these and other data points
enable more timely site review triggers, which
help to prevent the quality of the study from be-
ing threatened.
The ability to demonstrate that real-time ac-
cess to data is being utilized for the purposes of
improving the conduct of the trial is likely to be
Real-time Data to Drive Better Decisions S t a t i s t i c s 501
Drug Information Journal
well received by regulators. Real-time data ac-
cess can also support and inform a sponsor’s
quality plan.
Accelerating Timelines
Technology designed to capture and integrate
accruing patient data has enabled the increas-
ingly widespread use of adaptive clinical trials.
Adaptive designs, which use a set of design rules
to define a priori which modifications can be
incorporated into the trial design, reduce the
length of time required to complete a clinical
study and have potential to accelerate the de-
velopment of promising therapies. Use of adap-
tive designs allows stopping decisions at the
earliest possible time point, avoiding subjecting
patients to noneffective or unsafe therapies.
Adaptive designs involve interim analyses,
which are planned at the outset and well con-
trolled. It is understood that results must be
handled carefully, with firewalls put in place to
ensure that those with access to the data are
not a part of the study team.
Ongoing review of real-time data differs from
adaptive trials in that access to data is more
open and, to date, its use has been less well de-
fined. Furthermore, adaptive trials typically re-
quire access to treatment assignments whereas
real-time data access does not. It is important
to note, however, that one commonly used sam-
ple size reestimation method for adaptive trials
only utilizes the pooled estimate of the variance
(or the probability of events) from the blinded
aggregated data. But like adaptive trials, ongo-
ing review of data can play an important role in
the acceleration of a study’s timeline.
For example, open access to blinded, real-
time information makes faster data cleanup
possible. Undetected errors in data capture can
delay database lock or, worse, compromise the
overall study. Errors are much more quickly de-
tected with the use of real-time data than has
traditionally been the case. This helps to ensure
accuracy and encourages standardization of
training and processes across trial sites, as well
as quicker remediation when a problem is iden-
tified.
In addition, drug developers are able to con-
duct a blinded dry run of the output process at
various points throughout a study, which en-
hances data cleaning and aids in finalizing data
derivation rules and validating programming
code. The manner in which data are organized
in tables (of which there can be hundreds) can
be reviewed at preplanned intervals by the glob-
al clinical lead overseeing the trial and other
members of the sponsor organization months in
advance of the trial’s completion, thereby expe-
diting data output at a study’s conclusion.
It is imperative with dry runs to put in place
protections against the misuse of data and to es-
tablish at the outset exactly who has access to
what information, when. By ensuring that the
data under review are aggregated but un-
grouped, or grouped randomly using dummy as-
signments, those most concerned with the effi-
cacy of the trial are only able to review table
formatting and not mine the data in any way
that would jeopardize the integrity of the trial.
Finally, quotas for enrollment, which nearly
80% of trials fail to meet, are better managed
with real-time data review (8). Access to real-
time data offers researchers a timely look at the
state of enrollment for the trial as a whole, and
an opportunity to put a corrective action plan
in place when the data indicate enrollment is
slower than predicted.
Conclusion
As technology and tools better enable the cap-
ture, integration, and evaluation of data in real
time, drug developers will have new opportuni-
ties to enhance the process and outcomes of
clinical trials in the new health landscape. For
drug developers interested in leveraging real-
time data access, however, it is imperative to
recognize that it must be done in a planned,
controlled, and documented way to avoid risks
associated with biasing a trial.
It is important to consider formulating clear
review policies, as well as the use of appro-
priate data review tools that limit user access
to data. The intent to review real-time data
throughout the course of a trial should also be
502 S t a t i s t i c s Brown Stafford, Garrett
clearly established at the outset within a clini-
cal trial protocol or separate real-time data ac-
cess protocol. Finally, a clear audit trail must
be established so that regulators can easily de-
termine how and when data were reviewed and
by whom.
The advantages of reviewing real-time data far
outweigh the dangers and allow for earlier de-
tection of patient safety issues, faster timelines,
and higher-quality clinical trials.
Acknowledgments—The authors would like to thank
the following colleagues at Quintiles who reviewed
and provided valuable comment on earlier drafts of
the manuscript: Vladimir Dragalin, Thomas Grund-
strom, Michael Fiola, David McGowan, Michael
O’Kelly, and Lindsay Singler. The authors would also
like to express thanks to Gary Koch for his valuable
contribution.
R eferences
1.	 Clinical Data Interchange Standards Consortium.
http://www.cdisc.org/mission-and-principles.
2.	 Food and Drug Administration. Guidance for clin-
ical trial sponsors: establishment and operation of
clinical trial data monitoring committees, section
4.2.2. March 2006. http://www.fda.gov/OHRMS/
DOCKETS/98fr/01d-0489-gdl0003.pdf.
3.	 O’Neill R. Signal detection in clinical trials: some
perspectives on new tools and processes—a criti-
cal path update. 19th DIA Annual Euromeeting.
Vienna, Austria, March 26–28, 2007.
4.	 Food and Drug Administration. Guidance for in-
dustry: adaptive design clinical trials for drugs
and biologics. February 2010. http://www.fda.gov/
downloads/Drugs/guidancecomplianceregula
toryinformation/guidances/ucm201790.pdf.
5.	 Food and Drug Administration. Guidance for in-
dustry: part 11: electronic records; electronic sig-
natures—scope and application. August 2003.
http://www.fda.gov/downloads/drugs/guidance
complianceregulatoryinformation/guidances/
ucm070295.pdf.
6.	 Food and Drug Administration. Guidance for in-
dustry: drug-induced liver injury: premarketing
clinical evaluation. July 2009. http://www.fda
.gov/downloads/Drugs/GuidanceCompliance
RegulatoryInformation/Guidances/UCM174090
.pdf.
7.	 O’Kelly M. Using statistical techniques to detect
fraud: a test case. Pharm Stat. 2004;3(4):237–246.
http://onlinelibrary.wiley.com/doi/10.1002/
pst.137/abstract.
8.	 Hess J, Litalien S. Web-based patient recruitment:
best opportunity to accelerate clinical trials. Cut-
ting Edge Information, 2005.
Paula Brown Stafford has disclosed that she is an employee of Quintiles, that she received honoraria from the
UNC CTSA External Advisory Board, and that as an employee of Quintiles she currently undertakes, and in the
past has undertaken, extensive work for the biopharmaceutical industry. Andrew Garrett has disclosed that he is
an employee of Quintiles and that in this capacity he currently undertakes, and in the past has undertaken, ex-
tensive work for the biopharmaceutical industry.

More Related Content

PDF
Lessons Learned From a DDE Phase 2 CT, 2012
PDF
Data & Technology
PDF
Patients Recruitment Forecast in Clinical Trials
PDF
Clinical Trial Recruitment & Retention
PPTX
Clinical Data Management
PDF
CPT.RWE for decision making.2016
PDF
Capturing Patient-Reported Outcome (PRO) Data Electronically: The Past, Prese...
PDF
CROs Driving Changes in Patient Recruitment
Lessons Learned From a DDE Phase 2 CT, 2012
Data & Technology
Patients Recruitment Forecast in Clinical Trials
Clinical Trial Recruitment & Retention
Clinical Data Management
CPT.RWE for decision making.2016
Capturing Patient-Reported Outcome (PRO) Data Electronically: The Past, Prese...
CROs Driving Changes in Patient Recruitment

What's hot (20)

PDF
OTC Electronic Clinical Trial Recruitment
PDF
Genomic data and the electronic health record (ehr)
PPTX
Clinical research and clinical data management - Ikya Global
PDF
Results of an Online Survey of Stakeholders Regarding Barriers and Solutions ...
PDF
Clinical Trials Re-spec. The role of games and game concepts in the future of...
PDF
Best strategies for successful recruitment and retention
PPT
Clinical Trial Recruitment
PDF
Clinical trial recruitment overview
PDF
Pathway 2.0 for RWE and MA 2015 -John Cai
PDF
Clinical Data Review Best Practices - E. Herbel
DOCX
Predicting Diabetic Readmission Rates: Moving Beyond HbA1c
PDF
Patient Locate For Clinical Research
PDF
IRJET - E-Health Chain and Anticipation of Future Disease
PDF
Embracing Electronic PRO
PDF
2nd CUTEHeart Workshop Constantine Gatsosis Presentation
PPTX
AllTrials AAAS 2015 - Access to anonymised patient level data
PDF
Designing Risk Metrics for Risk-Based Monitoring
PDF
Mosio's Clinical Trial Patient Recruitment and Retention Ebook (First Edition)
DOCX
Oral presentation1
OTC Electronic Clinical Trial Recruitment
Genomic data and the electronic health record (ehr)
Clinical research and clinical data management - Ikya Global
Results of an Online Survey of Stakeholders Regarding Barriers and Solutions ...
Clinical Trials Re-spec. The role of games and game concepts in the future of...
Best strategies for successful recruitment and retention
Clinical Trial Recruitment
Clinical trial recruitment overview
Pathway 2.0 for RWE and MA 2015 -John Cai
Clinical Data Review Best Practices - E. Herbel
Predicting Diabetic Readmission Rates: Moving Beyond HbA1c
Patient Locate For Clinical Research
IRJET - E-Health Chain and Anticipation of Future Disease
Embracing Electronic PRO
2nd CUTEHeart Workshop Constantine Gatsosis Presentation
AllTrials AAAS 2015 - Access to anonymised patient level data
Designing Risk Metrics for Risk-Based Monitoring
Mosio's Clinical Trial Patient Recruitment and Retention Ebook (First Edition)
Oral presentation1
Ad

Similar to Using real-time-data-to-drive-better-decisions-faster (20)

PPTX
Challenges and Opportunities Around Integration of Clinical Trials Data
PDF
Unified Medical Data Platform focused on Accuracy
PDF
Risk-based SDV Approaches-Pros & Cons, 2010
PPT
Patient Safety in Clinical Trials
PDF
Therapeutic_Innovation_&_Regulatory_Science-2015-Tantsyura
PDF
Clinical Solutions White Paper
DOCX
AUTOMATED DATABASES INTRO.docx
PDF
Listening to the Patient - Leveraging Direct-to-Patient Data Collection to Sh...
PDF
Big data, RWE and AI in Clinical Trials made simple
PDF
The Role of Real-World Data in Clinical Development
PPTX
SDTM Fnal Detail Training
DOCX
What lies ahead for india
PDF
Final FDA guidance on Medical Devices and Risk Analysis
PDF
Decentralized trials white paper by Andaman7
PDF
Research methodologies that result in data collecting from the patient medica...
PDF
System for Recommending Drugs Based on Machine Learning Sentiment Analysis of...
PDF
Annotation Editorial
DOCX
Chapter 4 Knowledge Discovery, Data Mining, and Practice-Based Evi.docx
PPTX
Use of data analytics in health care
PDF
The Increasing Importance of Patient Reported Outcomes and the Patient Voice ...
Challenges and Opportunities Around Integration of Clinical Trials Data
Unified Medical Data Platform focused on Accuracy
Risk-based SDV Approaches-Pros & Cons, 2010
Patient Safety in Clinical Trials
Therapeutic_Innovation_&_Regulatory_Science-2015-Tantsyura
Clinical Solutions White Paper
AUTOMATED DATABASES INTRO.docx
Listening to the Patient - Leveraging Direct-to-Patient Data Collection to Sh...
Big data, RWE and AI in Clinical Trials made simple
The Role of Real-World Data in Clinical Development
SDTM Fnal Detail Training
What lies ahead for india
Final FDA guidance on Medical Devices and Risk Analysis
Decentralized trials white paper by Andaman7
Research methodologies that result in data collecting from the patient medica...
System for Recommending Drugs Based on Machine Learning Sentiment Analysis of...
Annotation Editorial
Chapter 4 Knowledge Discovery, Data Mining, and Practice-Based Evi.docx
Use of data analytics in health care
The Increasing Importance of Patient Reported Outcomes and the Patient Voice ...
Ad

Using real-time-data-to-drive-better-decisions-faster

  • 1. Drug Information Journal, Vol. 45, pp. 495–502, 2011  •  0092-8615/2011 Printed in the USA. All rights reserved. Copyright © 2011 Drug Information Association, Inc. Submitted for publication: August 17, 2010 Accepted for publication: January 10, 2011 Key Words Electronic data capture; Electronic data review; Real-time data access; Clinical trials Correspondence Address Paula Brown Stafford, Quintiles, Durham, NC (email: paulabrown .stafford@quintiles.com). Paula Brown Stafford, BS, MPH President, Clinical Development, Quintiles, Durham, North Carolina Andrew Garrett, PhD Vice President, Global Biostatistics and Medical Writing, Quintiles, Reading, United Kingdom The new health landscape requires biopharma- ceutical companies to conduct smarter clinical trials and produce better outcomes faster. Elec- tronic data capture and electronic data review tools enable drug developers to review data shortly after it is captured, creating opportuni- ties for improving the process and outcomes of clinical trials, and proactively managing quali- ty, patient safety, and risk. We examine the extraordinary opportunity for the biopharma- ceutical industry to use real-time data to drive better decisions faster. We identify the advan- tages of access to real-time data during a clini- cal trial, while recommending guidelines for mitigating associated risks. Topics discussed include the following: • Technologies that make access to real-time clinical trial data widely available • Regulatory implications of reviewing real-time data throughout clinical trials • Clinical trial roles and the rationale for granting real-time data access • Guidelines and controls for use of real-time data during clinical trials • How real-time data enhance patient safety, strength- en quality, and accelerate timelines Using Real-time Data to Drive Better Decisions, Faster s t a t i s t i c s 495 I ntroduction Advances in technology have enhanced data capture capabilities, making more timely and accurate information available to drug develop- ers. Specifically, electronic data capture (EDC) systems electronically collect patient data di- rectly from an investigative site, and data in- tegration tools combine electronic data from various sources in real time. These advances, alongside data standards initiatives brought forward by the Clinical Data Interchange Stan- dards Consortium (1), create new opportuni- ties to streamline the clinical research process. Drug developers can now access more complete and higher-quality data faster. Over the past several years, EDC has enabled the use of adaptive trial designs, or multistage trials during which drug developers review ac- cumulating data at predetermined intervals to decide how to modify aspects of a study after it has started. These trials and the technology that support them reduce the gap between ac- cruing clinical data and decision making, offer- ing drug developers an alternative to lengthy and increasingly expensive traditional studies. The same technology supporting adaptive clinical trials makes possible ongoing review of data throughout the course of a clinical trial. If used appropriately, real-time data open the door for drug developers to improve drug devel- opment in the following ways: 1. Enhancing patient safety 2. Strengthening quality 3. Accelerating timelines With opportunity, however, comes risk. Tech- nology has made information that once re- quired statistical expertise to assemble avail- able to a wider group of people. As with adaptive clinical trials, more formal and clearly estab- lished protocols describing how ongoing data review will be conducted and documented and plans for implementing an audit trail enabling regulators to easily follow a drug developer’s logic and method are required. With this, the use of real-time data can help to make signifi- cant strides in clinical research. Real-Time Data Access: Key Considerations The notion of allowing drug developers open access to data during clinical trials is somewhat controversial, with the concern that even ag- gregate, blinded data in the wrong hands can be used to bias a trial. Still, with access to tech- nology and a desire to enhance drug safety,
  • 2. 496 S t a t i s t i c s Brown Stafford, Garrett there is now more pressure than ever to utilize novel safety review tools to explore associations, event combinations, coding disparities, safety signals, and more by reviewing accruing data throughout the course of a clinical trial. The US Food and Drug Administration (FDA) has signaled both concern and encouragement for the use of real-time data and the novel tools that enable it. In its guidance “Establishment and Operation of Clinical Trial Data Monitoring Committees,” the FDA stated, “Even aggregate data on safety and efficacy may be informative; these data . . . are best limited to those who can- not otherwise carry out their trial management responsibilities” (2). Yet, during a presentation at the 19th annual Euromeeting in March 2007, the FDA conclud- ed, “The Critical Path Initiative and Opportuni- ties list and the FDA response to the IOM (Insti- tute of Medicine) report on safety highlight the need for new tools and processes for signal de- tection and clinical trial review” (3). However, the FDA has also made clear that protecting study blinding is “particularly important to avoid the introduction of bias in the study con- duct and to maintain confidence in the validity of the study’s result” (4). Utilizing real-time data, therefore, requires careful management of the risks associated with introducing bias into a study, and mitigating them by ensuring that those individuals most vested in whether a study is successful on safety or efficacy criteria are most remote from the data access. A potential temptation to be avoided is the case whereby a drug developer observes a high- er incidence of adverse events in a placebo- controlled trial at a particular investigative site. The temptation in such an instance might be to make an educated guess that the active group is the one experiencing more adverse events, po- tentially leading a drug developer to limit en- rollment at that site, while encouraging enroll- ment at another site with fewer adverse events. Similarly, a graphic of aggregated data for a quantitative efficacy outcome might reveal a bi- modal distribution, suggestive of a difference between the treatments. Although such shifts are seldom observed early in a trial, since limit- ed data are available, a small risk remains. Therefore, although the sponsor organization is charged to act on early safety signals—and must be supplied with the data to do so—data relating to efficacy could be more strictly con- trolled to prevent data mining aimed at deter- mining whether the study is proceeding suc- cessfully. The functioning of a typical data monitoring committee (DMC) for interim analyses provides a useful paradigm in the context of ongoing data access. In a closed session, the DMC acts as a third party, reviewing certain unblinded data prepared by an independent biostatistics center. This independence ensures that those running the trial are not influenced by knowl- edge of the treatment effect. However, the DMC also operates an open session, attended by the sponsor, where aggregated blinded data are re- viewed, such as patient enrollment, protocol ad- herence, descriptions of the patient population, and descriptions of safety and perhaps efficacy. To prohibit information from biasing a trial, the information provided for both open and closed sessions must be detailed in the DMC charter, updated as the trial progresses and well docu- mented. By specifying in study protocols or in separate data charters exactly how information will be used, who will have access to what type of data and when, and establishing appropriate fire- walls and a clear audit trail as per FDA guidance (5), controls over the use of real-time data can be established and enforced, thereby minimiz- ing risk. Additionally, sponsors can allay regula- tor concerns about the introduction of bias and avoid a compound being rejected for perceived mishandling of trial data. Ongoing review of real-time data is generally accepted by regula- tors when study sponsors set out a plan and fol- low it carefully. Controlling Data A ccess To establish the safe and effective use of real- time data, it is important to grant access to pro- fessionals on the basis of the role they play within the clinical trial, and in accordance with
  • 3. Real-time Data to Drive Better Decisions S t a t i s t i c s 497 Drug Information Journal international good clinical practice guidance. For example, while a clinical research associate (CRA) may engage in a range of activities, the primary responsibility of this role is to monitor and ensure the scientific integrity of the data being collected at investigator sites and the well-being of trial patients. To carry out their responsibilities, CRAs require access to the pa- tient data being collected at investigator sites; however, these individuals should not be grant- ed access to aggregated on-treatment data. In- stead, aggregated data is best placed in the hands of the DMC to perform activities such as sample size reestimation in an adaptive trial de- sign. Keeping in mind that specific job roles and responsibilities may differ from sponsor to sponsor, Tables 1 and 2 provide guidance for granting access to real-time data based on pro- fessional roles. Care must always be taken to ensure that real-time access to on-treatment data that may inadvertently unblind patients and groups of patients, such as pharmacokinetic or biomark- er samples, is not provided. Indeed, even open- label comparative studies can be reviewed in a blinded manner simply by restricting access to specific data—for instance, device informa- tion for medical device trials. However, access to pretreatment data across the range of data captured in a clinical trial is broadly accept- able since no knowledge of the treatments can be imparted or inferred. From a system per- spective, it is also important to balance the risks of providing real-time data access with the administrative burden of managing an overly complex security protocol. In fact, in Ta- ble 2, access levels have been deliberately sim- plified to balance such competing demands. Practical implementation along these lines ne- cessitates defining role groups, distinguishing between pretreatment and on-treatment data by visit (or domain for event data), restricting access according to investigator site ID, and re- stricting reporting to one patient at a time for some roles. However, a security protocol that is designed to classify each variable on an indi- vidual study basis (eg, is blood pressure a safe- ty or efficacy variable) is likely to be overly bur- densome. Alongside real-time data access, data may be summarized at certain cutoff times for specific purposes—for instance, to enable the DMC to perform a safety evaluation or to undertake a dry run on the database to prepare for the final analysis. In the case of dry runs, outputs are produced using a dummy randomization so that their format and layout can be finalized. Data derivation rules are also tested and refined to accommodate unexpected data combinations while highlighting data issues. In both cases there are clear advantages to having the most up-to-date data to summarize. Yet, although processes and documentation are most likely to be in place for DMC dissemination, there has been less control over the dissemination of dry- run outputs. Although the risk of influencing trial conduct may be viewed as remote, since most dry runs are performed toward the end of a trial or after the recruitment process is com- plete, it remains good practice to plan, control, and document which professionals have access to what data, when they have access, and exactly why access was granted. It is recommended that this information is also detailed in the protocol defining real-time data access. Enhancing Patient S afety Identifying patient safety concerns during clin- ical trials is of paramount importance, and the earlier issues associated with developmental treatments are flagged and addressed, the bet- ter. The ongoing review of patient data through- out a trial plays a critical role in a drug devel- oper’s ability to monitor patient safety events, identify early safety signals, build a safety pro- file, and check protocol compliance. The faster this information is made available, the quicker sponsors and health authorities will be empow- ered to stop a study that is found to put patients at risk, or to develop additional safety measures for their protection. The utility of real-time data with regard to evaluating safety is illustrated with the applica- tion of Hy’s law (6), which is used during drug development to determine whether a drug
  • 4. 498 S t a t i s t i c s Brown Stafford, Garrett TA B LE 1 Job Role Role Description Rationale for Granting Real-time Data Access Lead CRA/CRA Ensure the well-being of patients enrolled in trial. Ensure trial conduct is compliant with the approved protocol amendment(s). Monitor and ensure the scientific integrity of study data—in terms of accuracy, com- pleteness, and verifiability—collected at investigator sites. To ensure that enrolled patients meet the inclusion/exclusion criteria. To monitor the screening and recruitment numbers. To check the accuracy and completeness of data entries and that missed visits and assessments, withdrawals, and AEs are reported within required time frames. Clinical manager Coordinate cross-functional teams for a specific study to ensure quality deliver- ables on time, within budget, and in accor- dance with SOPs, policies, and procedures. To monitor patient enrollment and completion to ensure cross-functional tasks are planned and timelines are met. To investigate issues of protocol and SOP compliance— including issues raised by others. Global clinical lead Coordinate cross-functional teams for a specific compound or indication to ensure quality deliverables on time, within bud- get, and in accordance with SOPs, policies, and procedures. To monitor patient enrollment and completion across a pro- gram of studies to ensure regulatory submission tasks are planned and timelines are met. To investigate issues of patient eligibility, protocol compli- ance, and patient withdrawal in relation to the drug program as a whole. Safety physician Undertake regular surveillance activities directed at a specific developmental com- pound, and prepare safety evaluation doc- uments for review in cases where safety signals are identified or suspected. To identify potential safety issues as early as possible during a trial by exploring associations, event combinations, and temporal dependencies that may require action leading to protocol modification or termination, and that may impact other ongoing or planned studies. Project statistician/ statistical programmer Undertake statistical analysis and interpre- tation for a specific study. Write and test programs to perform data manipulation and derivation, and produce tables, listings, and figures. To identify potential data errors, inconsistencies, patterns, and outliers that may impact the results or analysis methods. To implement statistical methods to identify poor practices or potential fraud at investigator sites. To develop programs using real data to accelerate delivery times post-database lock. To support the ongoing evaluation of a study by undertaking programming tasks to facilitate safety surveillance activities and provide information for DMCs (via an independent, unblinded statistical center). Program statistician/ statistical programmer Undertake statistical planning, analysis, and interpretation for a program of stud- ies directed toward regulatory submission. Design integrated databases, write and test programs to combine and summarize data across studies. To develop analysis plans and programs to integrate and report data across studies to characterize thoroughly a devel- opmental compound and to accelerate submission delivery times. To support the ongoing evaluation of a developmental com- pound by undertaking programming tasks across studies to facilitate safety surveillance activities. Lead data manager Ensure the accuracy, completeness, and consistency of data through data capture, edit checks, and data clarification for a specific study. To check data accuracy, completeness, and consistency. To raise and resolve data queries. Medical writer Create documents for a specific study that describe research results accurately and clearly, and that comply with regulatory guidance in terms of content, format, and structure. To prepare patient narratives on an ongoing basis and high- light potential data issues to the project team. Description of Clinical Trial Roles and Rationale for Granting Real-time Data Access
  • 5. Real-time Data to Drive Better Decisions S t a t i s t i c s 499 Drug Information Journal could cause fatal liver injury or sufficient dam- age to require liver transplant. Most drugs caus- ing severe liver injury do so infrequently, and the incidence of damage is generally not picked up during the course of a trial. Access to accru- ing laboratory data enables faster detection of markers (eg, elevated serum aminotransferases accompanied by increased serum total biliru- bin) that cannot be explained by other causes, therefore, playing a critical role in assessing the potential toxicity of a drug. It is important, therefore, in a real-time way, to obtain data generated from patients’ med- ical history, vital signs, laboratory, concomitant medications and diseases, and adverse events. Early detection relies not only on accessing data in near real time, but on the ability to inte- grate information that has traditionally been housed in disparate locations so that meaning- ful trending reports can be created. Depend- able reports made available shortly after data capture are critical to the flow of useful infor- mation, and are necessary for all levels of study management to take action when safety con- cerns arise. Graphical displays are particularly helpful to identify patterns and to evaluate tem- poral relationships for individual patients. More advanced graphics include motion-enabled bubble plots that plot data in three dimensions through time. Review tools depend upon real- time data to allow drug developers to identify possible safety concerns as they materialize, rather than months after the fact. These inter- active tools enhance DMC meetings by en- abling reviews to be performed more effectively and thoroughly, providing the capability to drill down into specific patients and link informa- tion efficiently—a common frustration for committee members who have a limited window to undertake such tasks. TA B LE 2 Job Role Data Type/Access Levela   Administrative data (eg, CRF completion; protocol/patient com- pliance; enrollment and visit attendance data, etc) Pretreatment data (ie, any trial data col- lected up until the first exposure to study drug, including lab, vi- tal signs, pretreatment adverse events, pre- treatment efficacy, pre- treatment safety, de- mographic/patient accounting data, etc) On-treatment and follow-up efficacy data (eg, NINCDS-ADRDA scores for an Alzheimer study) On-treatment and follow-up safety data (ie, after treatment lab, vital signs, adverse events, ECG, concomi- tant medications, etc) Lead CRA/CRA Single study access Single study access Single study access (by- patient only) Single study access (by- patient only) Clinical manager Single study access Single study access No access No access Global clinical lead Multistudy access Multistudy access No access No access Safety physician Multistudy access Multistudy access Multistudy access Multistudy access Project statistician/ statistical programmer Single study access Single study access Single study access Single study access Program statistician/ statistical programmer Multistudy access Multistudy access Multistudy access Multistudy access Lead data manager Single study access Single study access Single study access Single study access Medical writer Single study access Single study access Single study access Single study access aBoth aggregated and by-patient data unless otherwise specified. Guidelines for Granting Access to Blinded Real-time Trial Data
  • 6. 500 S t a t i s t i c s Brown Stafford, Garrett As an additional protection, accumulating safety information can be combined and com- pared with safety data from previously complet- ed studies, essentially forming a dynamic, in- tegrated summary of safety. In this way, drug developers are more quickly able to identify and monitor for safety flags and adverse events, making the most current trial more informative with regard to a compound’s relative risk/ benefit. As information accrues, researchers are aware of the totality of safety information avail- able for the compound being studied by incor- porating it with previously acquired data. Ad- verse events in excess of previous studies might be an indicator for an independent safety re- viewer, such as a DMC or project safety physi- cian, to further scrutinize the data. Strengthening Quality The quality and integrity of a clinical study de- pends on the ability to maintain protocol com- pliance from the outset. With greater regulatory scrutiny over protocol deviators and violators, and a record number of FDA warning letters, drug developers must ensure that patients not only meet inclusion and exclusion criteria, but also adhere to the protocol throughout the course of a clinical study. Ensuring protocol compliance is becoming increasingly difficult as protocols increase in complexity; however, real-time data play a significant role in helping drug developers quickly and easily detect proto- col deviators and violators, thereby managing the ultimate quality of a clinical trial. Protocol adherence can also be strengthened by investi- gating the capture of all scheduled data at each visit, by monitoring visit attendance according to protocol schedules, and by reviewing reasons for patient withdrawal. The quality of a study is also dependent upon the highest level of data accuracy. For ex- ample, if errors at any trial site go undetected, they can compromise the quality and integrity of the entire study. Real-time data play a criti- cal role in helping researchers achieve data ac- curacy from the outset, spotting missing data, identifying data errors, or highlighting poten- tially fraudulent data early. For example, if a laboratory assessment occurs without the completion of a corresponding clinical visit, then a missing page report can be generated in real time. Furthermore, real-time data allow for signal detection of quality issues long before issues become real problems. By using data to spot trends or early warning signs, drug developers can proactively investigate and correct poten- tial issues early—ensuring both corrective and preventive actions are taken. Quality signals in- clude edit checks firing more often than the targeted value, prompting additional training or coaching to the problematic investigator site. On the other hand, edit checks firing too infre- quently could suggest that a review of the edit check code is required. The use of data to detect potential fraud is well documented (7), and statistical tech- niques have been developed that may point to further investigation of specific investigator sites. Methods include: end digit preference where the last digit of a set of assessments has an unusual pattern; and detecting inliers (rath- er than outliers) through the investigation of multivariate data structures where fraudulent data often exhibit markedly less variability rel- ative to that observed elsewhere. Fraud re- mains an uncommon problem, but real-time data access enables checking programs to be put in place that encourage proactive trial management in this area. The use of real-time data allows for ongoing review of a site’s past site performance, the number of subjects and the rate of site recruit- ment, staff feedback on protocol compliance, site contact and record keeping, information received from data management, inaccurate or repetitive data, and other variables. Real-time access to each of these and other data points enable more timely site review triggers, which help to prevent the quality of the study from be- ing threatened. The ability to demonstrate that real-time ac- cess to data is being utilized for the purposes of improving the conduct of the trial is likely to be
  • 7. Real-time Data to Drive Better Decisions S t a t i s t i c s 501 Drug Information Journal well received by regulators. Real-time data ac- cess can also support and inform a sponsor’s quality plan. Accelerating Timelines Technology designed to capture and integrate accruing patient data has enabled the increas- ingly widespread use of adaptive clinical trials. Adaptive designs, which use a set of design rules to define a priori which modifications can be incorporated into the trial design, reduce the length of time required to complete a clinical study and have potential to accelerate the de- velopment of promising therapies. Use of adap- tive designs allows stopping decisions at the earliest possible time point, avoiding subjecting patients to noneffective or unsafe therapies. Adaptive designs involve interim analyses, which are planned at the outset and well con- trolled. It is understood that results must be handled carefully, with firewalls put in place to ensure that those with access to the data are not a part of the study team. Ongoing review of real-time data differs from adaptive trials in that access to data is more open and, to date, its use has been less well de- fined. Furthermore, adaptive trials typically re- quire access to treatment assignments whereas real-time data access does not. It is important to note, however, that one commonly used sam- ple size reestimation method for adaptive trials only utilizes the pooled estimate of the variance (or the probability of events) from the blinded aggregated data. But like adaptive trials, ongo- ing review of data can play an important role in the acceleration of a study’s timeline. For example, open access to blinded, real- time information makes faster data cleanup possible. Undetected errors in data capture can delay database lock or, worse, compromise the overall study. Errors are much more quickly de- tected with the use of real-time data than has traditionally been the case. This helps to ensure accuracy and encourages standardization of training and processes across trial sites, as well as quicker remediation when a problem is iden- tified. In addition, drug developers are able to con- duct a blinded dry run of the output process at various points throughout a study, which en- hances data cleaning and aids in finalizing data derivation rules and validating programming code. The manner in which data are organized in tables (of which there can be hundreds) can be reviewed at preplanned intervals by the glob- al clinical lead overseeing the trial and other members of the sponsor organization months in advance of the trial’s completion, thereby expe- diting data output at a study’s conclusion. It is imperative with dry runs to put in place protections against the misuse of data and to es- tablish at the outset exactly who has access to what information, when. By ensuring that the data under review are aggregated but un- grouped, or grouped randomly using dummy as- signments, those most concerned with the effi- cacy of the trial are only able to review table formatting and not mine the data in any way that would jeopardize the integrity of the trial. Finally, quotas for enrollment, which nearly 80% of trials fail to meet, are better managed with real-time data review (8). Access to real- time data offers researchers a timely look at the state of enrollment for the trial as a whole, and an opportunity to put a corrective action plan in place when the data indicate enrollment is slower than predicted. Conclusion As technology and tools better enable the cap- ture, integration, and evaluation of data in real time, drug developers will have new opportuni- ties to enhance the process and outcomes of clinical trials in the new health landscape. For drug developers interested in leveraging real- time data access, however, it is imperative to recognize that it must be done in a planned, controlled, and documented way to avoid risks associated with biasing a trial. It is important to consider formulating clear review policies, as well as the use of appro- priate data review tools that limit user access to data. The intent to review real-time data throughout the course of a trial should also be
  • 8. 502 S t a t i s t i c s Brown Stafford, Garrett clearly established at the outset within a clini- cal trial protocol or separate real-time data ac- cess protocol. Finally, a clear audit trail must be established so that regulators can easily de- termine how and when data were reviewed and by whom. The advantages of reviewing real-time data far outweigh the dangers and allow for earlier de- tection of patient safety issues, faster timelines, and higher-quality clinical trials. Acknowledgments—The authors would like to thank the following colleagues at Quintiles who reviewed and provided valuable comment on earlier drafts of the manuscript: Vladimir Dragalin, Thomas Grund- strom, Michael Fiola, David McGowan, Michael O’Kelly, and Lindsay Singler. The authors would also like to express thanks to Gary Koch for his valuable contribution. R eferences 1. Clinical Data Interchange Standards Consortium. http://www.cdisc.org/mission-and-principles. 2. Food and Drug Administration. Guidance for clin- ical trial sponsors: establishment and operation of clinical trial data monitoring committees, section 4.2.2. March 2006. http://www.fda.gov/OHRMS/ DOCKETS/98fr/01d-0489-gdl0003.pdf. 3. O’Neill R. Signal detection in clinical trials: some perspectives on new tools and processes—a criti- cal path update. 19th DIA Annual Euromeeting. Vienna, Austria, March 26–28, 2007. 4. Food and Drug Administration. Guidance for in- dustry: adaptive design clinical trials for drugs and biologics. February 2010. http://www.fda.gov/ downloads/Drugs/guidancecomplianceregula toryinformation/guidances/ucm201790.pdf. 5. Food and Drug Administration. Guidance for in- dustry: part 11: electronic records; electronic sig- natures—scope and application. August 2003. http://www.fda.gov/downloads/drugs/guidance complianceregulatoryinformation/guidances/ ucm070295.pdf. 6. Food and Drug Administration. Guidance for in- dustry: drug-induced liver injury: premarketing clinical evaluation. July 2009. http://www.fda .gov/downloads/Drugs/GuidanceCompliance RegulatoryInformation/Guidances/UCM174090 .pdf. 7. O’Kelly M. Using statistical techniques to detect fraud: a test case. Pharm Stat. 2004;3(4):237–246. http://onlinelibrary.wiley.com/doi/10.1002/ pst.137/abstract. 8. Hess J, Litalien S. Web-based patient recruitment: best opportunity to accelerate clinical trials. Cut- ting Edge Information, 2005. Paula Brown Stafford has disclosed that she is an employee of Quintiles, that she received honoraria from the UNC CTSA External Advisory Board, and that as an employee of Quintiles she currently undertakes, and in the past has undertaken, extensive work for the biopharmaceutical industry. Andrew Garrett has disclosed that he is an employee of Quintiles and that in this capacity he currently undertakes, and in the past has undertaken, ex- tensive work for the biopharmaceutical industry.