![Brief History…
• “1997: Registra)on required for selec)ve trials
• 1999: Registry created (ClinicalTrials.gov)
• Early 2000s… Pa)ent advocacy for access to trial informa)on
• 2007: Registra)on/repor)ng requirements expanded
• Ethical Principles as outlined in Belmont Report [1978]
• 1. Respect for persons: protec)ng the autonomy of all people;
Researchers must be truthful and conduct no decep)on;
• 2. Beneficence: "Do no harm" while maximizing benefits for the
research project and minimizing risks to the subjects
• 3. Jus)ce: the fair distribu)on of costs and benefits”
2016-10-17 | UC Berkeley
Alasdair Cohen | Lecture for Publich Health 250B
26
Source: Maya Petersen, UCB SPH, 2016-06 BITSS
presenta)on “Pre-registra)on and pre-analysis
plans: Perspec)ves from biomedical research”
“On September 30,
1978, the Na)onal
Commission for the
Protec)on of
Human Subjects of
Biomedical and
Behavioral
Research submiked
its report en)tled
“The Belmont
Report: Ethical
Principles and
Guidelines for the
Protec)on of
Human Subjects of
Research.”
hkp://www2.umf.maine.edu/irb/
other-links/the-belmont-report/](https://image.slidesharecdn.com/transparency3-161015065618/85/Transparency3-3-320.jpg)
Transparency3
- 1. 2016-10-17 | UC Berkeley Alasdair Cohen | Lecture for Publich Health 250B 24 Protocol Registra(on & Pre-specified Analysis Plans
- 3. Brief History… • “1997: Registra)on required for selec)ve trials • 1999: Registry created (ClinicalTrials.gov) • Early 2000s… Pa)ent advocacy for access to trial informa)on • 2007: Registra)on/repor)ng requirements expanded • Ethical Principles as outlined in Belmont Report [1978] • 1. Respect for persons: protec)ng the autonomy of all people; Researchers must be truthful and conduct no decep)on; • 2. Beneficence: "Do no harm" while maximizing benefits for the research project and minimizing risks to the subjects • 3. Jus)ce: the fair distribu)on of costs and benefits” 2016-10-17 | UC Berkeley Alasdair Cohen | Lecture for Publich Health 250B 26 Source: Maya Petersen, UCB SPH, 2016-06 BITSS presenta)on “Pre-registra)on and pre-analysis plans: Perspec)ves from biomedical research” “On September 30, 1978, the Na)onal Commission for the Protec)on of Human Subjects of Biomedical and Behavioral Research submiked its report en)tled “The Belmont Report: Ethical Principles and Guidelines for the Protec)on of Human Subjects of Research.” hkp://www2.umf.maine.edu/irb/ other-links/the-belmont-report/
- 6. Where to Register? 2016-10-17 | UC Berkeley Alasdair Cohen | Lecture for Publich Health 250B 29 Study Type OrganizaNon/Registrar URL Randomized trials ClinicalTrials.gov hkp://clinicaltrials.gov InternaNonal clinical trials World Health Organiza)on ICTRP www.who.int/ictrp InternaNonal impact evaluaNons 3ie RIDIE - Registry for Interna)onal Development Impact Evalua)ons hkp://ridie. 3ieimpact.org Interna)onal Development Coordina)ng Group hkp:// www.campbellcollabora )on.org/ SystemaNc Reviews PROSPERO - Interna)onal prospec)ve register of systema)c reviews hkp:// www.crd.york.ac.uk/ PROSPERO/#index.php Any type of study Open Science Framework hkp://osf.io Social Science, other… AEA RCT Registry (for social science) AsPredicted (registra)on “simplified”) hkps:// www.socialscienceregist ry.org, hkps://aspredicted.org
- 10. Open Science Framework 2016-10-17 | UC Berkeley Alasdair Cohen | Lecture for Publich Health 250B 33
- 14. Pre-specified Analysis Plans: Pros/Cons • “On the one hand…without pre-specifica)on -> bias and misleading inference • ‘protocols need to be en)rely transparent and their analysis plans explicit in detail upfront. There should be no room for flexibility in the collected data and performed analyses.’ • Ioannidis, Philos Ethics Humanit Med 2008 • On the other hand…Op)mal analysis ojen requires flexibility • Ex: selec)on of adjustment covariates to op)mize power • Ex: Specifica)on of model used to adjust for informa)ve missingness/ loss to follow up 2016-10-17 | UC Berkeley Alasdair Cohen | Lecture for Publich Health 250B 37 Source: Maya Petersen, UCB SPH, 2016-06 BITSS presenta)on “Pre-registra)on and pre-analysis plans: Perspec)ves from biomedical research”
- 15. Pre-specified Analysis Plans in Prac(ce 2016-10-17 | UC Berkeley Alasdair Cohen | Lecture for Publich Health 250B 38 81% 31% “Flow diagram for 199 evaluated trials: availability of information on model adjustment for primary outcomes from trial registry, design paper, or full protocol provided by trial authors”
- 16. Pre-specified Analysis Plans: What about Observa(onal Studies? • “Even with a pre-specified hypothesis, observa)onal analyses ojen entail many more analy)c decisions • Iden)fica)on strategy • Difference in difference, adjustment for measured confounders, IV, etc. • Es)mator • Outcome regression methods, propensity score matching/adjustment/reweigh)ng, etc. • Model specifica)on • Which adjustment variables to include in outcome regression, func)onal form, etc. • And what about exploratory analyses, hypothesis genera)on, unexpected findings…?” 2016-10-17 | UC Berkeley Alasdair Cohen | Lecture for Publich Health 250B 39 Source: Maya Petersen, UCB SPH, 2016-06 BITSS presenta)on “Pre-registra)on and pre-analysis plans: Perspec)ves from biomedical research”
- 17. Register/pre-specify Observa(onal Studies: YES • “Same ra)onale as randomized trials • Ethics • Knowledge dissemina)on/avoidance of unnecessary duplica)on • Guard against publica)on bias • Ideally detailed analysis plans would also be registered • Likle burden • Observa)onal studies need IRB approval • Register the protocol • Can incorporate flexibility • Register changes to protocol • Delineate between pre-specified and post-hoc hypotheses” 2016-10-17 | UC Berkeley Alasdair Cohen | Lecture for Publich Health 250B 40 Source: Maya Petersen, UCB SPH, 2016-06 BITSS presenta)on “Pre- registra)on and pre-analysis plans: Perspec)ves from biomedical research”
- 18. Register/pre-specify Observa(onal Studies: NO • “We will test many fewer hypotheses • Reduce new and unexpected findings • We may test them less rigorously • Pre-specified analyses may give us less valid hypothesis tests • “Protocol adapta)ons can improve recruitment, allow more accurate measurement of study variables, implement alterna)ve analyses to control confounding, and incorporate new knowledge published by others.” (Lash, Epidemiology 2010) • We will learn more slowly • Simply allowing for post-hoc analyses designated as such is not sufficient” 2016-10-17 | UC Berkeley Alasdair Cohen | Lecture for Publich Health 250B 41 Source: Maya Petersen, UCB SPH, 2016-06 BITSS presenta)on “Pre- registra)on and pre-analysis plans: Perspec)ves from biomedical research”
- 19. Pre-specified Analysis Plans: SOP • Standard Opera)ng Procedures • Can be referenced for issues not explicitly addressed in your pre-specified analysis plan • Commonly used reference/resource: Don Green’s SOP (Columbia Uni.) 2016-10-17 | UC Berkeley Alasdair Cohen | Lecture for Publich Health 250B 42 hkps://github.com/ acoppock/Green-Lab- SOP
- 20. Registra(on & Pre- specified Analysis: Takeaways • Register the study protocol before data collec)on begins • IRB must be submiked and approved before data collec)on, so can usually use/shorten the IRB protocol for registra)on • Core parts of analysis plans can/should be addressed in protocol • Primary and secondary outcomes • Ideally, if possible, register pre-specified analysis plan before study begins • Or, can register it ajer study ini)a)on (but before data collec)on is finished) 2016-10-17 | UC Berkeley Alasdair Cohen | Lecture for Publich Health 250B 43
- 21. Registra(on & Pre- specified Analysis: Takeaways • Yes, register observa)onal studies: Protocol and analysis plans • Can specify the extent to which analysis will be exploratory in nature • Cu{ng-edge: Pre-specified adap4ve analysis plans • Targeted learning (machine learning with sta)s)cal inference). • For example: BALZER, L. B., VAN DER LAAN, M. J. & PETERSEN, M. L. 2015. Adap)ve Pre-specifica)on in Randomized Trials With and Without Pair-Matching 2016-10-17 | UC Berkeley Alasdair Cohen | Lecture for Publich Health 250B 44
- 22. Pre-analysis Plan Template TOC Courtesy of Maya Petersen 2016-10-17 | UC Berkeley Alasdair Cohen | Lecture for Publich Health 250B 45 2 Table of Contents 1 How to Use This Template 1 2 Table of Contents 2 3 Abbrevia)ons and Defini)ons 4 4 Introduc)on 4 4.1 Preface 4 4.2 Purpose of the analyses 4 5 Study Objec)ves and Endpoints 5 5.1 Study Objec)ves 5 5.2 Endpoints 5 5.3 Derived variables 5 6 Study Methods 5 6.1 General Study Design and Plan 5 6.2 Equivalence or Non-Inferiority Studies 6 6.3 Inclusion-Exclusion Criteria and General Study Popula)on 7 6.4 Randomisa)on and Blinding 7 6.5 Study Variables 7 7 Sample Size 9 8 General Considera)ons 10 8.1 Timing of Analyses 10 8.2 Analysis Popula)ons 10 8.2.1 Full Analysis Popula)on 10 8.2.2 Per Protocol Popula)on 11 8.2.3 Safety Popula)on 11 8.3 Covariates and Subgroups 11 8.4 Missing Data 12 8.5 Interim Analyses and Data Monitoring 13 8.5.1 Purpose of Interim Analyses 13 8.5.2 Planned Schedule of Interim Analyses 13 8.5.3 Scope of Adapta)ons 13 8.5.4 Stopping Rules 13 8.5.5 Analysis Methods to Minimise Bias 14 8.5.6 Adjustment of Confidence Intervals and p-values 14 8.5.7 Interim Analysis for Sample Size Adjustment 14 8.5.8 Prac)cal Measures to Minimise Bias 15 8.5.9 Documenta)on of Interim Analyses 15 8.6 Mul)-centre Studies 15 8.7 Mul)ple Tes)ng 16 9 Summary of Study Data 17 9.1 Subject Disposi)on 17 9.2 Protocol Devia)ons 18 9.3 Demographic and Baseline Variables 18 9.4 Concurrent Illnesses and Medical Condi)ons 18 9.5 Prior and Concurrent Medica)ons 19 9.6 Treatment Compliance 19 10 Efficacy Analyses 19 10.1 Primary Efficacy Analysis 21 10.2 Secondary Efficacy Analyses 21 10.3 Exploratory Efficacy Analyses 21 11 Safety Analyses 21 11.1 Extent of Exposure 22 11.2 Adverse Events 22 11.3 Deaths, Serious Adverse Events and other Significant Adverse Events 22 11.4 Pregnancies 23 11.5 Clinical Laboratory Evalua)ons 23 11.6 Other Safety Measures 23 12 Pharmacokine)cs 23 13 Other Analyses 24 14 Figures 24 15 Repor)ng Conven)ons 24 16 Technical Details 25 17 Summary of Changes to the Protocol 25 18 References 25 19 Lis)ng of Tables, Lis)ngs and Figures 27