Clinical Study Modeling & Simulation
1 like150 views
This document describes how modeling and simulation (M&S) can be used to project outcomes for clinical trials. M&S involves building statistical models based on incoming patient data and then simulating the remainder of the study multiple times. This allows researchers to predict milestones, test alternative scenarios, and validate study assumptions. The document provides examples of how M&S was used to accurately forecast timelines and inform decisions for trials experiencing issues with enrollment rates and event rates differing from initial assumptions. Management found the simulations to be very valuable for planning by providing projections when other methods would have involved guessing.
Clinical Study Modeling & Simulation
- 1. SERC M&S: Examples (Screening, Enrollment, Randomization, Completion Modeling & Simulation) Dennis Sweitzer, Ph.D. April 2016
- 2. Application Scopes A priori Assumptions ⟶ Simulate⟶ Expected Outcomes, Thresholds ( e.g., planned timeline, resources, and expected variability) Ongoing study⟶ Model⟶ Simulate⟶ Projections ( e.g., projected timeline, resources, and expected variability given real information) Projections v. A priori Assumptions ⟶Validation (Consistency) ( e.g., are projections from incoming data consistent with assumptions) Projections v. Observations ⟶Validation (Reality) ( e.g., do projections from incoming data match planning expectations) Model + Scenarios ⟶ Simulate ⟶ Alterative Projections ✔ ✔ ✔ ✔ ✔ Using patient milestone dates (blinded) (SERC ≣ Screening, Enrollment, Randomization, Discontinuation) And/or Assumptions used in planning Simple Modeling & Simulation can be used: Modeling: Survival analysis of time between events Simulation: Competing Events model using survival results Examples ⟹
- 3. Example: Multi-‐Segment Studies Study Flowchart Randomized Treatment Phase 28 to 104 weeks Screening & Enrollment Open-Label Treatment Phase 12 to 36 weeks Active Placebo Inclusion/Exclusion Criteria Inclusion/Exclusion Criteria Screen Failure Drop Outs Drop Outs • Long term randomized withdrawal maintenance studies (AstraZeneca) • Open Label Stabilization (3-‐9mo) + Follow to Relapse (1-‐2yr) • Standard design, but not in Schizophrenia, bipolar, & other mood – ⟶ Uncertain dropout, relapse, & response rates • Risks of enrolling – Too few (subjects dropout before relapse)⟶ Failed Study – Too many (subjects in Open Label at last relapse)⟶ Costs, Ethics
- 4. Competing Events Model 1. Best guess for initial planning 2. As study was running, every month: • Update Statistical Model using patient status data • Simulate remainder of study from model 3. Summarize Simulations to: • Predict milestones (timelines, resources) • Test scenarios (of changes in plans) • Validate study assumptions & detect deviations Enroll OL Pts OL Dropouts Relapse Rand’d Patients Rand’d Dropouts
- 5. M&S ProjectionTrial B, Dates of 200th Event Predicted on 29 Oct by Enrollment Cutoff 12-Feb-06 23-May-06 31-Aug-06 9-Dec-06 19-Mar-07 27-Jun-07 5-Oct-07 13-Jan-08 22-Apr-08 31-Jul-08 10-Sep-0524-Sep-058-O ct-0522-O ct-055-Nov-0519-N ov-053-Dec-05 17-D ec-0531-D ec-0514-Jan-0628-Jan-0611-Feb-0625-Feb-0611-M ar-0625-M ar-068-Apr-0622-Apr-066-M ay-06 20-M ay-06 Enrollment Cutoffs Region Based Simulation Actual Projected End of Study, IF… … Enrollment ends on this date Reduced costs: stop enrollment on 3 Dec Reduced Risks: stop by 11 March
- 6. Maintenance Studies in 2005 Trial A, Predicted Dates of 200th Event 22-Feb-06 8-Mar-06 22-Mar-06 5-Apr-06 19-Apr-06 3-May-06 17-May-06 31-May-06 14-Jun-06 28-Jun-06 12-Jul-06 26-Jul-06 9-Aug-06 23-Aug-06 6-Sep-06 20-Sep-06 4-Oct-06 9-O ct-05 23-O ct-05 6-N ov-05 20-N ov-05 4-D ec-05 18-D ec-05 1-Jan-06 15-Jan-06 29-Jan-06 12-Feb-06 26-Feb-06 12-M ar-06 26-M ar-06 9-Apr-06 23-Apr-06 Date of Prediction (Oct 1 Enrollment Cutoff) PredictedDateof200thEvent Region Based Model (Median) Trial Based Actual Stop enrolling Stop Randomizing Wait as Patients Relapse or Drop out
- 7. Another Case Study Management feedback: “… the simulations are very valuable and the only way we have to plan our timelines. As it has turned out, your simulations seems to be pretty accurate ...” ... We would have been guessing and spinning our wheels without them.” Date # Randomized Relapses / Dropouts Prediction: 101st Relapse 3 Aug’06 73 3 / 2 1 Dec … 15 June 6 Sep’06 182 16 / 7 12 Nov … 21 Feb 2 Oct’06 Stopped Enrolling Patients (NB: 3-‐4 month open label) Dec‘06 Stopped Randomizing Patients (All eligible or discontinued) 1 Jan’07 101st Relapse Event
- 8. Examples Validation: Protocols A&B assumed: (50% randomized, 30% Relapse) rate Models estimated: Trial A: (33%, 37%) Trial B: (55%, 41%) Early Issue Identification ⟶ Quick Corrections Scenario: ¿Add Sites to compensate for low enrollment? • Run simulation with additional sites • Compare between simulations Scenario: EMEA requested secondary endpoint of Late Relapses (>4wk off Tx), Trial A had stopped enrolling. Should Trial A be reopened? Should Trial B be extended? • Build new endpoint into simulations • Report
- 9. More A presentation I gave at JSM 2006 on the method, with a proceedings paper. https://sites.google.com/site/dennissweitzer/home/modeling-‐multiphase-‐clinical-‐trials-‐time-‐to-‐completion-‐ study-‐management Simple simulation methods using Excel. I’ve long used Excel simulations to aid in planning clinical trials (for quick & transparent models), although methods for doing so are not well publicized. Here’s a presentation of how-‐to: https://sites.google.com/site/dennissweitzer/home/quick-‐simple-‐simulation-‐using-‐ms-‐excel