Innovative sample size methods for adaptive clinical trials webinar web version (0.1)
Download as PPTX, PDF1 like434 views
The document discusses adaptive trial designs, specifically focusing on sample size re-estimation (SSR) and its implications for clinical trials. It highlights the advantages and disadvantages of adaptive trials, the regulatory background supporting them, and the methodology behind blinded and unblinded SSR. The document also outlines recent updates to sample size calculation tools, emphasizing their widespread acceptance and expected growth in use.
![Two Sample Proportion BSSR
Example
“For our sample size
calculations, we assumed that
20% of women in the control arm
would be using LARC after their
6-week postpartum visit, … An
analysis population with at least
626 women (313 in each arm)
was required to provide 80%
power (using a two-sided alpha
of 0.05) to detect an absolute
10% increase to 30% in LARC use
in the intervention arm [14].
Anticipating a maximum drop-
out rate of 20% at the time of
Follow-Up Survey #2, we planned
to randomize 800 participants”
Source: Contraception
Parameter Value
Significance Level (2-
Sided)
0.05
Control Rate 0.2
Intervention Rate 0.3
n per Group 313
Target Power (%) 80%
Nuisance Parameter Overall Success
Rate](https://image.slidesharecdn.com/innovativesamplesizemethodsforadaptiveclinicaltrialswebinar-webversion0-180926141725/85/Innovative-sample-size-methods-for-adaptive-clinical-trials-webinar-web-version-0-1-16-320.jpg)
Innovative sample size methods for adaptive clinical trials webinar web version (0.1)
- 2. Head of Statistics nQuery Lead Researcher FDA Guest Speaker Guest Lecturer Webinar Host HOSTED BY: Ronan Fitzpatrick
- 3. Webinar Overview Adaptive Designs and Sample Size Re-estimation (SSR) Blinded Sample Size Re-estimation GSD, Conditional Power & Unblinded SSR Discussion and Conclusions
- 4. Worked Examples Overview Blinded Sample Size Re- estimation Two Means Group Sequential Conditional Power Unblinded SSR Example WORKED EXAMPLES
- 5. In 2017, 90% of organizations with clinical trials approved by the FDA used nQuery for sample size and power calculation
- 6. PART I SSD & Adaptive Design
- 7. Context SSD finds the appropriate sample size for your study Common metrics are statistical power, interval width or cost SSD seeks to balance ethical and practical issues Crucial to arrive at valid conclusions, Type M/S errors High cost of failed clinical trials drug development
- 8. Adaptive Trials Overview Adaptive Trials are any trial where a change or decision is made to a trial while still on- going Encompasses a wide variety of potential adaptions E.g. Early stopping, SSR, enrichment, seamless, dose-finding Adaptive trials seek to give control to trialist to improve trial based on all available information
- 9. Adaptive Trials Pros & Cons Advantages 1. Earlier Decisions 2. Reduced Potential Cost 3. Higher Potential Success 4. Greater Control 5. Better Seamless Designs Disadvantages 1. More Complex 2. Logistical Issues (IDMC) 3. Modified Test Statistics 4. Greater Expertise 5. Regulatory Approval?
- 10. Adaptive Trials Regulatory Background Draft FDA CBER/CDER Guidance published in 2010 “Well-understood” and “Less well-understood” Designs EMA published similar reflection paper (2007) Increase in interest in encouraging adaptive design US: Innovative Cures Act, EU: Adaptive Pathways New FDA Guidance expected later this year Will likely to see proliferation of new designs
- 11. Sample Size Re-estimation (SSR) Will focus here on specific adaptive design of SSR Adaptive Trial focused on higher sample size if needed Strong adaption target due to intrinsic SSD uncertainty Note that more suited to knowable/short follow- up Note that could lower sample size but not encouraged
- 13. Blinded Sample Size Re- estimation BSSR uses interim blinded nuisance parameter estimate Use of blinded data reduces logistical/regulatory issues Considered a “well understood” type of adaptive design Multiple methods but focus on internal pilot approach Update N based on parameter estimate from internal pilot Use same methods as fixed term trial incl. pilot
- 14. Blinded SSR nQuery Summary (Autumn 2018) Blinded SSR Means SSR Criteria: Variance Three σ2 Estimate Methods 1. Two Sample Inequality 2. Two Sample NI 3. Two Sample Equiv Blinded SSR Props SSR Criteria: Overall Success Rate Assumes effect size true 1. Two Sample Inequality 2. Two Sample NI
- 15. Two Sample Mean Blinded SSR Example Source: nejm.com Parameter Value Significance Level (2- Sided) 0.05 Mean Difference (%) -9 Standard Deviation (%) 16 Dropout Rate 15% Target Power 90% Nuisance Parameter? Standard Deviation “We estimated that we would need to enrol 160 patients, given an expected mean (±SD) annual decline in the FVC of 9±16 percent of the predicted value and a dropout rate of 15 percent, to achieve a two-sided alpha level of 0.05 and a statistical power of 90
- 16. Two Sample Proportion BSSR Example “For our sample size calculations, we assumed that 20% of women in the control arm would be using LARC after their 6-week postpartum visit, … An analysis population with at least 626 women (313 in each arm) was required to provide 80% power (using a two-sided alpha of 0.05) to detect an absolute 10% increase to 30% in LARC use in the intervention arm [14]. Anticipating a maximum drop- out rate of 20% at the time of Follow-Up Survey #2, we planned to randomize 800 participants” Source: Contraception Parameter Value Significance Level (2- Sided) 0.05 Control Rate 0.2 Intervention Rate 0.3 n per Group 313 Target Power (%) 80% Nuisance Parameter Overall Success Rate
- 17. GSD, Conditional Power & Unblinded SSR PART III
- 18. Group Sequential Designs (GSD) GSD facilitate interim analyses Interim analyses occur while trial on-going Accrued data analysed at pre- specified times E.g. After 1/2 subjects have been measured Can stop for benefit or futility If neither found, continue trial until end/next interim Need to account for effect of multiple analyses Do this by “spending” α and/or β errors GSD Changes 1. Futility Only Designs 2. Additional Outputs 3. New Two Sample TTE 4. One Sample Mean GSD 5. One Sample Prop GSD
- 19. Error Spending (Lan & DeMets) Two Criteria for early stopping 1. Efficacy (α-spending) 2. Futility (β-spending) Multiple Error Spending Functions O’Brien Fleming, Pocock etc. Both α and β spending work similarly Can be very liberal or conservative At each interim analysis, spending a proportion of the total error Makes analysis at endpoint more conservative 𝛼 𝜏 = 2 1 − Φ 𝑧 𝛼/2 𝜏
- 20. Group Sequential Example “A sample size of 242 subjects (121 per treatment group) provides at least 80% power to detect a relative difference of 53% between botulinum toxin A and standardized anticholinergic therapy, assuming a treatment difference of -0.80 and a common SD of 2.1 (effect size = 0.381), and a two-sided type I error rate of 5%. Sample size has been adjusted to allow for a 10% loss to follow-up over the 6- months of treatment as well Parameter Value Significance Level (2- sided) 0.05 OnabotulinumtoxinA Mean -2.3 Anticholinergic Mean -1.5 Standard Deviation (Both) 2.1 Power 80% # Interim Analyses 1 α Spending Function O’Brien- Source: NEJM (2012)
- 21. Conditional Power (CP) CP gives prob. of rejecting null given interim test statistic Calculation still depends on what “true” difference set to Often used as ad-hoc criteria for futility testing in GSD More flexible than β-spending but less error guarantee Focus here on CP as measure of “promising” results “Promising” meaning less than target but close to target power
- 22. Conditional Power & Unblinded SSR Most common criteria proposed for unblinded SSR is CP SSR suggested when interim results “promising” (Chen et al) Gives third option vs GSD: continue, stop early, increase N “Promising” user-defined but based on unblinded effect size Power for optimistic effect but increase N for lower relevant effects? 2 methods here: Chen, DeMets & Lan; Cui, Hung & Wang 1st uses GSD statistics but only penultimate look & high CP nd
- 25. Discussion and Conclusions Adaptive Trials expected to become more common Reduction of costs, greater regulatory interest etc. SSR will be one common type of adaptive trial Blinded SSR already widely accepted, unblinded growing Blinded SSR targets initial variance under- estimates
- 26. nQuery Spring 2018 Update Initial release focused on Survival & Bayesian tables. April release adds 72 new tables in following areas: New Bayes tables in April update New tables in April update Epidemiology Non-inferiority/ Equivalence Correlation/ROC Bayesian Sample Size 52 20
- 27. nQuery Autumn 2018 Update Autumn 2018 release adds nQuery Adapt module, 32 new tables & undo/redo New Core Tables Proportions + Crossover Assurance Conditional Power GST + SSR 20 nQuery Bayes Tables 12 nQuery Adapt Tables 15
- 28. Q&A Any Questions? For further details, contact at: info@statsols.com Thanks for listening!
- 29. References Friede, T., & Kieser, M. (2006). Sample size recalculation in internal pilot study designs: a review. Biometrical Journal: Journal of Mathematical Methods in Biosciences, 48(4), 537-555. Tang, J. H., Dominik, R. C., Zerden, M. L., Verbiest, S. B., Brody, S. C., & Stuart, G. S. (2014). Effect of an educational script on postpartum contraceptive use: a randomized controlled trial. Contraception, 90(2), 162-167. Tashkin, D. P., Elashoff, R., Clements, P. J., Goldin, J., Roth, M. D., Furst, D. E., ... & Seibold, J. R. (2006). Cyclophosphamide versus placebo in scleroderma lung disease. New England Journal of Medicine, 354(25), 2655-2666. Jennison, C., & Turnbull, B. W. (1999). Group sequential methods with applications to clinical trials. CRC Press. Visco, A. G., et al (2012). Anticholinergic therapy vs. onabotulinumtoxina for urgency urinary incontinence. New England Journal of Medicine, 367(19), 1803-1813. Chen, Y. J., DeMets, D. L., & Gordon Lan, K. K. (2004). Increasing the sample size when the unblinded interim result is promising. Statistics in medicine, 23(7), 1023-1038. Cui, L., Hung, H. J., & Wang, S. J. (1999). Modification of sample size in group sequential clinical trials. Biometrics, 55(3), 853-857.
Editor's Notes
- #8: Point 1: http://rsos.royalsocietypublishing.org/content/1/3/140216 -> Screening problem analogy. Type S Error = Sign Error i.e. sign of estimate is different than actual population value Type M Error = Magnitude Error i.e. estimate is order of magnitude different than actual value Point 2: Know we have only 100 subjects available. Need to know what power will this give us, i.e. is there enough power to justify even doing the study. Stage III clinical trials constitute 90% of trial costs, vital to reduce waste and ensure can fulfil goal. Point 3: Sample Size requirements described in ICH Efficacy Guidelines 9: STATISTICAL PRINCIPLES FOR CLINICAL TRIALS See FDA/NIH draft protocol template here: http://osp.od.nih.gov/sites/default/files/Protocol_Template_05Feb2016_508.pdf (Section 10.5) Nature Statistical Checklist: http://www.nature.com/nature/authors/gta/Statistical_checklist.doc Point 4: In Cohen’s (1962) seminal power analysis of the journal of Abnormal and Social Psychology he concluded that over half of the published studies were insufficiently powered to result in statistical significance for the main hypothesis. Many journals (e.g. Nature) now require that authors submit power estimates for their studies. Power/Sample size one of areas highlighted when discussing “crisis of reproducibility” (Ioannidis). Relatively easy fix compared to finding p-hacking etc.
- #9: Point 1: http://rsos.royalsocietypublishing.org/content/1/3/140216 -> Screening problem analogy. Type S Error = Sign Error i.e. sign of estimate is different than actual population value Type M Error = Magnitude Error i.e. estimate is order of magnitude different than actual value Point 2: Know we have only 100 subjects available. Need to know what power will this give us, i.e. is there enough power to justify even doing the study. Stage III clinical trials constitute 90% of trial costs, vital to reduce waste and ensure can fulfil goal. Point 3: Sample Size requirements described in ICH Efficacy Guidelines 9: STATISTICAL PRINCIPLES FOR CLINICAL TRIALS See FDA/NIH draft protocol template here: http://osp.od.nih.gov/sites/default/files/Protocol_Template_05Feb2016_508.pdf (Section 10.5) Nature Statistical Checklist: http://www.nature.com/nature/authors/gta/Statistical_checklist.doc Point 4: In Cohen’s (1962) seminal power analysis of the journal of Abnormal and Social Psychology he concluded that over half of the published studies were insufficiently powered to result in statistical significance for the main hypothesis. Many journals (e.g. Nature) now require that authors submit power estimates for their studies. Power/Sample size one of areas highlighted when discussing “crisis of reproducibility” (Ioannidis). Relatively easy fix compared to finding p-hacking etc.
- #11: Point 1: http://rsos.royalsocietypublishing.org/content/1/3/140216 -> Screening problem analogy. Type S Error = Sign Error i.e. sign of estimate is different than actual population value Type M Error = Magnitude Error i.e. estimate is order of magnitude different than actual value Point 2: Know we have only 100 subjects available. Need to know what power will this give us, i.e. is there enough power to justify even doing the study. Stage III clinical trials constitute 90% of trial costs, vital to reduce waste and ensure can fulfil goal. Point 3: Sample Size requirements described in ICH Efficacy Guidelines 9: STATISTICAL PRINCIPLES FOR CLINICAL TRIALS See FDA/NIH draft protocol template here: http://osp.od.nih.gov/sites/default/files/Protocol_Template_05Feb2016_508.pdf (Section 10.5) Nature Statistical Checklist: http://www.nature.com/nature/authors/gta/Statistical_checklist.doc Point 4: In Cohen’s (1962) seminal power analysis of the journal of Abnormal and Social Psychology he concluded that over half of the published studies were insufficiently powered to result in statistical significance for the main hypothesis. Many journals (e.g. Nature) now require that authors submit power estimates for their studies. Power/Sample size one of areas highlighted when discussing “crisis of reproducibility” (Ioannidis). Relatively easy fix compared to finding p-hacking etc.
- #12: Point 1: http://rsos.royalsocietypublishing.org/content/1/3/140216 -> Screening problem analogy. Type S Error = Sign Error i.e. sign of estimate is different than actual population value Type M Error = Magnitude Error i.e. estimate is order of magnitude different than actual value Point 2: Know we have only 100 subjects available. Need to know what power will this give us, i.e. is there enough power to justify even doing the study. Stage III clinical trials constitute 90% of trial costs, vital to reduce waste and ensure can fulfil goal. Point 3: Sample Size requirements described in ICH Efficacy Guidelines 9: STATISTICAL PRINCIPLES FOR CLINICAL TRIALS See FDA/NIH draft protocol template here: http://osp.od.nih.gov/sites/default/files/Protocol_Template_05Feb2016_508.pdf (Section 10.5) Nature Statistical Checklist: http://www.nature.com/nature/authors/gta/Statistical_checklist.doc Point 4: In Cohen’s (1962) seminal power analysis of the journal of Abnormal and Social Psychology he concluded that over half of the published studies were insufficiently powered to result in statistical significance for the main hypothesis. Many journals (e.g. Nature) now require that authors submit power estimates for their studies. Power/Sample size one of areas highlighted when discussing “crisis of reproducibility” (Ioannidis). Relatively easy fix compared to finding p-hacking etc.
- #14: Point 1: http://rsos.royalsocietypublishing.org/content/1/3/140216 -> Screening problem analogy. Type S Error = Sign Error i.e. sign of estimate is different than actual population value Type M Error = Magnitude Error i.e. estimate is order of magnitude different than actual value Point 2: Know we have only 100 subjects available. Need to know what power will this give us, i.e. is there enough power to justify even doing the study. Stage III clinical trials constitute 90% of trial costs, vital to reduce waste and ensure can fulfil goal. Point 3: Sample Size requirements described in ICH Efficacy Guidelines 9: STATISTICAL PRINCIPLES FOR CLINICAL TRIALS See FDA/NIH draft protocol template here: http://osp.od.nih.gov/sites/default/files/Protocol_Template_05Feb2016_508.pdf (Section 10.5) Nature Statistical Checklist: http://www.nature.com/nature/authors/gta/Statistical_checklist.doc Point 4: In Cohen’s (1962) seminal power analysis of the journal of Abnormal and Social Psychology he concluded that over half of the published studies were insufficiently powered to result in statistical significance for the main hypothesis. Many journals (e.g. Nature) now require that authors submit power estimates for their studies. Power/Sample size one of areas highlighted when discussing “crisis of reproducibility” (Ioannidis). Relatively easy fix compared to finding p-hacking etc.
- #19: Group sequential trials differ from the fixed period trials in that the data from the trial is analysed at one or more stages prior to the conclusion of the trial. As a result the alpha value applied at each analysis or ‘look’ must be adjusted to preserve the overall Type 1 error. So, in effect you are ‘spending’ some of your alpha at each ‘look’. The alpha values used at each look are calculated based upon the spending function chosen, the number of looks to be taken during the course of the trial as well as the overall Type 1 error rate.
- #20: Multiple looks = multiple chance to find significance. Need to have adjustment for that. Note that beta-spending actually decreases chance of finding significance (since futility stops future alpha tests) and thus actually inflates critical p-value. See futility example only.
- #21: Actual mean values taken from elsewhere in paper.
- #22: Group sequential trials differ from the fixed period trials in that the data from the trial is analysed at one or more stages prior to the conclusion of the trial. As a result the alpha value applied at each analysis or ‘look’ must be adjusted to preserve the overall Type 1 error. So, in effect you are ‘spending’ some of your alpha at each ‘look’. The alpha values used at each look are calculated based upon the spending function chosen, the number of looks to be taken during the course of the trial as well as the overall Type 1 error rate.
- #23: Group sequential trials differ from the fixed period trials in that the data from the trial is analysed at one or more stages prior to the conclusion of the trial. As a result the alpha value applied at each analysis or ‘look’ must be adjusted to preserve the overall Type 1 error. So, in effect you are ‘spending’ some of your alpha at each ‘look’. The alpha values used at each look are calculated based upon the spending function chosen, the number of looks to be taken during the course of the trial as well as the overall Type 1 error rate.
- #26: Point 1: http://rsos.royalsocietypublishing.org/content/1/3/140216 -> Screening problem analogy. Type S Error = Sign Error i.e. sign of estimate is different than actual population value Type M Error = Magnitude Error i.e. estimate is order of magnitude different than actual value Point 2: Know we have only 100 subjects available. Need to know what power will this give us, i.e. is there enough power to justify even doing the study. Stage III clinical trials constitute 90% of trial costs, vital to reduce waste and ensure can fulfil goal. Point 3: Sample Size requirements described in ICH Efficacy Guidelines 9: STATISTICAL PRINCIPLES FOR CLINICAL TRIALS See FDA/NIH draft protocol template here: http://osp.od.nih.gov/sites/default/files/Protocol_Template_05Feb2016_508.pdf (Section 10.5) Nature Statistical Checklist: http://www.nature.com/nature/authors/gta/Statistical_checklist.doc Point 4: In Cohen’s (1962) seminal power analysis of the journal of Abnormal and Social Psychology he concluded that over half of the published studies were insufficiently powered to result in statistical significance for the main hypothesis. Many journals (e.g. Nature) now require that authors submit power estimates for their studies. Power/Sample size one of areas highlighted when discussing “crisis of reproducibility” (Ioannidis). Relatively easy fix compared to finding p-hacking etc.