Understanding Statistical Power for Non-Statisticians
Download as PPTX, PDF1 like1,140 views
The webinar on statistical power discussed its importance in clinical trials, emphasizing how sample size and assumptions about differences in means affect the power of a study. It provided insights on the statistical significance of results based on different power levels, highlighting the necessary sample sizes for achieving desired power outcomes. Attendees learned practical implications for study design, including how varying factors such as standard deviation and alpha levels can impact both power and required sample sizes.
Understanding Statistical Power for Non-Statisticians
- 1. Understanding Statistical Power for Non-Statisticians WEBINAR | JUNE 28, 2016 | 12:00PM EASTERN | DIA 2016 DALE W. USNER, PH.D. | PRESIDENT | SDC
- 2. General Housekeeping Use mic & speakers or call in Webinar users submit questions via chat
- 3. Dale W. Usner, Ph.D. 20 years in industry 50+ FDA and international regulatory body interactions Frequent study design support Therapeutic Expertise Anti-viral/Anti-infective, Cardiovascular, Gastrointestinal, Oncology/Immunology, Ophthalmology, Surgical, Other
- 4. Agenda What is Statistical Power How Assumptions Affect Statistical Power and Sample Size How Power Is Associated With What is Statistically Significant Q&A
- 6. What is Statistical Power? Clinical/Medical: If I want to compare our Test Product to the Control Product in systolic blood pressure, what sample size will I need? Statistician: Assuming a Difference in Means of 10 mm Hg and a common Standard Deviation (SD) of 20 mm Hg, 64 subjects per treatment group are required to have 80% power for a 2-sided = 0.05 test.
- 7. What is Statistical Power? Next?
- 8. What is Statistical Power? Questions What does it mean to have 80% power?
- 9. What is Statistical Power? Questions What does it mean to have 80% power? What does it mean to assume a difference in means of 10 mm Hg? What if the true diff <10? >10?
- 10. What is Statistical Power? Questions What does it mean to have 80% power? What does it mean to assume a difference in means of 10 mm Hg? What if the true diff <10? >10? What role does the SD play? What if the true SD is >20? What if the true SD is <20?
- 11. What is Statistical Power? Questions What does it mean to have 80% power? What does it mean to assume a difference in means of 10 mm Hg? What if the true diff <10? >10? What role does the SD play? What if the true SD is >20? What if the true SD is <20? What happens if the difference observed in the study is <10 mm Hg?
- 12. Motivation Assuming a Difference in Means of 10 mm Hg and a common SD of 20 mm Hg, N = 64 subjects per treatment group are required to have 80% power for a 2-sided = 0.05 test. The study will demonstrate a statistically significant result if the difference observed in the study is ≥7.0 mm Hg (assuming the observed SD is 20 mm Hg).
- 13. Motivation With 85% Power (N = 73 subjects / Tx Gp) observed differences ≥6.55 mm Hg would yield statistical significance With 90% Power (N = 86 subjects / Tx Gp) observed differences ≥6.03 mm Hg would be statistical significance
- 14. Background
- 15. Statistical Inference Statistical Inference: Drawing conclusions about an entire population based on a sample from that population.
- 16. Hypotheses (Efficacy) Superiority H0: Test Arm is no different from Control Arm H1: Test Arm is different (superior) than Control Arm Non-Inferiority H0: Test Arm is inferior to Control Arm H1: Test Arm is non-inferior to Control Arm Desired Outcome: Reject H0 in favor of H1
- 17. Statistical Inference: Coin Flip H0: Proportion of Heads = Prop Tails = 0.50 H1: Proportion of Heads > Prop Tails Flip a coin 4 times with result 3H and 1T. 75% Heads, should H0 be rejected?
- 18. Statistical Inference: Coin Flip H0: Proportion of Heads = Prop Tails = 0.50 H1: Proportion of Heads > Prop Tails Flip a coin 4 times with result 3H and 1T. 75% Heads, should H0 be rejected? No, probability of this occurring under H0 is 31.25%
- 19. Statistical Inference: Coin Flip H0: Proportion of Heads = Prop Tails = 0.50 H1: Proportion of Heads > Prop Tails Flip a coin 4 times with result 3H and 1T. 75% Heads, should H0 be rejected? No, probability of this occurring under H0 is 31.25% Flip a coin 40 times with result 30H and 10T. 75% Heads, should H0 be rejected?
- 20. Statistical Inference: Coin Flip H0: Proportion of Heads = Prop Tails = 0.50 H1: Proportion of Heads > Prop Tails Flip a coin 4 times with result 3H and 1T. 75% Heads, should H0 be rejected? No, probability of this occurring under H0 is 31.25% Flip a coin 40 times with result 30H and 10T. 75% Heads, should H0 be rejected? Yes, probability of this occurring under H0 is <0.1%
- 21. Define and Power (Type I Error) is the probability that the study concludes the Test Arm is different from the Control Arm, when the Test Arm truly is no different. This is a regulatory risk. Power is the probability that the study concludes the Test Arm is different from the Control Arm, when in the Test Arm truly is different. This is sponsor risk.
- 22. Continuous & Binary Measures Continuous Measures Generally testing differences in Means or Medians Standard Deviation also very important Binary Measures Generally testing differences or ratios of proportions Standard Deviations generally determined by assumed proportions
- 23. Power in Pictures Consider a therapy designed to lower systolic blood pressure (SysBP) by an additional 10 mm Hg more than the currently best selling therapy, which has been shown to lower the SysBP to an average of 140 mm Hg. that each treatment has an SD of 20 mm Hg Assume the data follow normal distributions
- 24. Power in Pictures: Sample Size = 1
- 25. Power in Pictures: Sample Size = 5
- 26. Power in Pictures: Sample Size = 10
- 27. Power in Pictures: Sample Size = 30
- 28. Power in Pictures: Sample Size = 50
- 29. Power in Pictures: Sample Size = 64
- 30. Power in Pictures: Sample Size = 73
- 31. Power in Pictures: Sample Size = 86
- 32. Power in Pictures: Sample Size = 1
- 33. Power in Pictures: Sample Size = 5 Power: 10%
- 34. Power in Pictures: Sample Size = 10 Power: 18%
- 35. Power in Pictures: Sample Size = 30 Power: 47%
- 36. Power in Pictures: Sample Size = 50 Power: 69%
- 37. Power in Pictures: Sample Size = 64 Power: 80%
- 38. Power in Pictures: Sample Size = 73 Power: 85%
- 39. Power in Pictures: Sample Size = 86 Power: 90%
- 40. Standard Powers 80%: If the test product is as efficacious as assumed under H1, 80% of trials should reject H0 in favor of H1 by design. Generally considered to be the lowest targeted power. 85%: 85% of trials should reject H0 in favor of H1 by design. 90%: 90% of trials should reject H0 in favor of H1 by design.
- 41. Sponsor Risk 80%: If the test product is as efficacious as assumed under H1, 20% of trials will fail to reject H0 in favor of H1 by design. 85%: 15% of trials will fail to reject H0 in favor of H1 by design. 90%: 10% of trials will fail to reject H0 in favor of H1 by design.
- 42. How Assumptions Affect Statistical Power and Sample Size
- 43. Sample Size x Assumed Difference x Power 0 100 200 300 400 500 6 7 8 9 10 11 12 13 14 TotalSampleSize Assumed Difference SD = 20, 2-sided alpha = 0.05 80% Power 85% Power
- 44. % Increase in Sample Size (N) for Increasing Power Assuming a 2-sided = 0.05 test, N increases by: ~14% from 80% to 85% power ~34% from 80% to 90% power Assuming a 2-sided = 0.10 test, N increases by ~16% from 80% to 85% power ~38% from 80% to 90% power Assuming a 2-sided = 0.20 test, N increases by ~19% from 80% to 85% power ~46% from 80% to 90% power
- 45. Sample Size x Assumed Difference x Standard Deviation 0 100 200 300 400 500 600 6 7 8 9 10 11 12 13 14 TotalSampleSize Assumed Difference 80% Power, 2-sided alpha = 0.05 SD = 16 SD = 18 SD = 20 SD = 22
- 46. % Increase in Sample Size (N) for Increasing Standard Deviation Regardless of the and power, N increases by a factor of (SDhigh / SDlow)2
- 47. Sample Size x Assumed Difference x Alpha (Type I Error) 0 100 200 300 400 6 7 8 9 10 11 12 13 14 TotalSampleSize Assumed Difference SD = 20, 80% Power 2-sided alpha = 0.20 2-sided alpha = 0.10
- 48. % Decrease in Sample Size (N) for Increasing Alpha (Type I Error) Assuming 80% power, N decreases by: ~21% from 2-sided alpha = 0.05 to 0.10 ~42% from 2-sided alpha = 0.05 to 0.20 Assuming 85% power, N decreases by: ~20% from 2-sided alpha = 0.05 to 0.10 ~40% from 2-sided alpha = 0.05 to 0.20 Assuming 90% power, N decreases by: ~18% from 2-sided alpha = 0.05 to 0.10 ~37% from 2-sided alpha = 0.05 to 0.20
- 49. Sample Size x Assumed Difference x Power and Ratio of Randomization 0 100 200 300 400 500 600 6 7 8 9 10 11 12 13 14 TotalSampleSize Assumed Difference SD = 20, 2-sided alpha = 0.05 80% Power 80% Power 2:1 85% Power 85% Power 2:1
- 50. % Increase in Sample Size (N) for Increasing Randomization Ratio Regardless of the and power, N increases by: ~4.2% from 1:1 to 3:2 randomization ratio ~12.5% from 1:1 to 2:1 randomization ratio ~33.3% from 1:1 to 3:1 randomization ratio
- 51. % Decrease in Min Obs Diff Required for Significance with Increasing N Regardless of the , the Minimum Observed Difference required for Significance decreases by a factor of sqrt(Nlow / Nhigh) With 80% Power, N = 64 / Tx Group required diff is 7.00 mm Hg With 85% Power, N = 73 / Tx Group required diff is (7.00*sqrt(64/73)) = 6.55 mm Hg With 90% Power, N = 86 / Tx Group required diff is (7.00*sqrt(64/86)) = 6.03 mm Hg
- 52. Effect on Statistical Significance Increasing Sample Size decreases the minimum difference required to show statistical significance (driven by H0) and therefore increases power. Required sample size increases with: Increasing: Power, SD, Randomization Ratio Decreasing: Alpha, Assumed Difference