Quantitative Synthesis II Quiz
- 1. Quantitative Synthesis II Interactive Quiz Prepared for: The Agency for Healthcare Research and Quality (AHRQ) Training Modules for Systematic Reviews Methods Guide www.ahrq.gov
- 2. Assume that you are performing a meta-analysis of randomized controlled trials on vitamin D supplementation and mortality in elderly people who are institutionalized. You want to explore whether or not the treatment effect differs across trials according to their maximum followup. The maximum length of followup is: A patient-level covariate A study-level covariate Covariates
- 3. You should perform only the subgroup analyses that are specified a priori and not perform post-hoc subgroup analyses. False True Subgroup Analysis
- 4. Because of the risk of ecological fallacy, meta-regressions on patient-level covariates should never be performed. True False Meta-Regressions and the Risk of Ecological Fallacy
- 5. Assessing Study Conclusions Based on this example, which of the following statements is more likely to be true? These conclusions are based on a well constructed meta-regression (see on the right) and are definitely valid. The analyses and conclusions are suspect. Mean 25-hydroxy-D Concentration
- 6. Meta-analyses require understanding of covariates and how to analyze them. Patient-level covariates differ across patients in the same study or in the same study arm. Study-level covariates pertain to the whole study and do not vary across patients in the same study. Results of subgroup analyses should be viewed with skepticism, especially when adjustments for multiple testing have not been performed. If an association from a meta-regression on the mean of a patient-level covariate is biologically plausible, it merits further study. Summary
- 7. This quiz was prepared by Joseph Lau, M.D., and Thomas Trikalinos, M.D., Ph.D., members of the Tufts Medical Center Evidence-based Practice Center. The information in this module is based on Chapter 9 in Version 1.0 of the Methods Guide for Comparative Effectiveness Reviews (available at: http://www.effectivehealthcare.ahrq.gov/repFiles/2007_10DraftMethodsGuide.pdf). Authors
Editor's Notes
- #2: Quantitative Synthesis II Interactive Quiz
- #3: Covariates Assume that you are performing a meta-analysis of randomized controlled trials on vitamin D supplementation and mortality in elderly people who are institutionalized. You want to explore whether or not the treatment effect differs across trials according to their maximum followup. The maximum length of followup is: Incorrect. Patient-level covariates differ across patients in the same study or in the same study arm. Maximum length of followup is the same for all patients in a study. Correct. Study-level covariates pertain to the whole study and do not vary across patients in the same study. Maximum length of followup is the same for all patients in a study.
- #4: Subgroup Analysis You should perform only the subgroup analyses that are specified a priori and not perform post-hoc subgroup analyses. Incorrect. Most meta-analyses use data that are published (and potentially known to those performing a meta-analysis). When you prepare adequately before embarking on a meta-analysis (as you should ), you will inevitably become acquainted with the very data that you will eventually analyze. This makes it difficult to claim that you specified subgroups without knowing anything about your data. Correct. Most meta-analyses use data that are published (and potentially known to those performing a meta-analysis). When you prepare adequately before embarking on a meta-analysis (as you should ), you will inevitably become acquainted with the very data that you will eventually analyze. This makes it difficult to claim that you specified subgroups without knowing anything about your data. Meta-analysts should do their best to define subgroups that make methodological and biological sense. Also, you should treat results of subgroup analyses with a healthy dose of skepticism, especially when adjustments for multiple testing are not performed.
- #5: Meta-Regressions and the Risk of Ecological Fallacy Because of the risk of ecological fallacy, meta-regressions on patient-level covariates should never be performed. Incorrect. Meta-regressions of patient-level covariates are not necessarily false. For example, the average received vitamin D dose in a well-conducted randomized controlled trial may be very close to the protocol dose and may be representative of the dose received by each patient. Meta-regression is about exploration and forming hypotheses. If an association from a meta-regression on the mean of a patient-level covariate is biologically plausible, it merits further study. Correct. Meta-regression is about exploration and forming hypotheses. If an association from a meta-regression on the mean of a patient-level covariate is biologically plausible, it merits further study.
- #6: Assessing Study Conclusions Based on this example, which of the following statements is more likely to be true? Incorrect. You should be suspicious of conclusions that are too strong and results that seem too good to be true. Correct. This is a problematic analysis. Ecological fallacy may operate here because the concentration of 25-hydroxy-D is a patient-level covariate. The meta-regression is not correctly specified. You have to read the study and the papers cited therein to know this for sure, but there is at least one telltale sign of a misspecified meta-regression: the scale of the y-axis should be logarithmic for the odds ratio and the risk ratio; it should be linear for the risk difference. Here it is linear for the odds ratio, which is a bizarre choice.
- #7: Summary
- #8: Authors This interactive quiz augments the second module on quantitative synthesis. This quiz was prepared by Joseph Lau, M.D., and Thomas Trikalinos, M.D., Ph.D., members of the Tufts –New England Medical Center Evidence-based Practice Center. The quiz is based on Chapter 9 in Version 1.0 of the Methods Guide for Comparative Effectiveness Reviews (available at: http://www.effectivehealthcare.ahrq.gov/ehc/products/60/294/2009_0805_principles1.pdf).