Introduction to Statistics - Moses Asori
0 likes96 views
The document provides an overview of evidence synthesis, focusing on meta-analysis, its methodologies, statistical considerations, and the importance of avoiding biases. Key concepts such as effect sizes, heterogeneity, and quality assessment tools are discussed, alongside software recommendations for conducting analyses. The material emphasizes the significance of transparency and unbiased practices in research.
Introduction to Statistics - Moses Asori
- 1. For more info on ACSRM, Please visit https://acsrm.info/
- 2. Introduction to Statistics in Evidence Synthesis Moses Asori Ph.D. Candidate – University of North Carolina, Charlotte Research interests: Environmental Epidemiology, Spatial Epidemiology, Geographies of Health, Public Health, GIS, and Remote Sensing.
- 3. Expected Outcomes • Key features of Meta-analysis • Difference between Fixed and Random Effect Meta-analysis • To load and do basic data manipulation in Rstudio • Conduct meta-analysis in R, and test for publication bias • Software for meta-analysis
- 4. Some Pitfalls to acknowledge • Apples and Oranges (don’t combine because you can!) • Garbage In, Garbage Out (Risk of bias and quality assessment is critical) • File Drawer problem (Publication bias: will be discussed in webinar 2) • Researcher Agenda (being transparent and unbiased help!) • For Defining your scope, question, and eligibility, check FINER, PICO, and PRISMA frameworks
- 5. Statistical Methods in Evidence Synthesis • Descriptive statistics (measures of central tendencies & dispersion) • Means (standard deviations, variance, etc) • Inferential statistics (hypothesis testing, confident/credible intervals) • Random vs Fixed Effects models • Effect sizes: RR, ORs, proportions, means, mean-difference, etc
- 6. Key Components in MA • The search protocol Make sure your search technique doesn’t intentionally or otherwise lead to statistical or any other biases • Heterogeneity How do you define what is heterogenous? Types include statistical, design, and clinical • Effect sizes How do you model the effect size (fixed or random?) • Bias & Quality assessment Types of bias (publication bias, selection bias, etc.). Tools for quality assessment (e.g., Cochrane Risk of Bias tool). How do you evaluate bias (funnel plot, statistical estimation, both?
- 7. Search Protocol • Most common databases include EMBASE and Medline • Preventing redundancy • Restrictive database a call for biases • Restrictive language, a call for biases • Increase your search base • Internet sources (Google Scholar), registries, FDA, etc, can be considered • Verify the authenticity of these grey sources • Remember to avoid anything that leads to statistical biases!!!!
- 8. Effect sizes • Different studies with different outcome measures: we need effect sizes (ES) • ES is a standardized outcome measure/metric that permits the pooling of results from different studies (may include magnitude, association, and direction) • ES should be (1) Comparable, (2) Computationally feasible, (3) Reliable, (4) Interpretable • The term is Heavily contested (effect suggests causation: that’s a problem), but it’s still the standard • Normally represented as theta (θ) as the true effect size for the overall study effect, whereas θk represents the true effect for study K • Imagine θ = θk + EK • Also θk (hat) = θk+ϵk • Where EK is the error associated with the theta • In reality, we never know the true measures and sampling error in our studies, but we estimate this by repeated re-estimation of our mean and calculating the standard error (deviations around our expectation) We can pool mean, proportions, Pearson correlation, Point-Biserial correlation, mean difference, Risk and odds Ratios, etc.
- 9. For Observational Studies • Means (SD/Var/SE): May need some transformation (log) • Proportion (SD/Var/SE): Log/arcsin transformation • Pearson correlation (SD/SE/Var): Fisher z • Point-Biserial Correlation (SD/Var/SE): SMD transformation For Experimental Designs • Mean difference (MD) • Standardized MD • Risk Ratio • Odds Ratio • Incidence Rate Ratios
- 10. Effect sizes Pooling the ES • We are aggregating our study outcomes • We can pool in two ways (models) • Fixed effect (no real differences between studies except for sampling error) • Random Effect (Real differences between studies)
- 12. Before you pool…, • Effect Size Correction • Small Sample (Hedges’ g) • Unreliability (test-retest-reliability or attenuation) • We can correct for unreliability using the Hunter and Schmidt method in R or any standard software environment that supports this. • Know the model to use: But what exactly is a model?
- 13. Fixed Effect Model 1. Since all factors are assumed fixed, the only reason results will vary is random sampling error 2. Although the error is random, the sampling distribution of the errors can be estimated Weighted Mean Study weight Study variance Study ES (r2, % etc.)
- 14. Random-Effects Models • Does not assume that the true effect is identical across studies • Because study characteristics vary (e.g., participant characteristics, treatment intensity, outcome measurement), there may be different effect sizes underlying different studies • Error, therefore, comes from many sources (internal and external factors)
- 15. Estimating the true variance (tau-squared) • The tau (expected variance) is quite complicated to calculate manually. • DerSimonian-Laird; Restricted Maximum Likelihood; Paule- Mandel; Empirical Bayes; Sidik-Jonkman. • Which one to use? It depends on many factors (e.g., sample size, number of studies, variation in sample, etc.)
- 16. Heterogeneity • It quantifies the degree of variance within and between studies. • To what degree do we conclude there is a significant degree? • Default one used is 50% --But is that right? • Let's consider different reasons for the variance: statistical, design-related, or clinically related. • Let’s reconsider the default of 50% • Some researchers suggest going for 25% or less! • But how do we quantify? See the next page….
- 17. Heterogeneity • Baseline or design-related heterogeneity • Statistical heterogeneity • Cochran’s Q Outliers & Influential Cases 1. Basic Outlier Removal (how do we define it?) 2. Influence Analysis (InfluenceAnalysis in dmetar) Which Heterogeneity measure should I use? Maybe use all? Or report tau2 or prediction interval I2 increases as the number of studies increases because the SE reduces
- 18. Publication Bias • Citation bias • Time-lag bias • Language bias • Outcome reporting bias Addressing Publication Bias • Funnel plot • Eger’s test • Peter’s regression • All based on small study effect
- 19. Presenting your results • Forest plots, funnel plots. • Sensitivity analysis. • Guidelines for reporting (PRISMA, MOOSE).
- 20. Notable Software for Meta-Analysis • Comprehensive Meta-Analysis (Not for free) • R with Meta-Analysis Packages (For free) • RevMan (Review Manager) (Free) • STATA (Not for free) • JASP (Free) • MetaXL (Not sure) • WinBUGS/OpenBUGS (Not sure) • MixMeta (Not sure)
- 21. References • Cummings, Steven R, Warren S Browner, and Stephen B Hulley. 2013. “Conceiving the Research Question and Developing the Study Plan.” Designing Clinical Research 4: 14–22. • DerSimonian, Rebecca, and Nan Laird. 1986. “Meta-Analysis in Clinical Trials.” Controlled Clinical Trials 7 (3): 177–88. • lwood, Peter. 2006. “The First Randomized Trial of Aspirin for Heart Attack and the Advent of Systematic Overviews of Trials.” Journal of the Royal Society of Medicine 99 (11): 586–88. • Eysenck, Hans J. 1978. “An Exercise in Mega-Silliness.” American Psychologist 33 (5). • Fisher, Ronald A. 1935. The Design of Experiments. Oliver & Boyd, Edinburgh, UK. • Glass, Gene V. 1976. “Primary, Secondary, and Meta-Analysis of Research.” Educational Researcher 5 (10): 3–8.