Presenting the results of meta-analysis

Saudi Board of Preventive Medicine, Riyadh Ministry of Health, KSA Lecture 09/10
Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course
Presenting the results of
meta-analysis
Dr. S. A. Rizwan M.D.,
Public Health Specialist & Lecturer,
Saudi Board of Preventive Medicine – Riyadh,
Ministry of Health, KSA
26.11.2019 1

Outline
• Panels
• Charts
• Tables
• Graphs
26.11.2019 2

Panels
26.11.2019 3

Panel
26.11.2019 4

Panel
26.11.2019 5

Panel
26.11.2019 6

Panel
26.11.2019 7

Charts
26.11.2019 8

Flowchart
Study selection
process
26.11.2019 9

Flowchart
Study selection
process
26.11.2019 10

Other charts
26.11.2019 11

Other charts
26.11.2019 12

Tables
26.11.2019 13

Types of analyses and tables
• Main or primary analysis
• The main objective of the paper
• Pooled estimate of effect size
• Secondary analysis
• Subgroup analysis
• To see if effects are different across interest groups
• Need to have sufficient sample
• Sensitivity analysis
• To see if estimates change by changing some parameters like excluding low quality studies
• Meta-regression
• Publication bias analysis
26.11.2019 14

Description of
study features
26.11.2019 15

Primary analyses table
26.11.2019 16

Subgroup
analyses table
26.11.2019 17

Subgroup
analyses table
26.11.2019 18

Sensitivity analyses table
26.11.2019 19

Meta-regression table
26.11.2019 20

Graphs
26.11.2019 21

Graphs
• Graphs are an essential tool for conveying the results of a meta-
analysis to readers
• But if poorly constructed, graphs can be misleading and/or confuse
readers
• Graphs should strive for accuracy, simplicity, clarity, and aesthetics
26.11.2019 22

Basic Graphing Principles
• Descriptive titles and/or captions
• Use of legends (when appropriate)
• Representative range of scale
• Properly labeled axes
• Inclusion of reference points on axes
• Graphs should reflect the statistical precision of results
• Explicit mention of any excluded data
• Data in graphs should generally be available elsewhere in the review (except
in very large reviews)
• Aesthetics (line thickness, symbol size, symbol types, parsimony)
26.11.2019 23

Forest plots
26.11.2019 24

Forest plots
• The “workhorse” graph in meta-analysis
• Display effect size estimates and confidence intervals for each study
included in the meta-analysis
• Effect size estimates typically shown with blocks proportionate to the
weight assigned to a given study
• Functions to draw the eye toward studies with larger sample size/larger
weights, and away from smaller studies with wider confidence intervals
26.11.2019 25

Forest plots
• Estimated mean effect size with confidence interval shown at the
bottom, typically with a diamond
• In random effects meta-analyses, prediction intervals can be used to
display dispersion in the estimated effect
• Studies should be ordered in a meaningful way
• Effect size magnitude
• Study weight (precision)
• Chronological order
• Other meaningful study characteristic
26.11.2019 26

Forest plot
26.11.2019 27

Forest plot
26.11.2019 28

Forest plots with subgroups
• Display effect size estimates and confidence intervals for each study,
split by some grouping variable
• Useful for depicting results from subgroup or moderator analyses
• May include the overall summary effect across groups, if appropriate
• Results from statistical tests of moderation
• (e.g., QB or b from a meta-regression) should be summarized on the
graph or in footnotes, when appropriate
26.11.2019 29

Forest plots with subgroups
26.11.2019 30

Forest plots with
subgroups
26.11.2019 31

Summary forest plots
• Display summary (mean) effect sizes and confidence intervals for
different groups of studies
• Does not include effect size estimates from individual studies
• Useful for very large reviews where traditional forest plots may not be
feasible, but effects can be categorized into meaningful groups (e.g.,
across intervention, study, participant types)
• May include the overall summary effect across groups, if appropriate
• Results from statistical tests of moderation (e.g., QB or b from a meta-
regression) should be summarized on the graph or in footnotes, when
appropriate
26.11.2019 32

Summary
forest plot
26.11.2019 33

Cumulative meta-analysis forest plots
• Display results from iterative estimation of summary (mean) effect sizes,
cumulatively adding one study at a time
• Useful for showing the accumulation of evidence over time, or the
in/stability of intervention effects over time
• May also be used to explore small sample bias, cumulatively adding
studies according to sample size of primary studies
• Title should clearly specify it is a forest plot showing results from a
cumulative meta-analysis
26.11.2019 34

Cumulative
forest plot
26.11.2019 35

General suggestions – forest plots
• Always include forest plots (or summary forest plots) if possible/appropriate
• Not recommended with fewer than 2 studies
• Plot ratio effect size measures on the log scale, but include axis labels on the
original anti-logged scale
• Include reference lines at the null value
• State the confidence level for confidence intervals
• Blocks for each study should be proportionate to study weight
• Sort studies in a meaningful order (e.g., effect size magnitude)
• State the direction of results
• Include prediction intervals for random effects analyses
• Include numerical data on plots (if possible)
26.11.2019 36

What’s wrong with
this forest plot?
• Uninformative study labels
• Seemingly random order of
effect sizes
• Unclear direction of effect sizes
• Does not include data
• Unspecified confidence level
• General aesthetics (white
space)
26.11.2019 37

Funnel plots
26.11.2019 38

Funnel plots
• Exploratory tool used to visually assess the possibility of
publication/small study bias in a meta-analysis
• Scatter plot of effect size (x-axis) against some measure of study size
(y-axis)
• x-axis: use log scale for ratio effect size measures, e.g., ln(OR), ln(RR)
• y-axis: the standard error of the effect size is generally recommended,
• y-axis: other options also available
• Not recommended in very small meta-analyses (e.g., n<10)
26.11.2019 39

Funnel plots
• If publication bias is present, you would expect null or ‘negative’
findings from small n studies to be suppressed (i.e., missing from the
plot)
• Asymmetry in the funnel plot for small n studies may provide
evidence of possible publication bias
• Symmetry in the funnel plot provides some evidence against the
possibility of publication bias
26.11.2019 40

Dr. S. A. Rizwan, M.D.Demystifying statistics series: Meta-analysis course 26.11.2019 41
Funnel plot

Funnel plot
26.11.2019 42

Funnel plot
26.11.2019 43

Funnel plots
• Asymmetry could be due to factors other than publication bias, e.g.,
• Poor methodological quality
• Other reporting biases
• Artefactual variation
• Chance
• True heterogeneity
• Assessing funnel plot symmetry relies entirely on subjective visual
judgment
26.11.2019 44

Contour enhanced funnel plots
• Funnel plot with additional contour lines associated with ‘milestones’
of statistical significance: p = .001, .01, .05, etc.
• If studies are missing in areas of statistical non-significance, publication bias
may be present
• If studies are missing in areas of statistical significance, asymmetry may be
due to factors other than publication bias
• If there are no studies in areas of statistical significance, publication bias may
be present
• Can help distinguish funnel plot asymmetry due to publication bias
versus other factors
26.11.2019 45

Contour enhanced
funnel plot
26.11.2019 46

General suggestions – funnel plots
• Not recommended with fewer than 10 studies
• Plot effect sizes on the horizontal axis
• Plot the standard error of the effect size on the vertical axis (generally)
• Plot ratio effect size measures on the log scale, but include axis labels on the
original anti-logged scale
• All points should be the same size (weights/precision represented in the vertical
axis)
• Include 95% pseudo-confidence limits from a fixed effect analysis
• Include contours if possible
• Data in graphs should generally be available elsewhere in the review (except in
very large reviews)
• Use different plotting symbols to distinguish subgroups, when appropriate
26.11.2019 47

What’s wrong with
this funnel plot?
• Effect size on vertical axis
• Points are not all the same size
• Vague labeling of axes and
reference line
• No confidence bands
26.11.2019 48

Bubble plots
• Scatter plot of a study covariate (x-axis) against effect size (y-axis)
• Useful to characterize covariates that may be a source of
heterogeneity
• Provides a visual representation of results from a bivariate meta-
regression model
26.11.2019 49

Bubble plot
26.11.2019 50

Bubble plot
26.11.2019 51

General suggestions – bubble plots
• Plot effect sizes on the vertical axis
• Plot the covariate on the horizontal axis
• Plot ratio effect size measures on the log scale, but include axis labels
on the original anti-logged scale
• Points should be proportionate to study weight
• Include fitted meta-regression line (if appropriate)
• Data in graphs should generally be available elsewhere in the review
(except in very large reviews)
26.11.2019 52

Galbraith plot
• Scatter plot of inverse standard error (x-axis) against a standardized
effect size (i.e., effect size divided by its standard error) (y-axis)
• Includes an unweighted regression line constrained through the origin
with slope equal to the fixed effect summary effect size estimate
• Useful for displaying heterogeneity and aiding detection of outliers
• Useful for displaying effect sizes in very large reviews where forest
plots may be impractical
26.11.2019 53

Galbraith plot
26.11.2019 54

Galbraith plot
• Points should be the same size for study (weight/precision is
represented in the horizontal axis)
• Include confidence intervals around the fixed effect summary effect
line
• Use different plotting symbols to distinguish subgroups, when
appropriate
26.11.2019 55

L’abbé plot
• Plot of control group risk (x-axis) against treatment group risk (y-axis)
• Commonly used to depict risks, but can also be plotted on log risk or
log odds
• Most commonly used for binary outcome data, but can be extended
to depict means for continuous outcomes or ROC plot for
diagnostic/screening test accuracy
• Can also be used to contrast different effect size metrics (odds ratio,
risk ratio, risk difference)
26.11.2019 56

L’abbé plot
26.11.2019 57

L’abbé plot
26.11.2019 58

L’abbé plot
• Can be used to investigate
heterogeneity
• Ideally a L'Abbé plot should have the
symbols appropriate to the size of
the trials
• There is an inset for the symbol size,
and the two colours show trazodone
used for erectile dysfunction in two
different conditions (and with clear
clinical heterogeneity)
26.11.2019 59
Trazodone for erectile dysfunction in
psychogenic erectile dysfunction (dark
symbols) and with physiological or mixed
aetiology (light symbols)

Other graphs
• Density strips
• Raindrop plots
• Graphical display of study heterogeneity (GOSH)
• CUSUM chart
• Veritas plot
• Summary receiver-operator curve (SROC) graphs
• Cross hairs ROC plot
• Harvest plot
• Baujat plots
26.11.2019 60

Density strip
• Density Strip Plot of Relative
Risk of Progression to
Clinical Disease/AIDS.
Bayesian random effects
analysis (favors treatment on
the left of 1.0); density plots
on the figure on the left
represent combined data;
• on the right accounts for
individual study arms and the
impact of adding selenium or
vitamin A to MNS
26.11.2019 61
PLoS ONE 10(4):e0120113

Raindrop
forest plot
• The width of the likelihood
raindrops corresponds to the
confidence intervals, as also
shown in the classic forest plot.
In addition, for each likelihood
drop the height (and color
shading) visualizes the
plausibility of true values given
the observed estimate.
26.11.2019 62

GOSH
• New method for visualizing
between-study heterogeneity using
combinatorial meta-analysis.
• The method is based on performing
separate meta-analyses on all
possible subsets of studies in a
meta-analysis.
• Summary effect sizes and other
statistics produced by the all-
subsets meta-analyses to generate
graphs that can be used to
investigate heterogeneity, identify
influential studies, and explore
subgroup effects.
26.11.2019 63
(wileyonlinelibrary.com) DOI: 10.1002/jrsm.1053

Veritas plot
• The Veritas plot is an adaption of
the radar plot, a graphic tool for the
description of multiattribute data.
• Key elements of meta-analytical
quality such as heterogeneity,
publication bias and study design
are assessed.
26.11.2019 64
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2769126/

SROC plot
• Summary receiver operating
characteristic (SROC) analysis is
used in diagnostic accuracy meta-
analyses
• It is suited to this purpose with its
use of sensitivity and specificity.
• However, it is only meaningful
when similar endpoints,
diagnostic threshold, study
quality, and test characteristics
are compared.
• PLoS ONE 11(7):e0159860
26.11.2019 65

Cross-hair
ROC plot
• The use of plots of receiver-operator
curve (ROC) space showing
individual studies and summary
estimates of diagnostic accuracy has
become common but can be
difficult to interpret.
• ‘Cross-hairs’ plots display the
individual studies in ROC space with
paired confidence intervals
representing sensitivity and
specificity, and allow for the results
of meta-analysis to be overlaid on
the plot.
26.11.2019 66
Phillips, B., Stewart, L. A., & Sutton, A. J. (2010). “Cross hairs” plots for diagnostic
meta-analysis. Research Synthesis Methods, 1(3-4), 308–315. doi:10.1002/jrsm.26

Harvest plot
• Harvest plots are used to
graphically display evidence
from complex and diverse
studies or results.
• Overviews of reviews bring
together evidence from two
or more systematic reviews
26.11.2019 67
Harvest plots for overview of
reviews on acute otitis media

Baujat plot
• A diagnostic plot to detect
sources of heterogeneity
• Contribution of each study to the
overall Q-test statistic for
heterogeneity on the horizontal
axis
• Influence of each study (defined
as the standardized squared
difference between the overall
estimate based on a fixed-effects
model with and without the ith
study included in the model) on
the vertical axis.
26.11.2019 68
http://www.metafor-project.org/doku.php/plots:baujat_plot

Take home messages
• Panels and flowcharts help to understand the study methods
• Tables provide valuable information on results in glance
• Tables should be split into meaningful groups with all statistical info
• Graphs are an important part of any meta-analysis and can greatly
facilitate interpretation
• Forest plots should (almost always) be included
• Funnel plots and other various plots may also be appropriate
• Always follow standard graphing principles, and strive for accuracy,
simplicity, clarity, aesthetic appeal, and good structure
26.11.2019 69

Thank you
Kindly email your queries to sarizwan1986@outlook.com
26.11.2019 70

Presenting the results of meta-analysis

More Related Content

What's hot (20)

Similar to Presenting the results of meta-analysis (20)

More from Rizwan S A (18)

Recently uploaded (20)

Presenting the results of meta-analysis