Modelling differential clustering and treatment effect heterogeneity in parallel and stepped wedge clsuter trials

Modeling differential clustering and treatment effect
heterogeneity in parallel and stepped wedge cluster trials
Karla Hemming
University of Birmingham
Monica Taljaard (Ottawa) and Andrew Forbes (Monash)
March 4, 2016
Karla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 1 / 26

Treatment effect heterogeneity - in iRCTs
Sub-group analysis common - desire to identify in whom, for whom
and where the intervention works.
Whilst it is recommended that this be pre-specified and by the use of
interaction tests, this guidance is poorly adhered to. 1
Treatment effect heterogeneity can be incorporated using either fixed
or random interaction terms.
1
Dealing with heterogeneity of treatment effects: is the literature up to the
challenge? Nicole B Gabler, Naihua Duan, Diana Liao, Joann G Elmore, Theodore G
Ganiats, Richard L Kravitz Trials. 2009; 10: 43.

Treatment effect heterogeneity - random or fixed effects?
Fixed effect interaction terms straightforward - preferable when
number of subgroups small.
Random effects approaches preferable when number of groups large
(i.e. site).
Different ways of parameterizing these models. 1 2
These different ways have gone largely unnoticed.
1
Applied Mixed Models in Medicine by Helen Brown and Robin Prescott
2
Modeling site effects in the design and analysis of multisite trials Daniel J. Feaster,
Susan Mikulich-Gilbertson, Ahnalee M. Brincks Am J Drug Alcohol Abuse.

Cluster randomized trials (CRTs) - basic set up
Cluster randomized trials are frequently used in health service
evaluation.
It is common practice to use an analysis model with a random effect
to combine between cluster information about treatment effects.
This is typically done by using a linear or generalized linear mixed
model with a random effect for cluster. 1
1
Donner, Allan and Klar, Neil; Design and analysis of cluster randomization trials in
health research; 2000

Differential variation between arms - in parallel CRTs
Can’t estimate treatment by cluster heterogeneity in CRTs
In parallel cluster trials - whilst it may be anticipated that the treatment
effect might vary across clusters, because clusters are either fully exposed
or unexposed to the intervention, the trial design does not allow
estimation of this.
Different treatments might be expected to induce homogeneity - or
even heterogeneity. 1
This will manifest itself as different ICCs in intervention and control
arms.
Also important in trials with differential clustering between arms -
perhaps where a group therapy is compared with an individual
therapy. 2
1
Ukoumunne OC, Thompson SG. Analysis of cluster randomized trials with repeated
cross-sectional binary measurements. Stat Med. 2001
2
Roberts, Chris and Roberts, Stephen A; Design and analysis of clinical trials with
clustering effects due to treatment; 2005; Clinical Trials

Treatment effect heterogeneity - in cluster cross-over trials
Can estimate treatment by cluster heterogeneity in cross-over CRTs
In cross-over cluster trials (i.e. stepped wedge) - because clusters are
crossed with treatment - the design does allow estimation of cluster by
treatment effects.
In cross-over designs and stepped wedge designs, because each cluster
receives the intervention and control, possible to estimate cluster by
treatment interactions.
Again different ways of parameterizing these models. 1 2
These different ways AGAIN have gone largely unnoticed.
1
Hughes JP, Granston TS, Heagerty PJ. Current issues in the design and analysis of
stepped wedge trials. Contemp Clin Trials. 2015
2
Baio G, Copas A, Ambler G, Hargreaves J, Beard E, Omar RZ. Sample size
calculation for a stepped wedge trial. Trials. 2015

When should such extra heterogeneity be allowed for?
At the design stage.
At the analysis stage.
Model choice is therefore important from the outset.

Objectives
Here we show that careful choice is needed over model
parameterization of heterogeneity.
Outline diﬀerent parameterizations.
Demonstrate that some parameterizations induce unnecessary
assumptions.
Show early results from a simulation study investigating statistical
optimality of the diﬀerent methods.
Preliminary recommendations for appropriate model
parameterizations.

Assumptions
Two arm trial:
Continuous outcome.
Testing for superiority.
Cluster randomization:
Assume reasonable number clusters (more than 40).
Analysis using individual level data, using mixed models with a random
eﬀect for cluster.

CRT: Basic model (single random eﬀect)
Let us consider a two arm parallel CRT. The conventional analysis model
for this simple setup is 1:
yij = µ + xij θ + αj + eij (1)
αj ∼ N[0, τ2
]
eij ∼ N[0, σ2
w ]
Under this model the correlation between two observations in the same
cluster, the intra-cluster correlation (ICC), will be:
ρ =
τ2
τ2 + σ2
w
i : individual; j : cluster
1
Donner, Allan and Klar, Neil; Design and analysis of cluster randomization trials in
health research; 2000

Model extension A: two independent random eﬀects
Two separate random eﬀects, one for treatment and one for control, are
incorporated 1:
yij = µ + xij θ + xij α(T)j + (1 − xij )α(C)j + eij (2)
α(T)j ∼ N[0, τ2
T ]
α(C)j ∼ N[0, τ2
C ]
eij ∼ N[0, σ2
w ]
The ICC in the control and intervention clusters will be:
ρC =
τ2
C
τ2
C + σ2
w
ρT =
τ2
T
τ2
T + σ2
w
Assumptions
No restrictions made on relative magnitude of two ICCs
1
cross-sectional binary measurements. Stat Med. 2001

Model extension B: a (SIMPLE) random interaction
Parametrization that includes a random interaction between the treatment
covariate and cluster 1:
yij = µ + xij θ + α(M)j + xij α(I)j + eij (3)
α(M)j ∼ N[0, τ2
M]
α(I)j ∼ N[0, τ2
I ]
eij ∼ N[0, σ2
w ]
The ICC in the control and treatment clusters will be:
ρC =
τ2
M
τ2
M + σ2
w
ρT =
τ2
M + τ2
I
τ2
I + τ2
M + σ2
w
Assumptions
ICC in treatment clusters greater than or equal to ICC in control clusters
1
Roberts, Chris and Roberts, Stephen A; Design and analysis of clinical trials with
clustering eﬀects due to treatment; 2005; Clinical Trials

The stepped wedge cluster randomized trial (SW-CRT) -
basic model
We extend the basic model (1) for parallel CRTs to the SW-CRT, by
incorporating ﬁxed eﬀects for each step 1:
yijs = µ + xijsθ + αj + ts + eijs (4)
αj ∼ N[0, τ2
]
eijs ∼ N[0, σ2
w ]
Note
This notation also applies without any loss of generality to cross-over trials
and before and after parallel studies, with repeated cross-sectional
measurements.
i : individual; j : cluster; s : step
1
Hussey and Hughes; Design and analysis of stepped wedge cluster randomized trials;

Model extension A: two independent random eﬀects
Two separate random eﬀects, one for treatment and one for control 1:
yijs = µ + xijsθ + xijsα(T)j + (1 − xijs)α(C)j + ts + eijs (5)
α(T)j ∼ N[0, τ2
T ]
α(C)j ∼ N[0, τ2
C ]
eijs ∼ N[0, σ2
w ]
The ICC in the control and intervention periods will be:
ρCC =
τ2
C
τ2
C + σ2
w
ρTT =
τ2
T
τ2
T + σ2
w
ρCT = 0
Assumptions
No restrictions made on relative magnitude of two ICCs. But, ρCT
assumed zero.
1
cross-sectional binary measurements. Stat Med. 2001 - FOR REPEATED
CROSS-SECTION DESIGN

Model extension B: an independent random interaction
Model treatment heterogeneity using a SIMPLE interaction term 1 2:
yijs = µ + xijsθ + α(M)j + xijsα(I)j + ts + eijs (6)
eijs ∼ N[0, σ2
w ]
and that
α(M)
α(I)
∼ N
0
0
,
τ2
M 0
0 τ2
I
ρCC =
τ2
M
τ2
M + σ2
w
ρTT =
τ2
M + τ2
I
τ2
I + τ2
M + σ2
w
Assumptions
ICC in treatment clusters greater than or equal to ICC in control clusters;
ρCT is non-zero.
1
Baio G, Copas A, Ambler G, Hargreaves J, Beard E, Omar RZ. Trials. 2015
2
Hughes JP, Granston TS, Heagerty PJ. Contemp Clin Trials. 2015

Model extension C: a non-independent random interaction
Model treatment eﬀect heterogeneity using an interaction term and
allowing for a covariance term (like in RCT literature):
yijs = µ + xijsθ + α(M)j + xijsα(I)j + ts + eijs (7)
eijs ∼ N[0, σ2
w ]
and that
αM
αI
∼ N
0
0
,
τ2
M σMI
σMI τ2
I
ρCC =
τ2
M
τ2
M + σ2
w
ρTT =
τ2
M + τ2
I + 2σMI
τ2
I + τ2
M + σ2
w + 2σMI
Assumptions
No restrictions made on relative magnitude of two ICCs; ρCT is non-zero.
1
Modeling site eﬀects in the design and analysis of multisite trials Daniel J. Feaster,
Susan Mikulich-Gilbertson, Ahnalee M. Brincks Am J Drug Alcohol Abuse.

Model fitting
Stata
We have fitted these models in Stata 14 using the xtmixed function, using
REML methods. ICCs were calculated using variance ratios.
Care needed
Some care is needed to make sure you are fitting the model you think you
are fitting.

Example: A parallel cluster randomised trial - with more
variability in outcomes in intervention clusters
Parallel cluster trial conducted in 53 schools (clusters).
Behavioral intervention to prevent obesity.
Outcome is BMI measured at the end of the trial.
Total of 689 observations in the intervention arm and 778
observations in the control arm.
Average cluster size of 24.

Example: usefully advocates careful choice needed over
parameterizations

Simulation study for parallel CRT
Simulation study parameters
Continuous outcome; two arms; large trial (100 centers each of size 50);
SES=0.1; Cluster randomization; Compare model A and model B only;
Generate data from model A; 1000 simulations
Scenarios
Scenario 1: No treatment by site heterogeneity.
Scenario 2: Variation between control clusters smaller than that of
intervention clusters (concordant with model B).
Scenario 3: Variation between control clusters greater than that of
intervention clusters (discordant with model B).
Findings:
No model comes out as preferable for bias of treatment eﬀects
No model comes out as preferable for coverage of treatment eﬀects
Model B gives biased estimate of ICCs under scenario 3

Simulation study for cross-over CRT
Simulation study - greater heterogeneity in control clusters.
Continuous outcome; two arms; large trial (100 centers each of size 50);
SES=0.1; Cluster cross-over randomization; Compare model A, B and C;
Generate data from model C; 1000 simulations
Model A:
No evidence of bias.
Coverage slightly too high.
Model B:
No evidence of bias.
Coverage too low.
Model C:
No evidence of bias
Coverage slightly too low.

Simulation study - with greater heterogeneity between
control clusters than intervention clusters (scenario 3)
ICC for control clusters 0.01; for intervention clusters 0.001
Model C coverage and Model A coverage > Model B coverage (0.953 vs
0.936)
0.926)
0.881)

Conclusion and discussion
Need for appreciation of assumptions implicit when fitting random
effects treatment heterogeneity models.
Choice of parameterization important - some parameterizations make
fewer assumptions and this would seem the sensible model choice.
Early simulations suggest model paramaterisation might have more
important implications for cross-over trials but not be important for
parallel trials.
Simulations suggest little evidence for bias for treatment effect; but
potential for under coverage (standard error too small) from some
models.
Some models give poor estimate of ICC.

Thank you!

Model extension E: alternative
Treatment heterogeneity modeled by what we call a true interaction term
1:
yijs = µ + xijsθ + αj + xijsα(T)j + (1 − xijs)α(C)j + ts + eijs (8)
eijs ∼ N[0, σ2
w ]
and that:


αj
α(C)j
α(T)j

 ∼ N




0
0
0

 ,


τ2
α 0 0
0 τ2
αT 0
0 0 τ2
αc




Assumed that τ2
αT = τ2
αC in 2.
1
Turner RM, White IR, Croudace T; PIP Study Group. Analysis of cluster
randomized cross-over trial data: a comparison of methods. Stat Med. 2007 Jan
30;26(2):274-89. PubMed PMID: 16538700.
2
Applied Mixed Models in Medicine by Helen Brown and Robin Prescott

Example: A SW cluster randomised trial - with more
variability in outcomes in intervention clusters
SWl cluster trial conducted in X clusters
Across X steps
Outcome is XX
Total of XXX observations
Average cluster size of XX per cluster per period

Modelling differential clustering and treatment effect heterogeneity in parallel and stepped wedge clsuter trials

More Related Content

What's hot (20)

Similar to Modelling differential clustering and treatment effect heterogeneity in parallel and stepped wedge clsuter trials (20)

Recently uploaded (20)

Modelling differential clustering and treatment effect heterogeneity in parallel and stepped wedge clsuter trials