SlideShare a Scribd company logo
A ROBUST BAYESIAN ESTIMATE OF THE
CONCORDANCE CORRELATION COEFFICIENT (2)
Dai Feng, Richard Baumgartner & Vladimir Svetnik
Jan 18, 2019
Dai Feng, Richard Baumgartner & Vladimir SvetnikA ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICJan 18, 2019 1 / 16
1 INTRODUCTION
2 A ROBUST PARAMETRIC MODEL BASED ON MULTIVARIATE
t-DISTRIBUTION
3 SIMULATION STUDY
4 REAL-LIFE EXAMPLES
5 CONCLUSION AND DISCUSSION
Dai Feng, Richard Baumgartner & Vladimir SvetnikA ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICJan 18, 2019 2 / 16
INTRODUCTION
INTRODUCTION
Dai Feng, Richard Baumgartner & Vladimir SvetnikA ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICJan 18, 2019 3 / 16
A ROBUST PARAMETRIC MODEL BASED ON MULTIVARIATE
t-DISTRIBUTION
A ROBUST PARAMETRIC MODEL BASED ON
MULTIVARIATE t-DISTRIBUTION
Dai Feng, Richard Baumgartner & Vladimir SvetnikA ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICJan 18, 2019 4 / 16
A ROBUST PARAMETRIC MODEL BASED ON MULTIVARIATE
t-DISTRIBUTION
2.1. Robust Bayesian Method for the CCC Estimation
Under the multivariate t-distribution, the CCC is defined as
CCCt =
2 d−1
i=1
d
j=i+1
ν
ν−2 σij
(d − 1) d
i=1
ν
ν−2 σ2
i + d−1
i=1
d
j=i+1(µi − µj)2
(1)
where ν = the degrees of freedom,
µi s = components of the location vector,
σ2
i s = diagonal elements of the scale matrix,
σij s = off diagonal elements of the scale matrix.
Dai Feng, Richard Baumgartner & Vladimir SvetnikA ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICJan 18, 2019 5 / 16
A ROBUST PARAMETRIC MODEL BASED ON MULTIVARIATE
t-DISTRIBUTION
2.1. Robust Bayesian Method for the CCC Estimation
(conti.)
Conjugate priors
µ ∼ MVN(µ0, Σ0)
Σ−1
∼ Wishart(ρ, V)
ν ∼ U(νmin, νmax)
Noninformative priors
µ0 = 0, Σ0 = very large,
ρ = d, V = diagonal matrix,
νmin = 4, νmax = 25 ← produces accurate estimates
for the CCC in the scenarios they studied
Dai Feng, Richard Baumgartner & Vladimir SvetnikA ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICJan 18, 2019 6 / 16
A ROBUST PARAMETRIC MODEL BASED ON MULTIVARIATE
t-DISTRIBUTION
2.1. Robust Bayesian Method for the CCC Estimation
(conti.)
Posterior distributions under the conjugate priors
µ ∼ MVN(A−1
b, A−1
)
Σ−1
∼ Wishart

ρ + n, V−1
+
n
i=1
λi (Yi − µ)(Yi − µ)T
−1


λi ∼ Γ
ν + d
2
,
ν + (Yi − µ)T Σ−1
(Yi − µ)
2
where A =
n
i=1
λi Σ−1
+ Σ−1
0 , b = Σ−1
n
i=1
λi Yi + Σ−1
0 µ0.
Dai Feng, Richard Baumgartner & Vladimir SvetnikA ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICJan 18, 2019 7 / 16
A ROBUST PARAMETRIC MODEL BASED ON MULTIVARIATE
t-DISTRIBUTION
2.1. Robust Bayesian Method for the CCC Estimation
(conti.)
The logarithm of the posterior distribution of ν is proportional to
n
i=1
ν
2
log
ν
2
− log Γ
ν
2
+
ν
2
− 1 log λi −
ν
2
λi
and within the interval of [νmin, νmax].
To update ν with a bounded support, they use the slice sampling (Neal,
2003), which is easy to choose the magnitude of changes adaptively.
To run multiple sequences of the MCMC, the MLE-based estimates could
be used to create a starting point as suggested by Liu (1994).
Dai Feng, Richard Baumgartner & Vladimir SvetnikA ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICJan 18, 2019 8 / 16
A ROBUST PARAMETRIC MODEL BASED ON MULTIVARIATE
t-DISTRIBUTION
2.1. Robust Bayesian Method for the CCC Estimation
(conti.)
Slice sampling (Neal, 2003)
Neal, R. M. (2003). Slice Sampling. The Anals of Statistics 31:705-741.
To implement Gibbs sampling, one may need to devise methods for
sampling from nonstandard univariate distributions.
To use the Metropolis algorithm, one must find an appropriate
“proposal” distribution that will lead to efficient sampling.
Simple forms of univariate slice sampling are an alternative to Gibbs
sampling that avoids the need to sample from nonstandard
distributions.
It is easier to tune than Metropolis methods and also avoids problems
that arise when the appropriate scale of changes varies over the
distribution.
Dai Feng, Richard Baumgartner & Vladimir SvetnikA ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICJan 18, 2019 9 / 16
A ROBUST PARAMETRIC MODEL BASED ON MULTIVARIATE
t-DISTRIBUTION
2.1. Robust Bayesian Method for the CCC Estimation
(conti.)
Slice sampling (Neal, 2003)
The single variable slice sampling methods.
x0 : current value , x1 : a new value
1 Draw a real value, y, uniformly from (0, f (x0)), thereby defining a
horizontal “slice” : S = {x : y < f (x)}.
2 Find an interval, I = (L, R), around x0 that contains all, or much, of
the slice.
3 Draw the new point, x1, from the part of the slice within this interval.
Dai Feng, Richard Baumgartner & Vladimir SvetnikA ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICJan 18, 2019 10 / 16
A ROBUST PARAMETRIC MODEL BASED ON MULTIVARIATE
t-DISTRIBUTION
2.1. Robust Bayesian Method for the CCC Estimation
(conti.)
Slice sampling (Neal, 2003)
Multivariate slice sampling methods.
x0 = (x0,1, . . . , x0,n) : current value
x1 = (x1,1, . . . , x1,n) : a new value
1 Draw a real value, y, uniformly from (0, f (x0)), thereby defining the
slice S = {x : y < f (x)}.
2 Find a hyperrectangle, H = (L1, R1) × · · · × (Ln, Rn), around x0, which
preferably contains at least a big part of the slice.
3 Draw the new point, x1, from the part of the slice within this
hyperrectangle.
Dai Feng, Richard Baumgartner & Vladimir SvetnikA ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICJan 18, 2019 11 / 16
A ROBUST PARAMETRIC MODEL BASED ON MULTIVARIATE
t-DISTRIBUTION
2.1. Robust Bayesian Method for the CCC Estimation
(conti.)
Slice sampling (Neal, 2003)
Various other things
Finding an appropriate hyperrectangle.
Sampling from the part of the slice within the hyperrectangle.
Overrelaxed slice sampling.
Reflective slice sampling.
Dai Feng, Richard Baumgartner & Vladimir SvetnikA ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICJan 18, 2019 12 / 16
A ROBUST PARAMETRIC MODEL BASED ON MULTIVARIATE
t-DISTRIBUTION
2.2 Robust Bayesian Estimation of the CCC with
Accommodation of Covariates and Multiple Replications
Dai Feng, Richard Baumgartner & Vladimir SvetnikA ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICJan 18, 2019 13 / 16
SIMULATION STUDY
SIMULATION STUDY
Dai Feng, Richard Baumgartner & Vladimir SvetnikA ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICJan 18, 2019 14 / 16
REAL-LIFE EXAMPLES
REAL-LIFE EXAMPLES
Dai Feng, Richard Baumgartner & Vladimir SvetnikA ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICJan 18, 2019 15 / 16
CONCLUSION AND DISCUSSION
CONCLUSION AND DISCUSSION
Dai Feng, Richard Baumgartner & Vladimir SvetnikA ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICJan 18, 2019 16 / 16

More Related Content

PPT
Formont.ppt
PPT
DCS-IGARSS11_v2-aguilera.ppt
PDF
Geometrical interpretation of the magnitude of the cross product
PDF
S2 9 areas and volumes
PDF
Ravasi_etal_EAGE2014
DOCX
Using scale coordination and semantic information for robust 3 d object recog...
PPT
YARCA (Yet Another Raycasting Application) Project
PPTX
NUMERICAL INTEGRATION AND ITS APPLICATIONS
Formont.ppt
DCS-IGARSS11_v2-aguilera.ppt
Geometrical interpretation of the magnitude of the cross product
S2 9 areas and volumes
Ravasi_etal_EAGE2014
Using scale coordination and semantic information for robust 3 d object recog...
YARCA (Yet Another Raycasting Application) Project
NUMERICAL INTEGRATION AND ITS APPLICATIONS

What's hot (8)

PPTX
Applications of integration
PPTX
Nsm ppt.ppt
PDF
Tipler volume 2 resolução 6 edição
PPTX
NO HEXAGONS. The Central place theory illusion revealed - ECTQG2013
PDF
A NEW STUDY OF TRAPEZOIDAL, SIMPSON’S1/3 AND SIMPSON’S 3/8 RULES OF NUMERICAL...
PPTX
Karnaugh Maps
PPTX
Calculus Application of Integration
DOC
Cf dfinal1 (may11)
Applications of integration
Nsm ppt.ppt
Tipler volume 2 resolução 6 edição
NO HEXAGONS. The Central place theory illusion revealed - ECTQG2013
A NEW STUDY OF TRAPEZOIDAL, SIMPSON’S1/3 AND SIMPSON’S 3/8 RULES OF NUMERICAL...
Karnaugh Maps
Calculus Application of Integration
Cf dfinal1 (may11)
Ad

Similar to Review: A ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICIENT (PART 2) (14)

PDF
Review: A ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICIENT...
PDF
Review: A ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICIENT...
PDF
Sarjinder singh
PDF
A Note on Confidence Bands for Linear Regression Means-07-24-2015
PDF
Petrini - MSc Thesis
PDF
asymptotics of ABC
PDF
the ABC of ABC
PDF
Laplace's Demon: seminar #1
PDF
ABC-Gibbs
PDF
Bayesian hybrid variable selection under generalized linear models
PDF
Bayesian/Fiducial/Frequentist Uncertainty Quantification by Artificial Samples
PDF
outiar.pdf
PDF
CDT 22 slides.pdf
Review: A ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICIENT...
Review: A ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICIENT...
Sarjinder singh
A Note on Confidence Bands for Linear Regression Means-07-24-2015
Petrini - MSc Thesis
asymptotics of ABC
the ABC of ABC
Laplace's Demon: seminar #1
ABC-Gibbs
Bayesian hybrid variable selection under generalized linear models
Bayesian/Fiducial/Frequentist Uncertainty Quantification by Artificial Samples
outiar.pdf
CDT 22 slides.pdf
Ad

Recently uploaded (20)

PDF
CAPERS-LRD-z9:AGas-enshroudedLittleRedDotHostingaBroad-lineActive GalacticNuc...
PPTX
famous lake in india and its disturibution and importance
PPTX
cpcsea ppt.pptxssssssssssssssjjdjdndndddd
PDF
Formation of Supersonic Turbulence in the Primordial Star-forming Cloud
PPTX
INTRODUCTION TO EVS | Concept of sustainability
PPTX
Microbiology with diagram medical studies .pptx
PPTX
Derivatives of integument scales, beaks, horns,.pptx
PPTX
Introduction to Cardiovascular system_structure and functions-1
PDF
Phytochemical Investigation of Miliusa longipes.pdf
PPT
protein biochemistry.ppt for university classes
PDF
. Radiology Case Scenariosssssssssssssss
PPTX
Introduction to Fisheries Biotechnology_Lesson 1.pptx
PPTX
EPIDURAL ANESTHESIA ANATOMY AND PHYSIOLOGY.pptx
PDF
The scientific heritage No 166 (166) (2025)
PDF
Mastering Bioreactors and Media Sterilization: A Complete Guide to Sterile Fe...
PDF
SEHH2274 Organic Chemistry Notes 1 Structure and Bonding.pdf
PPTX
TOTAL hIP ARTHROPLASTY Presentation.pptx
PPTX
2Systematics of Living Organisms t-.pptx
PPT
The World of Physical Science, • Labs: Safety Simulation, Measurement Practice
PDF
Sciences of Europe No 170 (2025)
CAPERS-LRD-z9:AGas-enshroudedLittleRedDotHostingaBroad-lineActive GalacticNuc...
famous lake in india and its disturibution and importance
cpcsea ppt.pptxssssssssssssssjjdjdndndddd
Formation of Supersonic Turbulence in the Primordial Star-forming Cloud
INTRODUCTION TO EVS | Concept of sustainability
Microbiology with diagram medical studies .pptx
Derivatives of integument scales, beaks, horns,.pptx
Introduction to Cardiovascular system_structure and functions-1
Phytochemical Investigation of Miliusa longipes.pdf
protein biochemistry.ppt for university classes
. Radiology Case Scenariosssssssssssssss
Introduction to Fisheries Biotechnology_Lesson 1.pptx
EPIDURAL ANESTHESIA ANATOMY AND PHYSIOLOGY.pptx
The scientific heritage No 166 (166) (2025)
Mastering Bioreactors and Media Sterilization: A Complete Guide to Sterile Fe...
SEHH2274 Organic Chemistry Notes 1 Structure and Bonding.pdf
TOTAL hIP ARTHROPLASTY Presentation.pptx
2Systematics of Living Organisms t-.pptx
The World of Physical Science, • Labs: Safety Simulation, Measurement Practice
Sciences of Europe No 170 (2025)

Review: A ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICIENT (PART 2)

  • 1. A ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICIENT (2) Dai Feng, Richard Baumgartner & Vladimir Svetnik Jan 18, 2019 Dai Feng, Richard Baumgartner & Vladimir SvetnikA ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICJan 18, 2019 1 / 16
  • 2. 1 INTRODUCTION 2 A ROBUST PARAMETRIC MODEL BASED ON MULTIVARIATE t-DISTRIBUTION 3 SIMULATION STUDY 4 REAL-LIFE EXAMPLES 5 CONCLUSION AND DISCUSSION Dai Feng, Richard Baumgartner & Vladimir SvetnikA ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICJan 18, 2019 2 / 16
  • 3. INTRODUCTION INTRODUCTION Dai Feng, Richard Baumgartner & Vladimir SvetnikA ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICJan 18, 2019 3 / 16
  • 4. A ROBUST PARAMETRIC MODEL BASED ON MULTIVARIATE t-DISTRIBUTION A ROBUST PARAMETRIC MODEL BASED ON MULTIVARIATE t-DISTRIBUTION Dai Feng, Richard Baumgartner & Vladimir SvetnikA ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICJan 18, 2019 4 / 16
  • 5. A ROBUST PARAMETRIC MODEL BASED ON MULTIVARIATE t-DISTRIBUTION 2.1. Robust Bayesian Method for the CCC Estimation Under the multivariate t-distribution, the CCC is defined as CCCt = 2 d−1 i=1 d j=i+1 ν ν−2 σij (d − 1) d i=1 ν ν−2 σ2 i + d−1 i=1 d j=i+1(µi − µj)2 (1) where ν = the degrees of freedom, µi s = components of the location vector, σ2 i s = diagonal elements of the scale matrix, σij s = off diagonal elements of the scale matrix. Dai Feng, Richard Baumgartner & Vladimir SvetnikA ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICJan 18, 2019 5 / 16
  • 6. A ROBUST PARAMETRIC MODEL BASED ON MULTIVARIATE t-DISTRIBUTION 2.1. Robust Bayesian Method for the CCC Estimation (conti.) Conjugate priors µ ∼ MVN(µ0, Σ0) Σ−1 ∼ Wishart(ρ, V) ν ∼ U(νmin, νmax) Noninformative priors µ0 = 0, Σ0 = very large, ρ = d, V = diagonal matrix, νmin = 4, νmax = 25 ← produces accurate estimates for the CCC in the scenarios they studied Dai Feng, Richard Baumgartner & Vladimir SvetnikA ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICJan 18, 2019 6 / 16
  • 7. A ROBUST PARAMETRIC MODEL BASED ON MULTIVARIATE t-DISTRIBUTION 2.1. Robust Bayesian Method for the CCC Estimation (conti.) Posterior distributions under the conjugate priors µ ∼ MVN(A−1 b, A−1 ) Σ−1 ∼ Wishart  ρ + n, V−1 + n i=1 λi (Yi − µ)(Yi − µ)T −1   λi ∼ Γ ν + d 2 , ν + (Yi − µ)T Σ−1 (Yi − µ) 2 where A = n i=1 λi Σ−1 + Σ−1 0 , b = Σ−1 n i=1 λi Yi + Σ−1 0 µ0. Dai Feng, Richard Baumgartner & Vladimir SvetnikA ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICJan 18, 2019 7 / 16
  • 8. A ROBUST PARAMETRIC MODEL BASED ON MULTIVARIATE t-DISTRIBUTION 2.1. Robust Bayesian Method for the CCC Estimation (conti.) The logarithm of the posterior distribution of ν is proportional to n i=1 ν 2 log ν 2 − log Γ ν 2 + ν 2 − 1 log λi − ν 2 λi and within the interval of [νmin, νmax]. To update ν with a bounded support, they use the slice sampling (Neal, 2003), which is easy to choose the magnitude of changes adaptively. To run multiple sequences of the MCMC, the MLE-based estimates could be used to create a starting point as suggested by Liu (1994). Dai Feng, Richard Baumgartner & Vladimir SvetnikA ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICJan 18, 2019 8 / 16
  • 9. A ROBUST PARAMETRIC MODEL BASED ON MULTIVARIATE t-DISTRIBUTION 2.1. Robust Bayesian Method for the CCC Estimation (conti.) Slice sampling (Neal, 2003) Neal, R. M. (2003). Slice Sampling. The Anals of Statistics 31:705-741. To implement Gibbs sampling, one may need to devise methods for sampling from nonstandard univariate distributions. To use the Metropolis algorithm, one must find an appropriate “proposal” distribution that will lead to efficient sampling. Simple forms of univariate slice sampling are an alternative to Gibbs sampling that avoids the need to sample from nonstandard distributions. It is easier to tune than Metropolis methods and also avoids problems that arise when the appropriate scale of changes varies over the distribution. Dai Feng, Richard Baumgartner & Vladimir SvetnikA ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICJan 18, 2019 9 / 16
  • 10. A ROBUST PARAMETRIC MODEL BASED ON MULTIVARIATE t-DISTRIBUTION 2.1. Robust Bayesian Method for the CCC Estimation (conti.) Slice sampling (Neal, 2003) The single variable slice sampling methods. x0 : current value , x1 : a new value 1 Draw a real value, y, uniformly from (0, f (x0)), thereby defining a horizontal “slice” : S = {x : y < f (x)}. 2 Find an interval, I = (L, R), around x0 that contains all, or much, of the slice. 3 Draw the new point, x1, from the part of the slice within this interval. Dai Feng, Richard Baumgartner & Vladimir SvetnikA ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICJan 18, 2019 10 / 16
  • 11. A ROBUST PARAMETRIC MODEL BASED ON MULTIVARIATE t-DISTRIBUTION 2.1. Robust Bayesian Method for the CCC Estimation (conti.) Slice sampling (Neal, 2003) Multivariate slice sampling methods. x0 = (x0,1, . . . , x0,n) : current value x1 = (x1,1, . . . , x1,n) : a new value 1 Draw a real value, y, uniformly from (0, f (x0)), thereby defining the slice S = {x : y < f (x)}. 2 Find a hyperrectangle, H = (L1, R1) × · · · × (Ln, Rn), around x0, which preferably contains at least a big part of the slice. 3 Draw the new point, x1, from the part of the slice within this hyperrectangle. Dai Feng, Richard Baumgartner & Vladimir SvetnikA ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICJan 18, 2019 11 / 16
  • 12. A ROBUST PARAMETRIC MODEL BASED ON MULTIVARIATE t-DISTRIBUTION 2.1. Robust Bayesian Method for the CCC Estimation (conti.) Slice sampling (Neal, 2003) Various other things Finding an appropriate hyperrectangle. Sampling from the part of the slice within the hyperrectangle. Overrelaxed slice sampling. Reflective slice sampling. Dai Feng, Richard Baumgartner & Vladimir SvetnikA ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICJan 18, 2019 12 / 16
  • 13. A ROBUST PARAMETRIC MODEL BASED ON MULTIVARIATE t-DISTRIBUTION 2.2 Robust Bayesian Estimation of the CCC with Accommodation of Covariates and Multiple Replications Dai Feng, Richard Baumgartner & Vladimir SvetnikA ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICJan 18, 2019 13 / 16
  • 14. SIMULATION STUDY SIMULATION STUDY Dai Feng, Richard Baumgartner & Vladimir SvetnikA ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICJan 18, 2019 14 / 16
  • 15. REAL-LIFE EXAMPLES REAL-LIFE EXAMPLES Dai Feng, Richard Baumgartner & Vladimir SvetnikA ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICJan 18, 2019 15 / 16
  • 16. CONCLUSION AND DISCUSSION CONCLUSION AND DISCUSSION Dai Feng, Richard Baumgartner & Vladimir SvetnikA ROBUST BAYESIAN ESTIMATE OF THE CONCORDANCE CORRELATION COEFFICJan 18, 2019 16 / 16