Group 1 ROC assignment explaining ROC and AUC

• RECEIVER OPERATING CHARACTERISTIC
CURVE
• GROUP 1
• EAB 1
• FEB 2025

OUTLINE
1. Introduction
2. Types of ROC Curves
3. Area Under the Curve (AUC) and
Interpretation
4. Practical Example and
Visualization
5. Partial AUC and Its Importance
6. Effect of Thresholds
7. Advantages and Disadvantages of
ROC Curves
8. Potential Issues in ROC Analysis
9. Comparison with Other Methods

Introduction
• The ROC curve is a statistical tool used to evaluate
the performance of a binary diagnostic classification
method.
• Many diagnostic tests provide continuous results,
requiring a cut-off value for decision-making.
• ROC analysis helps determine the optimal cut-off
point

Introduction
• Purpose of ROC Curves
• Assess the overall diagnostic performance of a test.
• Compare multiple diagnostic tests.
• Determine the optimal cut-off value to balance
sensitivity and specificity.

TPES OF ROC Curves
Parametric (Binary) Method
• Forms a smooth curve by expanding sample size and
connecting countless points
• Compare plots at any sensitivity and specificity values
• AUC may be biased if normality assumptions are
incorrect
Assumption
• Normal distribution of data
Nahm, F. S. (2022). Receiver operating characteristic curve : overview
and practical use for clinicians.

TPES OF ROC Curves
Non-Parametric (Empirical)
Method
• Produces a jagged or staircase-like
curve.
• Uses all observed data points.
• No normality assumption required,
making it more robust.

Receiver Operating Characteristic (ROC) Curve
Rajeev, K. A. I. (2025). Receiver Operating Characteristic (ROC) Curve for Medical
Researchers.

Area under the curve (AUC)
• AUC summarizes (quantifies) the overall accuracy of
a diagnostic test.
• Values range from 0 to 1:
 AUC=1 : perfectly accurate test
 AUC > 0.8: Good-Excellent test
 AUC 0.5: No discrimination (random guessing)
 AUC < 0.5: Worse than random guessing
 AUC=0: perfectly inaccurate test

Interpretation of the Area Under the Curve
Nahm, F. S. (2022). Receiver operating characteristic curve : overview and practical use for clinicians.

• Practical example and visualization

• Software
1. SPSS
2. Stata
3. R
4. SAS
5. Python
Data
• Cervical cancer screening: Pap-smear, HPV-test ,Colposcopy

PARTIAL AUC
• (ROC) curves with an equal AUC
• Although the AUC is the same, the features
of the ROC curves are not identical.
• Test B shows better performance in the high
false-positive rate range than test A
• Test A is better in the low false-positive
range.
• In this example, the partial AUC (pAUC) can
compare these two ROC curves at a specific
false positive rate range.
Nahm, F. S. (2022). Receiver operating characteristic curve :
overview and practical use for clinicians.

EFFECT OF THRESHOLDS
• For the same diagnostic test sensitivity and specificity
vary with the thresholds used.
• Generally:
• High threshold: good specificity, medium sensitivity
• Medium threshold: medium specificity, medium
sensitivity
•
• Low threshold: good sensitivity, medium specificity

common interpretations of AUC
i. the average value of sensitivity for all possible values of
specificity
ii. the average value of specificity for all possible values of
sensitivity
iii. the probability that a randomly selected patient with disease
has positive test result that indicates greater suspicion than a
randomly selected patient without disease when higher values
of the test are associated with disease and lower values are
associated with non disease.

The ROC curve : advantages.
• Displays all possible cut-off points and one can read the optimal cut-
off
• Independent of diseases prevalence, unlike predictive values
(PPV,NPV), therefore, samples can be taken regardless of the
prevalence of a disease in the population
• Allows comparison of multiple tests in one graph
• Sometimes sensitivity is more important than specificity or vice
versa- ROC helps in finding the required value of sensitivity at fixed
values of specificity
• Useful summary of measures can be obtained for determining the
validity of diagnostic test such as AUC and partial area under the
curve

The ROC curve : Disadvantages.
• The cut-off value for distinguishing normal from
abnormal is not directly displayed on the ROC curve
and neither is the number of samples.
• The ROC curve appears more jagged with a smaller
sample size, a larger sample does not necessarily
result in a smoother curve.

Potential ROC issues
• Lack of gold standard for diagnosis
• Lack of reproducibility
–E.g., disagreement among pathologists
• Bias in sample selection, spectrum of disease used in evaluating test
–Choose sickest patients, healthy controls
• Problems in ascertainment
–Genetic disease may not be manifest
• Can’t always reliably measure ROC area (Few cases with disease,

Solutions to Potential ROC issues
• Lack of gold standard for diagnosis
• composite reference standard: combining multiple diagnostic criteria
clinical findings, laboratory tests, imaging results, or expert opinions
• Lack of reproducibility
 Trainings.
 Multiple independent raters and measure agreement (e.g., Cohen’s kappa).
 Implement AI-assisted decision tools to reduce human variability

Solutions to Potential ROC issues
Bias in sample selection
• Ensure representative sampling
Problems in ascertainment
• Longitudinal follow-up to assess test performance over time.
• Implement repeat testing to detect cases that initially appear negative.
• Can’t always reliably measure ROC area
 Report confidence intervals for AUC instead of a single point estimate.
 Use bootstrapping to improve AUC estimation.

References
• Nahm, F. S. (2022). Receiver Operating Characteristic Curve:
Overview and Practical Use for Clinicians.

Group 1 ROC assignment explaining ROC and AUC

More Related Content

Similar to Group 1 ROC assignment explaining ROC and AUC (20)

Recently uploaded (20)

Group 1 ROC assignment explaining ROC and AUC

Editor's Notes