Validity and Reliability
7 likes4,625 views
This document discusses the validity and reliability of analytical tests used for screening and diagnosis. It defines key terms like sensitivity, specificity, predictive value and discusses how changing cutoff levels can impact false positives and negatives. Screening tests are used to separate populations into those with and without a disease, while considering a test's accuracy. Continuous variable tests may require an artificial cutoff versus dichotomous screening tests. The document also examines how prevalence impacts predictive value and how using multiple screening tests can improve accuracy.
Validity and Reliability
- 1. Validity and Reliability of Analytical Tests BY DR MUHAMMAD TAUSEEF JAVED CONSULTANT MINISTRY OF HEALTH MAKKAH & IPH LAHORE
- 2. Analytical Tests include both: • Screening Tests • Diagnostic Tests
- 3. •Two Important Objectives To distinguish between people in the population who have the diseases and those who do not To determine how good the test is in separating populations of people with and without the disease in question?
- 5. Epidemiological Surveillance • What is it? • Why do it?
- 6. Epidemiological Surveillance • Definition - ongoing & systematic collection, analysis & interpretation of data related to health, disease & conditions • Two types Passive Surveillance – uses available data or reporting from health care provider or regional health officer Active Surveillance – periodic field visits to health care facilities to identify new cases • The present approach is the survey
- 7. Epidemiological Surveillance • Why do it? Can help discover and control the transmission of infectious diseases Prevention and control programs can be planned and implemented
- 8. Screening • Definition - use of quick and simple testing procedures to identify and separate persons: who have a disease from those that do not OR who are apparently (appear to be) well, but who may be at risk of a disease, from those who probably don’t have the disease.
- 9. Terms Related to Screening Tests • Validity - relates to accuracy (correctness) • Reliability - repeatability • Yield - the # of tests that can be done in a time period
- 10. Terms Related to Screening Tests (cont’d) • Sensitivity - ability of a test to identify those who have disease • Specificity - ability of a test to exclude those who don’t have disease
- 11. Terms Related to Screening Tests (cont’d) • Tests with dichotomous results – tests that give either positive or negative results • Tests of continuous variables – tests that do not yield obvious “positive” or “negative” results, but require a cutoff level to be established as criteria for distinguishing between “positive” and “negative” groups
- 12. An important public health consideration, particularly in screening free-living populations, is: How good is the test at identifying people with the disease and without the disease? In other words: If we screen a population, what proportion of people who have the disease will be correctly identified?
- 13. POPULATION Test Results With Disease Without Disease Positive True Positive (TP) False Positive (FP) Negative False Negative (FN) True Negative (TN)
- 14. Sensitivity = = X 100 True positives True positives + false negatives True positives All persons with the disease = TP TP + FN
- 15. Specificity = = X 100 True negatives True negatives+ false positives True negatives All persons without the disease = TN TN + FP
- 16. Percent false negatives = % of people with the disease who were not detected by the test FN FN + TP X 100
- 17. Percent false positives = % of people without the disease who were incorrectly labeled by the test as having the disease FP FP + TN X 100
- 18. In the clinical setting, a more important question is: If the test results are positive (or negative) in a given patient, what is the probability that this patient has (or does not have) the disease? In other words: What proportion of patients who test positive (or negative) actually have (or do not have) the disease in question?
- 19. Predictive Value Pos. PV = X 100 = %True Positives TP + FP Neg. PV = X 100 = %True Negatives TN + FN
- 20. Biologic Variation of Human Populations & Diagnostic Issues
- 21. Distribution of Tuberculin Reactions Bimodal Distribution Easy to distinguish between exposed group and those not exposed.
- 22. Distribution of Systolic Blood Pressure Unimodal Distribution With continuous variables, a cutoff level must be established to separate the hypertensive group. Could choose based on statistics, but better to base on biologic considerations.
- 23. Effects of Choosing Different Cutoff Levels for Diabetes Diagnosis in Population with 50% Prevalence Real World Pseudo- Real World
- 24. The major issue with deciding to set a cutoff high or low is the problem of false positives and false negatives.
- 25. Possible Groups with Dichotomous Test True Disease Status is Known, as with dichotomous tests.
- 26. Grouping All Positives and All Negatives True Disease Status is Unknown, as with continuous variables. Artificial Cutoff
- 27. •Use of Multiple Screening Tests Sequential (Two-stage) Testing Simultaneous Testing
- 28. Hypothetical Two-Stage Screening Only Pos. Test 1 are given Test 2
- 29. Hypothetical Two-Stage Screening (cont.) TEST 2 (Glucose Tolerance Test) Sensitivity = 90% Specificity = 90% DIABETES + - TEST RESULTS + 315 190 505 - 35 1710 1745 350 1900 2250 Net Sensitivity = 315/500 = 63% Net Specificity = 7600 + 1710 = 98% 9500
- 30. Predictive Value
- 31. Prevalence & Predictive Value Positive As prevalence increases, positive predictive value increases.
- 32. Prevalence & Predictive Value Note: Test has 95% sensitivity and 95% specificity
- 33. Specificity & Predictive Value As specificity increases, positive predictive value increases. As sensitivity increases, positive predictive value also increases, but to a much lesser extent.
- 34. Specificity & Predictive Value As specificity increases, positive predictive value increases.
- 35. Results reliable but NOT valid Results reliable and valid Reliability (Repeatability) of Tests