Methodology Checklist : Studies of Diagnostic Accuracy
Download as DOC, PDF1 like154 views
This document presents a methodology checklist for studies of diagnostic accuracy based on the work of the QUADAS2 team at Bristol University. The checklist consists of 4 domains to assess risk of bias and applicability: 1) patient selection, 2) index test, 3) reference standard, and 4) flow and timing. Each domain contains criteria to determine if a study has introduced bias or is applicable to the diagnostic accuracy question. The overall quality of studies should be assessed as high, acceptable, or unacceptable based on how many criteria are met.
Methodology Checklist : Studies of Diagnostic Accuracy
- 1. S I G N Methodology Checklist 5: Studies of Diagnostic Accuracy This checklist is based on the work of the QUADAS2 team at Bristol Univeristy (http://www.bris.ac.uk/quadas/). Study identification (Include author, title, reference, year of publication) Guideline topic: Key Question No: Before completing this checklist, consider: 1. Is the paper really a study of diagnostic accuracy? It should be comparing a specific diagnostic test against another, and not a general paper or comment on diagnosis. 2. Is the paper relevant to key question? Analyse using PICO (Patient or Population Intervention Comparison Outcome). IF NO REJECT (give reason below). IF YES complete the checklist.. Reason for rejection: Reason for rejection: 1. Paper not relevant to key question □ 2. Other reason □ (please specify): Checklist completed by: All the questions in the following sections have associated footnotes providing short explanations behind each of the questions. Users who want more detailed explanations should consult the QUADAS-2: Background Document. DOMAIN 1 – PATIENT SELECTION Risk of bias In a well conducted diagnostic study… Is that true in this study? 1.1 A consecutive sequence or random selection of patients is enrolled.i Yes No Can’t say 1.2 Case – control methods are not used.ii Yes No Can’t say 1.3 Inappropriate exclusions are avoided.iii Yes No Can’t say Applicability 1.4 The included patients and settings match the key question.iv Yes No Can’t say DOMAIN 2 – INDEX TEST Risk of bias In a well conducted diagnostic study… Is that true in this study? 2.1 The index test results interpreted without knowledge of the results of the reference standard.v Yes No Can’t say © Scottish Intercollegiate Guidelines Network, January 2006
- 2. 2.2 If a threshold is used, it is pre-specified.vi Yes No Can’t say © Scottish Intercollegiate Guidelines Network, January 2006
- 3. Applicability 2.3 The index test, its conduct, and its interpretation is similar to that used in practice with the target population of the guideline.vii Yes No Can’t say DOMAIN 3 – REFERENCE STANDARD Risk of bias In a well conducted diagnostic study… Is that true in this study? 3.1 The reference standard is likely to correctly identify the target condition.viii Yes No Can’t say 3.2 Reference standard results are interpreted without knowledge of the results of the index test.ix Yes No Can’t say Applicability 3.3 The target condition as defined by the reference standard matches that found in the target population of the guideline.x Yes No Can’t say DOMAIN 4 – FLOW AND TIMING Risk of bias In a well conducted diagnostic study… Is that true in this study? 4.1 There is an appropriate interval between the index test and reference standard.xi Yes No Can’t say 4.2 All patients receive the same reference standard.xii Yes No Can’t say 4.3 All patients recruited into the study are included in the analysis.xiii Yes No Can’t say SECTION 5: OVERALL ASSESSMENT OF THE STUDY 5.1 How well was the study done to minimise bias? Code as follows:xiv High quality (++) Acceptable (+) Unacceptable – reject 0 5.2 What is your assessment of the applicability of this study to our target population? Directly applicable Some indirectness (Please explain in the following section for Notes) 5.2 Notes. Summarise the authors conclusions. Add any comments on your own assessment of the study, and the extent to which it answers your question. © Scottish Intercollegiate Guidelines Network, January 2006
- 4. i Studies should enrol either all eligible patients suspected of having the target condition during a specified period, or a random sample of those patients. The essential point is that investigators should have no freedom of choice as to which individual patients are or are not included. ii There is evidence that studies comparing patients with known disease with a control group without the condition tend to exaggerate diagnostic accuracy. iii Inappropriate exclusions may result in either overestimates (eg by excluding ‘difficult to diagnose’ patients) or underestimates (eg by excluding patients with ‘red flags’ suggesting presence of disease) of the degree of diagnostic accuracy. iv Patients included in the study should match the target population of the guideline in terms of severity of the target condition, demographic features, presence of differential diagnosis or co-morbidity, setting of the study and previous testing protocols. v This is similar to the question of ‘blinding’ in intervention studies. The index test should always been done first, or by a separate investigator with no knowledge of the outcome of the reference test. vi Bias can be introduced if a threshold level is set after data has been collected. Any minimum threshold should be specified at the start of the trial. vii Variations in test technology, execution, or interpretation (eg use of a higher ultrasound transducer frequency) may affect estimates of diagnostic accuracy. viii Estimates of test accuracy are based on the assumption that the reference standard is 100% sensitive (=accurately diagnoses the target condition). ix This is the similar to question 2.1, but in this case relates to making sure the reference standard is applied without any prior knowledge of the outcome of previous tests. x The definition of the target condition used when testing the reference standard may differ from that used by the NHS in Scotland. eg threshold levels used in laboratory cultures may differ. xi The index test and reference standard should be performed as close together in time as possible, otherwise changes in the patients condition is likely to invalidate the results. xii In some cases the choice of reference standard may be influenced by the outcome of the index test or the urgency of the need for diagnosis. Use of different reference standards is likely to lead to overestimates of both sensitivity and specificity. xiii Not including all patients in the analysis may lead to bias as there may be some systematic difference between those lost to follow-up and those analysed. xiv Rate the overall methodological quality of the study, using the following as a guide: High quality (++): Majority of criteria met. Little or no risk of bias. Results unlikely to be changed by further research. Acceptable (+): Most criteria met. Some flaws in the study with an associated risk of bias, Conclusions may change in the light of further studies. Low quality (0): Either most criteria not met, or significant flaws relating to key aspects of study design. Conclusions likely to change in the light of further studies.