Searching for Relevant Studies Quiz
- 1. Searching for Relevant Studies Interactive Quiz Prepared for: The Agency for Healthcare Research and Quality (AHRQ) Training Modules for Systematic Reviews Methods Guide www.ahrq.gov
- 2. You are interested in evaluating the benefits and harms associated with the use of thrombectomy devices in acute coronary syndromes. You need to determine what literature is available to conduct your review. Overview
- 3. A comprehensive search is required. When conducting this search, what do you need to assure? You balance precision and recall. You have gotten every possible relevant citation, regardless of the workload involved. You limit the search so that only relevant citations are found. A Comprehensive Search
- 4. Given the need to balance precision and recall, what should you do before initiating a search on the use of thrombectomy in patients with acute coronary syndromes? Enter the search as “thrombectomy AND acute coronary syndromes.” Enter the search as “thrombectomy OR acute coronary syndromes.” Understand the topic, devise an analytic framework, pose clearly defined key questions, and understand the scope of the review before devising your strategy. The First Step
- 5. Your due diligence pays off. You now realize that your initial search “thrombectomy AND acute coronary syndrome” would compromise the recall. Using “((Thrombectomy OR MerciClot.mp OR “other devices names”) AND (acute coronary syndromes OR myocardial infarction OR angina, unstable))” would assure better recall without appreciably impacting precision. The Power of Due Diligence
- 6. You try to decide whether to perform one search or two searches (one for “benefits” and another for “harms”). Which of the following would cause you to perform two searches? Performing two searches is redundant and should be avoided. You decide a priori to allow only data from randomized controlled trials (RCTs) when evaluating benefits but allow RCTs and observational trials for harms. The available literature base on the topic is small. More Than One Search May Be Needed
- 7. You need to decide which databases to use. The minimum requirement for a thorough search of the literature would include which two databases? MEDLINE + TOXNET MEDLINE + CINAHL MEDLINE + Cochrane Central Database Selection
- 8. Why would you decide to use hand searching of references from identified systematic reviews, studies, and abstract booklets from prominent meetings within the field of interest? Hand searching can capture citations that are not indexed or are improperly indexed. Limiting hand searching to these sources can target this time-intensive activity to areas where the yield will be the greatest. Both are correct. Hand Searching
- 9. You decide to search the U.S. Food and Drug Administration Web site and ClinicalTrials.gov to identify studies that might be appropriate for your systematic review. These Web sites are sources of what kind of documents? Black literature Grey literature SIPs Other Documents
- 10. You conduct a thorough search of MEDLINE and Cochrane Central, use two searches (one for “benefits” and another for “harms”), and employ targeted hand searching and grey literature evaluation. What is the last thing you need to remember to do that is germane to searching? Choose only citations that agree with your preconceived notions. Selectively report aspects of your search strategy. Provide transparent reporting of search strategies and the citations you identified. The Final Task
- 11. Literature searching can have many pitfalls. Readers will have confidence in the results of your systematic review if you: demonstrate a comprehensive understanding of the topic, use multiple searches and databases, use hedges, use hand searching and grey literature, and develop a transparent reporting structure. Summary
- 12. This quiz was p repared by C. Michael White, Pharm.D., FCP, FCCP, a member of the University of Connecticut/Hartford Hospital Evidence-based Practice Center. T his module is based on chapter 5 in version 1.0 of the Methods Reference Guide for Effectiveness and Comparative Effectiveness Reviews (available at: http://effectivehealthcare.ahrq.gov/repFiles/ 2007_10Draft MethodsGuide.pdf) . Author
Editor's Notes
- #2: Searching for Relevant Studies Interactive Quiz This interactive quiz augments the module on searching for relevant studies. It was prepared by C. Michael White, Pharm.D., FCP, FCCP, a member of the University of Connecticut/Hartford Hospital Evidence-based Practice Center. The module is based on chapter 5 in version 1.0 of the Methods Reference Guide for Effectiveness and Comparative Effectiveness Reviews (available at: http://effectivehealthcare.ahrq.gov/repFiles/2007_10DraftMethodsGuide.pdf).
- #3: Overview
- #4: A Comprehensive Search A comprehensive search is required. When conducting this search, what do you need to assure? Correct. Precision is the proportion of retrieved articles that are relevant, while recall is the proportion of potentially relevant articles retrieved by the search. You want to employ methods to balance precision and recall. Overly restrictive searches increase recall but compromise precision. Overly broad searches increase precision but compromise recall. Incorrect. Although it is laudable to want to acquire every possible relevant citation, this approach compromises precision and makes the systematic review unnecessarily onerous. You need to balance precision and recall. Incorrect. Every citation found is relevant, but it is likely that many relevant citations would be missed by this overly restrictive search strategy. You need to balance precision and recall.
- #5: The First Step Given the need to balance precision and recall, what should you do before initiating a search on the use of thrombectomy in patients with acute coronary syndromes? Incorrect. Searching based on only readily known disease and treatment information would be problematic. You need to understand the topic area before starting your search. Incorrect. This would be a voluminous search for citations for thrombectomy anywhere in the body in addition to any citation published on acute coronary syndromes. Correct. Understanding the topic and devising an analytic framework drive the development of clearly defined key questions. Combining this with understanding of the scope of the project allows an optimal balance between precision and recall when sculpting the search.
- #6: The Power of Due Diligence
- #7: More Than One Search May Be Needed You try to decide whether to perform one search or two searches (one for “benefits” and another for “harms”). Which of the following would cause you to perform two searches? Incorrect. If the size of the available literature is small, it may be appropriate to conduct one single, broad search. However, if the literature is extensive — and observational trials are allowed in the harms but not in the benefits evaluations — a single, broad search would reduce precision for studies reviewing efficacy end points without appreciably improving the recall. Correct. Benefits searches are usually limited to randomized controlled trials, because such trials have greater internal validity. Searches usually include the disease in question, the intervention, and a hedge for controlled trials. Harms searches are usually broader and include observational trials. Searches usually include the harm (or subheadings for general harms) and the intervention. When the available literature is extensive, conducting two searches improves precision without appreciably impacting recall. Incorrect. If the size of the available literature is small, it may be appropriate to conduct one single, broad search. However, if the literature is extensive — and observational trials are allowed in the harms but not in the benefits evaluations — a single, broad search would reduce precision for studies reviewing efficacy end points without appreciably improving the recall.
- #8: Database Selection You need to decide which databases to use. The minimum requirement for a thorough search of the literature would include which two databases? Incorrect. Searching MEDLINE and Cochrane Central is needed to fulfill the minimum, broad search requirements. TOXNET is a database that focuses on adverse events or toxicology and can be used in addition to these others databases. Incorrect. Searching MEDLINE and Cochrane Central is needed to fulfill the minimum, broad search requirements. CINAHL is a database that focuses on nursing literature and can be used in addition to these other databases. Correct. Combining MEDLINE with Cochrane Central is the minimum requirement for a thorough search of the literature. Using MEDLINE alone, however, may miss between 12 and 25 percent of relevant citations found in other databases and may produce a North American bias.
- #9: Hand Searching Why would you decide to use hand searching of references from identified systematic reviews, studies, and abstract booklets from prominent meetings within the field of interest? Correct, but there is a better answer. Hand searching can yield up to 13 to 25 percent of citations that do not come up in database searches. Because hand searching is the most time-intensive search strategy, it should be targeted to maximize the yield. Correct, but there is a better answer. Hand searching can yield up to 13 to 25 percent of citations that do not come up in database searches. Because hand searching is the most time-intensive search strategy, it should be targeted to maximize the yield.
- #10: Other Documents Incorrect. You chose the wrong color. Correct. Grey literature — which is produced at all levels of government, academics, business, and industry and is not controlled by commercial publishers — can be important for identifying trials that were conducted but are, as of now, unpublished or incompletely published. Searching for grey literature is a useful way to minimize publication bias. Incorrect. SIPs are scientific information packets, which are provided by the manufacturer of a drug or device.
- #11: The Final Task You conduct a thorough search of MEDLINE and Cochrane Central, use two searches (one for “benefits” and another for “harms”), and employ targeted hand searching and grey literature evaluation. What is the last thing you need to remember to do that is germane to searching? Choose only citations that agree with your preconceived notions. Incorrect. In order to build confidence in the systematic review results, transparent and defensible methods need to be employed. You need to report enough information about your search strategy so that a person skilled in literature searching could reproduce it. Correct. Transparent reporting is important in supporting the credibility of systematic review results. Reporting of search strategies and disposition of the citations should allow a person skilled in literature searching to reproduce the search you conducted.
- #12: Summary
- #13: Author This interactive quiz augments the module on searching for relevant studies. It was prepared by C. Michael White, Pharm.D., FCP, FCCP, a member of the University of Connecticut/Hartford Hospital Evidence-based Practice Center. The module is based on chapter 5 in version 1.0 of the Methods Reference Guide for Effectiveness and Comparative Effectiveness Reviews (available at: http://effectivehealthcare.ahrq.gov/repFiles/2007_10DraftMethodsGuide.pdf).