Putting search theory to work on large datasets
Download as PPTX, PDF0 likes291 views
The document discusses the need for improved search strategies for systematic reviews. It notes that keyword searching alone is not sufficient and that professional search needs to be comprehensive, repeatable, and transparent. Currently, search strategies are error-prone, difficult to communicate and scale. The document proposes visualizing search strategies to make them easier to develop, debug, maintain and share. Diagramming search concepts and relationships could improve accuracy and efficiency over textual search strings.
Putting search theory to work on large datasets
- 1. Dr.Tony Russell-Rose FBCS CITP CEng
- 2. We all need search but... Some need it more than others Patent, healthcare, recruitment, media… Keyword search is not good enough Professional search needs to be: Comprehensive Repeatable Transparent Think outside the search box – www.2dsearch.com 2
- 3. Think outside the search box – www.2dsearch.com 3
- 4. 1. randomized controlled trial.pt. 2. controlled clinical trial.pt. 3. randomized.ab. 4. placebo.ab. 5. clinical trials as topic.sh. 6. randomly.ab. 7. trial.ti. 8. 1 or 2 or 3 or 4 or 5 or 6 or 7 9. (animals not (humans and animals)).sh. 10. 8 not 9 11. exp Child/ 12. ADOLESCENT/ 13. exp infant/ 14. child hospitalized/ 15. adolescent hospitalized/ 16. (child$ or infant$ or toddler$ or adolescen$ or teenage$).tw. Think outside the search box – www.2dsearch.com 4 17. or/11-16 18. Child Nutrition Sciences/ 19. exp Dietary Proteins/ 20. Dietary Supplements/ 21. Dietetics/ 22. or/18-21 23. exp Infant, Newborn/ 24. exp Overweight/ 25. exp Eating Disorders/ 26. Athletes/ 27. exp Sports/ 28. exp Pregnancy/ 29. expViruses/ 30. (newborn$ or obes$ or "eating disorder$" or pregnan$ or childbirth or virus$ or influenza).tw. 31. or/23-30 32. 10 and 17 and 22 33. 32 not 31 Francis, D. K., Smith, J., Saljuqi,T., &Watling, R. M. (2015). Oral protein calorie supplementation for children with chronic disease. Cochrane Database of Systematic Reviews, (5).
- 5. Think outside the search box – www.2dsearch.com 5 Java AND (Design OR develop OR code OR Program) AND ("* Engineer" OR MTS OR "* Develop*" OR Scientist OR technologist) AND (J2EE OR Struts OR Spring) AND (Algorithm OR "Data Structure" OR PS OR “Problem Solving”)
- 6. 1 A01N0025-004/CPC 2 RODENT OR RAT OR RATS OR MOUSE OR MICE 3 BAIT OR POISON 4 2 AND 3 5 1 OR 4 6 AVERSIVE OR ADVERSIVE OR DETER? OR REPEL? 7 NONTARGET OR (NON WITH TARGET) OR HUMAN OR DOMESTIC OR PET OR DOG OR CAT 8 6 AND 7 9 8 AND 5 10 BITREX OR DENATONIUM OR BITREXENE OR BITTERANT OR BITTER 11 10 AND 5 12 9 OR 11 Think outside the search box – www.2dsearch.com 6 ( ( (~Magnolia OR ~Magnolias) AND ('~Paul ~Anderson' OR '~Paul ~Thomas' OR paul* OR thom* OR anders*) ) OR 'jeremy blackman*' OR jeremy OR Jerem* OR ( (~incredible OR ~coincidences) AND (role* OR chance* OR life* OR investigat* OR disturb*) ) OR ( (~C NOT ('~John C' OR ( ( ('~Mister C' AND ~Shamen) OR ('~Tom C' AND ~Scientology) ) ) ) ) ) OR ~XCs OR ~CV OR ~CVs OR ( ( (~CKR OR ~CKRs) NOT ('CKR 123' OR lawyer) ) ) )
- 7. (topic:"d5f485b9-aab0-4d95-a83b-28b2decd900c" AND (topic:"ba942587-854e-407d-81dc-812ee29624bd" OR topic:"805a2281-8f04-42b7-a451-e2b7daac4db1" OR topic:"1fc2a4f5- 3f23-4c80-943c-42f789d11644" OR topic:"91e4df78-db55-4d63-be65-2fad1afa77a7" OR organization:"Argos"OR organization:"Sainsbury"OR organization:"Marks and Spencer" OR organization:"Morrisons" OR organization:"Signet" OR organization:"Rapha"OR organization:"Vodafone" OR organization:"Evans Cycle" OR organization:"The Body Shop" OR organization:"Homebase" OR organization:"Topps Tiles" OR organization:"Carphone Warehouse"OR organization:"B&Q" OR organization:"Tesco" OR organization:"Lidl"OR organization:"Aldi"OR organization:"Waitrose"OR organization:"John Lewis" OR organization:"Dixons" OR organization:"Alliance Boots" OR organization:"AS Watson" OR organization:"Halfords"OR organization:"Vision Express" OR organization:"Pets at Home" OR organization:"Asda" OR organization:"Pret a Manger" OR organization:"Greggs" OR organization:"Majestic Wine" OR organization:"Ocado" OR organization:"Selfridges" OR organization:"Nike" OR organization:"Adidas" OR organization:"Walmart" OR organization:"Woolworths"OR organization:"Debenhams" OR organization:"Halfords"OR organization:"Punch Taverns" OR organization:"Monarch Group" OR organization:"Whitbread" OR organization:"Carnival Cruise" OR organization:"KFC" OR organization:"Yum!" OR organization:"Costa Coffee" OR organization:"Pizza Hut" OR organization:"Greene King" OR organization:"Carluccio's" OR organization:"Gondola"OR organization:"Starbucks"OR organization:"Sodexo" OR organization:"SSP" OR organization:"Nando's" OR organization:"Avis"OR organization:"Paddy Power" OR organization:"Easyjet" OR organization:"Emaar" OR organization:"Nuffield Health" OR organization:"Safestore" OR organization:"TUI" OR organization:"Merlin Entertainment" OR organization:"Virgin Atlantic" OR organization:"IHG" OR organization:"Virgin Active" OR organization:"Thomas Cook" OR organization:"Odeon Hilton" OR organization:"Pizza Express" OR organization:"Zizzi's" OR organization:"Gemfields"OR organization:"Orlebar Brown" OR organization:"Rebecca Trowell" OR organization:"Spring Studios" OR organization:"Loewe" OR organization:"Jimmy Choo" OR organization:"Burberry" OR organization:"Conde Nast" OR organization:"Zadig & Voltaire" OR organization:"Prada"OR organization:"Fendi"OR organization:"French Connection" OR organization:"Lyle & Scott" OR organization:"LK Bennett" OR organization:"Alexander Wang" OR organization:"Sandro"OR organization:"Isabel Marant" OR organization:"Rag & Bone" OR organization:"J Crew" OR organization:"Maje" OR organization:"Ralph Lauren" OR organization:"Tommy Hilfiger" OR organization:"Michael Kors" OR organization:"Kenzo"OR organization:"Pringle of Scotland" OR organization:"Heidi Klein" OR organization:"Louis Vuitton" OR organization:"Lanvin" OR organization:"Loewe" OR organization:"Chanel" OR organization:"Valentino" OR organization:"Celine" OR organization:"Dior" OR organization:"Alexander McQueen" OR organization:"Victoria Beckham"OR organization:"Sonia Rykel"OR organization:"Bally" OR organization:"Notonthehighstreet" OR organization:"Rupert Sanderson" OR organization:"Dunhill" OR organization:"De Beers" OR organization:"Swarovski"OR organization:"Hermes" OR organization:"Cartier" OR organization:"David Yurman" OR organization:"Mulberry" OR organization:"Tiffany & Co" OR organization:"Boucheron" OR organization:"Lancel" OR organization:"Marc Jacobs" OR organization:"Longchamp" OR organization:"Kate Spade" OR organization:"Salvatore Ferragamo" OR organization:"Asos"OR organization:"Net a porter" OR organization:"Astley Clarke" OR organization:"Fenwick" OR organization:"Yoox" OR organization:"Harrods" OR organization:"Selfridges" OR organization:"MyTheresa" OR organization:"Lane Crawford" OR organization:"Liberty of London" OR organization:"Gucci"OR organization:"Armarni"OR organization:"Zara"OR organization:"Topshop"OR organization:"Arcadia" OR organization:"Balenciaga" OR organization:"Stella McCartney" OR organization:"Brioni"OR organization:"Bottega Veneta" OR organization:"Christopher Kane" OR organization:"Anya Hindmarch" OR organization:"Business of Fashion" OR organization:"3i" OR organization:"Active Private Equity" OR organization:"Advent International" OR organization:"Advent" OR organization:"ApolloGlobal Management" OR organization:"Apollo"OR organization:"Apax" OR organization:"Apax" OR organization:"Augmentum Capital"OR organization:"Bain" OR organization:"Endless" OR organization:"BC Partners" OR organization:"Better Capital" OR organization:"HuttonCollins" OR organization:"Hutton Collins" OR organization:"Bridgepoint" OR organization:"Business Growth Fund" OR organization:"CDR" OR organization:"Darwin" OR organization:"Duke Street Capital" OR organization:"Electra Partners" OR organization:"Exponent" OR organization:"European Capital" OR organization:"General Atlantic" OR organization:"Hilco" OR organization:"Inflexion" OR organization:"KKR"OR organization:"LGV"OR organization:"Lion" OR organization:"Magenta Partners" OR organization:"Montagu" OR organization:"Octopus"OR organization:"Octopus Capital" OR organization:"Piper Private Equity" OR organization:"Piper" OR organization:"Sun Capital" OR organization:"TA Associates" OR organization:"Warburg Pincus" OR organization:"Charterhouse" OR organization:"Primary Capital" OR organization:"Bowmark"OR organization:"Beringea" OR organization:"Business Growth Fund" OR organization:"Bridgepoint" OR organization:"Caird Capital" OR organization:"CapVest"OR organization:"Cerberus Capital" OR organization:"CBPE" OR organization:"CCMP" OR organization:"CDR" OR organization:"Change Capital" OR organization:"Cinven" OR organization:"CVC" OR organization:"Duke Street" OR organization:"EC1" OR organization:"Electra" OR organization:"Encore" OR organization:"EQT" OR organization:"Equistone" OR organization:"Graphite Capital" OR organization:"Caledonian"OR organization:"Kings Park Capital" OR organization:"L Capital" OR organization:"LDC" Think outside the search box – www.2dsearch.com 7
- 8. Evidence based medicine “A systematic review summarises the results of healthcare studies (controlled trials) and provides a high level of evidence on the effectiveness of healthcare interventions. Judgments may be made about the evidence and inform recommendations for healthcare.” Cochrane Consumer Network, https://consumers.cochrane.org/what-systematic- review Think outside the search box – www.2dsearch.com 8
- 9. Think outside the search box – www.2dsearch.com 9 Growth in systematic reviews over 25 years (N=233,785 MEDLINE citations) BekhuisT,Tseytlin E, Mitchell KJ. A Prototype for a Hybrid System to Support Systematic ReviewTeams:A Case Study of Organ Transplantation. Proceedings IEEE International Conference on Bioinformatics and Biomedicine. 2015;2015:940- 947.
- 10. Think outside the search box – www.2dsearch.com 10 Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med 2010 Sep 21;7(9) The number of published trials, 1950 to 2007
- 11. Think outside the search box – www.2dsearch.com 11 Tang, K., Gerling, K., Chen,W., & Geurts, L. (2019). Information and Communication Systems toTackle Barriers to Breastfeeding: Systematic Search and Review. Journal of Medical Internet Research, 21(9), e13947. PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) is an evidence-based minimum set of items aimed at helping authors to report a wide array of systematic reviews and meta- analyses that assess the benefits and harms of a health care intervention. https://en.wikipedia.org/wiki/Preferred_Repo rting_Items_for_Systematic_Reviews_and_ Meta-Analyses
- 12. Ineffective for communicating structure Visual cues needed Poor scalability No abstraction Error-prone Syntactic + semantic errors Salvador-Oliván, José Antonio, Gonzalo Marco-Cuenca, and Rosario Arquero-Avilés. "Errors in search strategies used in systematic reviews and their effects on information retrieval." Journal of the Medical Library Association: JMLA 107.2 (2019): 210. Think outside the search box – www.2dsearch.com 12
- 13. Auditing published search strategies Sampson M, McGowan J. Errors in search strategies were identified by type and frequency. J Clin Epidemiol 2006 Oct;59(10):1057-1063 Salvador-Oliván, J.A., Marco-Cuenca, G. and Arquero-Avilés, R., 2019. Errors in search strategies used in systematic reviews and their effects on information retrieval. Journal of the Medical Library Association: JMLA, 107(2), p.210. 63 MEDLINE search strategies examined 90.5% contained ≥1 errors 82.5% contained errors that could potentially lower recall of relevant studies ▪ missed MeSH terms (44.4%) ▪ unwarranted explosion of MeSH terms (38.1%) ▪ irrelevant MeSH or free text terms (28.6%) ▪ Missed spelling variants, combining MeSH and free text terms in the same line, and failure to tailor the search strategy for other databases (20.6%) ▪ Logical operator error occurred in 19.0% of searches Think outside the search box – www.2dsearch.com 13
- 14. Search strategies are: Complex, logical constructs Read many more times that they are written Exhibit abstraction, modularity, encapsulation Maintained, shared, optimised… “For the IBM PC in the early 1980s, the software development process was text edit, compile, write down the errors, and debug with your eyes.” The History ofVisual Development Environments, https://www.mendix.com/blog/the-history-of- visual-development-environments-imagine-theres-no-ides-its-difficult-if-you-try/ Think outside the search box – www.2dsearch.com 14
- 15. What if we could visualize search strategies? Use metaphors the user already understands Separate keyword selection from query manipulation Provide a ‘scratchpad’ & ‘debugging aids’ Provide instant feedback Support abstraction (levels of composition) Provide query suggestions Think outside the search box – www.2dsearch.com 15
- 16. Think outside the search box – www.2dsearch.com 16
- 17. Think outside the search box – www.2dsearch.com 17
- 18. Think outside the search box – www.2dsearch.com 18
- 19. Think outside the search box – www.2dsearch.com 19
- 20. Concepts are objects on a 2D canvas Relationships via direct manipulation Syntactically correct expressions Transparent semantics Arbitrary levels of abstraction ‘Lego’ analogy Optimisation thru exploration ▪ Execute, enable/disable ▪ Hit counts Think outside the search box – www.2dsearch.com 20
- 21. Integrations 8 databases & counting... Universal representation Auto-translation Error-checking Interoperability with legacy formats Boolean string I/O Think outside the search box – www.2dsearch.com 21
- 22. Think outside the search box – www.2dsearch.com 22
- 23. Term selection is common source of inefficiency & error Apply query suggestions? (“business analyst” or “systems analyst” or “system analyst” or “data analyst” or “requirements analyst” or “functional analyst”) and crystal and report* and analy* and data near analy* and not inventory and not retail and not (ecommerce or “e-commerce” or b2b or b2c) Think outside the search box – www.2dsearch.com 23
- 24. Think outside the search box – www.2dsearch.com 24
- 25. Think outside the search box – www.2dsearch.com 25 T.G. Russell-Rose and Phil Gooch ,"2dSearch: a visual approach to search strategy formulation", Proceedings of DESIRES: Design of Experimental Search & Information REtrieval Systems, Bertinoro, Italy, 28-31 August 2018
- 26. Open content repository: Templates and best practices Library of shared strategies (aka ‘GitHub’) Community of practice (aka ‘Stack Overflow’) Think outside the search box – www.2dsearch.com 26 Technology: Search strategy formulation Community: Sharing & review Content: Samples & templates
- 27. User experience Visual methods vs. traditional query builders Quality of query suggestions Data science Semi-automated strategy generation Investigation of ConceptNet,Wikidata, etc. Combining set retrieval with ranked retrieval Contextual language modelling (BERT, ELMo, etc.) Think outside the search box – www.2dsearch.com 27
- 28. Reimagining ‘advanced search’ Eliminate syntactic error Make semantics transparent Provide support for reuse, optimisation, best practice Try it for yourself: https://www.2dsearch.com Tutorial videos: https://www.2dsearch.com/help Think outside the search box – www.2dsearch.com 28
- 29. Dr.Tony Russell-Rose, FBCS CITP CEng Senior Lecturer, Goldsmiths Founder: 2Dsearch, Director: UXLabs RAEVisiting Professor, Essex University Web: http://www.2dsearch.com Email: info@2dsearch.com Blog: http://isquared.wordpress.com LinkedIn: http://uk.linkedin.com/in/tonyrussellrose Twitter: @tonygrr, @2d_search 29Think outside the search box – www.2dsearch.com