Individual Differences, User Perceptions and Eye Gaze in Biomedical Search Interfaces
1 like872 views
This study examines how individual differences, user perceptions, and eye gaze relate to the use of different biomedical search interfaces. It finds that searchers look most at abstracts and that interfaces and perceptions of task difficulty affect what elements are viewed. There are also interaction effects between cognitive style/search experience and interfaces for Medical Subject Headings area of interest. The findings suggest design of search engine results pages should consider these factors and allow detection of task difficulty. Individual differences should also inform search interface design.
Individual Differences, User Perceptions and Eye Gaze in Biomedical Search Interfaces
- 1. Individual Differences, User Perceptions and Eye Gaze in Biomedical Search Interfaces 7 September 2015 Ying-Hsang Liu 1,2 1 School of Information Studies Charles Sturt University 2 Research School of Computer Science The Australian National University
- 2. 1Outline Introduction Research Questions Interfaces Research Design Results Summary and Discussion
- 3. 2Introduction Interactive Information Retrieval (IIR) ▶ Current IR systems designed for specified search (Belkin, 2008) ▶ Natural search user interfaces (Hearst, 2011) ▶ Usefulness of controlled indexing languages (Salton, 1972; Svenonius, 1986) Medical Subject Headings (MeSH) terms
- 4. 3Research Questions Research questions ▶ What elements of search interfaces do searchers look at when searching for documents to answer complex questions? ▶ What is the relation between user perceptions of an interface and the interface elements they look at? ▶ What is the relation between individual differences and the interface elements which are looked at? User experiment in a laboratory setting
- 5. 4Interface A: Google style
- 6. 5Interface B: Per query, ProQuest
- 7. 6Interface C: Per query, ProQuest+EBSCOhost
- 8. 7Interface D: Per document, EBSCOhost
- 9. 8Test Collection Selection of search topics ▶ Document test collection from OHSUMED (Hersh, Buckley, Leone, & Hickam, 1994) ▶ MEDLINE from 1987 to 1991; 348,566 records ▶ Randomly select 8 topics based on proportion of judged relevant documents ▶ 2 topics from each of the quartiles (4 search topic pairs) Sample search topic ▶ ID: 78 ▶ Imagine that you are 42-year-old black man with hypertension. ▶ You would like to find information about beta blockers and blacks with hypertension, utility.
- 10. 9Experimental Design Factorial design ▶ 4 × 2 × 2 Factorial design; 4 interfaces, controlled search topic pairs and cognitive styles ▶ 4 × 4 Graeco-Latin Square to arrange experimental conditions ▶ Power Analysis for ANOVA Design; medium effect size of .25, α < .05 and N = 256, statistical power of .93 (Cohen, 1988; Faul, Erdfelder, Lang, & Buchner, 2007) 4 × 4 Graeco-Latin Square
- 11. 10Software and Hardware Experimental system setup ▶ Experimental search system based on Solr ▶ Gaze tracking uses FaceLab software and hardware ▶ EyeWorks for data recording and analysis ▶ Emotiv headset for EEG data ▶ Search logs and mouse clicks recorded Gaze tracking by FaceLab
- 12. 11Experimental Procedure Experimental procedure Data collection ▶ User characteristics (background questionnaire and cognitive style test) ▶ User perceptions (exit questionnaire) ▶ Search behaviours (search logs, mouse clicks and documents saved) ▶ Physiological signals (eye gaze and EEG)
- 13. 12Searcher Characteristics ▶ 32 subjects; male (50.0%), female (50.0%) ▶ Student: postgraduate (46.9%), undergraudate (40.6%) ▶ Age: 18–25 (59.4%), 25–35 (28.1%) ▶ Online database experience: < 5 years (62.5%), 5–10 years (21.9%) ▶ Search engine: every day (50.0%), several times a day or more (37.5%) ▶ Pilot study (Liu, Thomas, Schmakeit, & Gedeon, 2012) Biology background
- 14. 13Searcher Characteristics (cont’d) ▶ Cognitive style: Individual’s preference or tendency to process information ▶ E-CSA-WA (Extended Cognitive Style Analysis–Wholistic Analytic) test (Peterson, Deary, & Austin, 2003) ▶ Wholistic Analytic Ratio ▶ WA ratio (M = 1.31, SD = .24); cut-off = 1.32 (Clewley, Chen, & Liu, 2010; Chen, Magoulas, & Macredie, 2004; Yuan, Zhang, Chen, & Avery, 2011) E-CSA-WA Test
- 15. 14Data Analysis ▶ Where do people look? Area of interest (AOI) ▶ Logarithmic cross ratio analysis between individual differences/user perceptions and AOI (Fleiss, Levin, & Paik, 2003; Saracevic, Kantor, Chamis, & Trivison, 1988) ▶ ANOVA between interface and searcher characteristics, such as cognitive style and search experience Heat map and AOI
- 16. 15Search Interfaces and AOI Title Author Abstract MeSH q q qq q q q q qq q qq q q q q q q q qq q q q q 0 25 50 75 A B C D A B C D A B C D A B C D Types of Interface ProportionoffixationsinAOI
- 17. 16User Perceptions and AOI Table: Summary of the relation between user perceptions and AOI Difficulty Usefulness Notice of Keywords Use of Keywords B C D B C D Title H H G G G G G — Author H — H H H H H H Abstract H G — — — — — G MeSH H — — — — — — — Note. The relation is not statistically significant (—), positively significant (G), or negatively significant (H) at 95%).
- 18. 17Individual Differences and AOI Table: Summary of the relation between individual differences and AOI Domain Knowledge Search Experience Cognitive Style UG PG Search Engine Online Database Title H H — — — Author — — G — — Abstract — — H — — MeSH — — G — — Note. The relation is not statistically significant (—), positively significant (G), or negatively significant (H) at 95%).
- 19. 18Interface and Search Experience Interaction
- 20. 19Interface and Cognitive Style Interaction
- 21. 20Summary and Discussion Research findings ▶ Searchers look at abstract more often than other interface elements ▶ Interfaces and user perception of search task difficulty significantly affects elements look at ▶ Significant interaction effect between cognitive style/search experience and interface for MeSH AOI Discussion ▶ Design of Search Engine Results Page (SERP) ▶ Detection of search task difficulty ▶ Individual differences for search user interface design
- 22. 21 Thank You! Questions or Comments? This study is partially funded by 2014 ALIA Research Grant Award, led by Dr Ying-Hsang Liu with Marijana Bacic (Monash Health), Dr Paul Thomas (CSIRO) and Professor Tom Gedeon (ANU).