SlideShare a Scribd company logo

From Exploration to Construction
 - How to Support the Complex Dynamics of Information Seeking

T
TimelessFuture

The document discusses the complexities of information seeking and how current search systems often fall short in supporting these processes, particularly in multi-stage tasks. It highlights the relationship between macro-level information seeking models and the micro-level design of search user interfaces, emphasizing the need for better tools to facilitate user exploration and understanding. The findings from a user study illustrate the varying utility of search interface features across different stages of the information seeking process.

Education
1 of 81
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
From Exploration to Construction
 How to Support the Complex Dynamics of Information Seeking 
 
 Hugo C. Huurdeman PhD candidate University of Amsterdam webarchiving.nl
Introduction: a paradox • Models of information 
 seeking describe fundamentally different macro- level stages in complex tasks +uncertainty-uncertainty optimism confusion clarity confidence (dis)satisfaction doubt direction FormulationInitiation Selection Exploration Collection Presentation
Introduction: a paradox • Models of information 
 seeking describe fundamentally different macro- level stages in complex tasks +uncertainty-uncertainty optimism confusion clarity confidence (dis)satisfaction doubt direction FormulationInitiation Selection Exploration Collection Presentation Search • However, current search systems usually provide a streamlined and static feature set • To what extent do current search approaches support complex tasks?
Multistage Information Seeking Models macro perspective1 Read more: Huurdeman & Kamps (2014), From Multistage Information Seeking Models 
 to Multistage Search Systems. Proc. IIiX 2014. http://dx.doi.org/10.1145/2637002.2637020 Huurdeman & Kamps (2015). Supporting the Process - Adapting Search Systems to Search Stages. Proc. ECIL 2015. http://dx.doi.org/10.1007/978-3-319-28197-1_40
1.1 Information Seeking Models • Information seeking modeled in a multitude of ways: • as behavioral patterns (Ellis) • as nonlinear activities (Foster) • as problem-solving (Wilson) • as temporal stages (Kuhlthau), .. • Our main focus: • temporally based IS models • Kuhlthau [1991] (Vakkari [2001]) • cognitively complex (work) tasks • involving learning & construction information search information
 seeking information 
 behavior [Wilson99]
1.2 Kuhlthau: Information Search Process [1991]+uncertainty- feelings thoughts actions vague focused seeking relevant information (exploring) seeking pertinent information (documenting) uncertainty optimism confusion clarity confidence (dis)satisfaction doubt direction FormulationInitiation Selection Exploration Collection Presentation
1.2 Vakkari’s adaptation (in [Vakkari01]) Prefocus Focus formulation Postfocus seeking general background information seeking specific information faceted backgr.
 information relevance hard to judge relevance easier to judge decrease of number of broader terms information
 sought relevance search terms increase of number of search terms, synonyms, narrower terms
1.3 Implications for design of search systems • Observation: good general understanding of macro level inf. seeking stages, but hard to translate to concrete micro level system design choices. macro micro system design inf. seeking stages
Search user interfaces supporting seeking micro perspective2
3.1 Search user interfaces supporting seeking • Search User Interface (SUI) design: • no straightforward task to design a UI with a high usability [Shneiderm05] • A (limited) number of available frameworks, guidelines and design pattern libraries for SUIs • e.g. M. Wilson’s framework of SUI features [Wilson11] Search Input Control Informational Personalizable
2.1 SUI approaches: traditional search • Streamlined interfaces • Focus on • query formulation • result list inspection
 • Advantages: [Hearst09] • lower cognitive load • more accessible • more understandable Highly optimized for lookup tasks, less for open-ended queries
• Research-based tasks • WebART project • new media researchers • action research setting, structured literature review • Search systems allow for answering new research questions, but also have limitations • lack of transparency • lack of process support 2.2 When traditional search does not work well researcher research activities corpus creation analysis dissemination webarchiving.nl
2.3 SUI approaches: exploratory search • Supporting open- ended inf. seeking • Support learning and investigation activities for complex information problems [Marchionini06] • Many potential exploratory SUI features [White09], e.g. • rapid query refinement, facets (input, control) • leveraging context, visualizations (informational) • histories/workspaces/task management (personalizable)
2.4 SUI approaches: sensemaking & analytics • Support analysis & synthesis in interface • Potential functions facilitating notetaking, hypothesis formulation & collaborative search [Hearst09] • some overlap with exploratory search
From Exploration to Construction
 - How to Support the Complex Dynamics of Information Seeking
2.5 Implications for search stage support (2/2) • Observation: good understanding of search system features at the micro level, but fragmented understanding of how they can support information seeking stages at the macro level macro micro search system features inf. seeking stages
Reconciling macro and micro views • Would it be possible to reconcile the macro level and micro level views? macro micro search system features inf. seeking stages
“The Utility of SUI Features” Our study: investigating the utility of various 
 SUI features at different macro-level stages From: Huurdeman, Wilson & Kamps (2016), Active and Passive Utility of 
 Search Interface Features in Different Information Seeking Task Stages. 
 Proc. ACM CHIIR 2016. http://dx.doi.org/10.1145/2854946.2854957
3. Setup • User study (26 participants; 24 analyzed) • Undergrads Univ. of Nottingham (6 F, 12 M, 18-25y) • Experimental SUI resembling common Search Engine • Within-participants • Task stage independent variable • Task design: explicit multistage approach
3. Setup: Multistage Task Design sim. work task: writing essay subtask subtask subtask prepare list of 
 3 topics choose topic;
 formulate specific
 question find and select 
 additional
 pages to cite 15 minutes 15 minutes 15 minutes initiation
 topic selection
 exploration focus formulation collecting presenting
3. Setup: Multistage Task Design sim. work task: writing essay subtask subtask subtask prepare list of 
 3 topics choose topic;
 formulate specific
 question find and select 
 additional
 pages to cite 15 minutes 15 minutes 15 minutes initiation
 topic selection
 exploration focus formulation collecting presenting General Assigned Topics (b/o discussions teaching staff) • Autonomous Vehicles (AV) • Virtual Reality (VR)
3. Setup: Protocol Training task Pre- Questionnaire Topic Assignment Introduction system Task Post-task Questionnaire 3x Post-experiment questionnaire Debriefing interview
• Experimental system: SearchAssist • Results, Query Corrections, Query Suggestions: Bing Web API • Category Filters: DMOZ • Categorization and analysis: • Max Wilson’s framework of SUI features [Wilson11]
From Exploration to Construction
 - How to Support the Complex Dynamics of Information Seeking
Control
Control Input
Control Input Informational
Control Input PersonalizableInformational
3. Setup: Logging eyetribe.com
3. Setup: Data / Task details • AV & VR topics invoked comparable behaviours: • analysed as one topic set • Total duration main tasks • Total task time: 32:56 • 36.8% SUI, 33% Task screen, 30.2% Webpages Stage 1: 11:32 Stage 2: 8:24 Stage 3: 12:59
Findings: Active Behaviour behaviour directly and indirectly derivable from logs4
4.1 Active Behaviour: Clicks 0 4 8 Sig. clicks on interface 
 features over time Stage 1 Stage 2 Stage 3
4.1 Active Behaviour: Clicks 0 4 8 Sig. clicks on interface 
 features over time Category filters ➡ Stage 1 Stage 2 Stage 3
4.1 Active Behaviour: Clicks 0 4 8 Sig. clicks on interface 
 features over time Category filters ➡ Tag Cloud ➡ Stage 1 Stage 2 Stage 3
4.1 Active Behaviour: Clicks 0 4 8 Sig. clicks on interface 
 features over time Category filters ➡ Tag Cloud ➡ Search button ➡ Stage 1 Stage 2 Stage 3
4.1 Active Behaviour: Clicks 0 4 8 Sig. clicks on interface 
 features over time Category filters ➡ Tag Cloud ➡ Search button ➡ Saved Results Stage 1 Stage 2 Stage 3
4.2 Active Behaviour: Queries •Mean number of queries** (unique): • Stage 1: 9.5 (8.1) ➡ Stage 2: 5.5 (5.1) ➡ Stage 3: 5.9 (5.3) 0 2,5 5 7,5 10 Stage 1 Stage 2 Stage 3 Search Box Query Suggestions Recent Queries
4.3 Active Behaviour: Query words •Mean number of query words**: “virtual reality” (P.02) “impact of virtual reality on society art and culture“ “autonomous vehicles” (P.06) “autonomous vehicles costs
 insurance industry” 0 1,25 2,5 3,75 5 Stage 1 Stage 2 Stage 3 Mean Number of Query words
4.4 Active Behaviour: Visited pages • Visited pages (unique)**: • Stage 1: 8.0 (7.3) • Stage 2: 6.4 (5.9) • dwell time highest • Stage 3: 14.2 (10.8) • Mean rank visited pages • from 3.1 to 6.4 0 4 8 12 16 Stage 1 Stage 2 Stage 3 Results List Saved Results
4.5 Active Behaviour: Wrapup • Clicks: • decreasing for Query Box (input), Category Filters & Tag Cloud (control) • increasing for Saved Results (personalizable) • Queries: • decreasing over time, but more complex • Popularity of certain features and impopularity of others: •Some features used in passive instead of active ways?
Findings: Passive Behaviour behaviour not typically caught in interaction logs5 eyetribe.com
Passive behaviour: mouse hovers • Mouse movements: • movements to reach a feature, also to aid processing contents [Rodden08] •Focus here on mouse movements not leading to click • Tendencies mostly overlap with active interaction measure 0% 25% 50% 75% 100% 1 2 3
Passive behaviour: mouse hovers Category filters** ➡ • Mouse movements: • movements to reach a feature, also to aid processing contents [Rodden08] •Focus here on mouse movements not leading to click • Tendencies mostly overlap with active interaction measure 0% 25% 50% 75% 100% 1 2 3
Passive behaviour: mouse hovers Category filters** ➡ Tag Cloud* ➡ • Mouse movements: • movements to reach a feature, also to aid processing contents [Rodden08] •Focus here on mouse movements not leading to click • Tendencies mostly overlap with active interaction measure 0% 25% 50% 75% 100% 1 2 3
Passive behaviour: mouse hovers Category filters** ➡ Tag Cloud* ➡ Query Box** ➡ • Mouse movements: • movements to reach a feature, also to aid processing contents [Rodden08] •Focus here on mouse movements not leading to click • Tendencies mostly overlap with active interaction measure 0% 25% 50% 75% 100% 1 2 3
Passive behaviour: mouse hovers Category filters** ➡ Tag Cloud* ➡ Query Box** ➡ Results List* ⤻ • Mouse movements: • movements to reach a feature, also to aid processing contents [Rodden08] •Focus here on mouse movements not leading to click • Tendencies mostly overlap with active interaction measure 0% 25% 50% 75% 100% 1 2 3
5.2 Passive Behaviour: eye fixations Stage 1 (exploration) Stage 2 (focus formulation) Stage 3 (postfocus, collection) • Overview of eye movement via heatmaps:
Passive behaviour: eye tracking eye tracking fixations 0 25 50 75 100 1 2 3 • Further insights via eye tracking fixation counts • fixations > 80 ms, similar to e.g. [Buscher08]
Passive behaviour: eye tracking eye tracking fixations 0 25 50 75 100 1 2 3 • Further insights via eye tracking fixation counts • fixations > 80 ms, similar to e.g. [Buscher08] Query Suggestions* ➡
Passive behaviour: eye tracking eye tracking fixations 0 25 50 75 100 1 2 3 Tag Cloud* ➡ • Further insights via eye tracking fixation counts • fixations > 80 ms, similar to e.g. [Buscher08] Query Suggestions* ➡
Passive behaviour: eye tracking eye tracking fixations 0 25 50 75 100 1 2 3 Category filters** ➡ Tag Cloud* ➡ • Further insights via eye tracking fixation counts • fixations > 80 ms, similar to e.g. [Buscher08] Query Suggestions* ➡
Passive behaviour: eye tracking eye tracking fixations 0 25 50 75 100 1 2 3 Category filters** ➡ Tag Cloud* ➡ Query Box** ➡ • Further insights via eye tracking fixation counts • fixations > 80 ms, similar to e.g. [Buscher08] Query Suggestions* ➡
Passive behaviour: eye tracking eye tracking fixations 0 25 50 75 100 1 2 3 Category filters** ➡ Tag Cloud* ➡ Query Box** ➡ Results List* ⤻ • Further insights via eye tracking fixation counts • fixations > 80 ms, similar to e.g. [Buscher08] Query Suggestions* ➡
3.4 Passive Behaviour: Active vs. Passive 0% 2% 4% 6% 8% Stage 1 Stage 2 Stage 3 Subtle differences between passive and active use:
3.4 Passive Behaviour: Active vs. Passive 0% 2% 4% 6% 8% Stage 1 Stage 2 Stage 3 Tag Cloud [5.8% fixations ⬌ 3.1% clicks] Subtle differences between passive and active use:
3.4 Passive Behaviour: Active vs. Passive 0% 2% 4% 6% 8% Stage 1 Stage 2 Stage 3 Query Suggestions [3.6% fix. ⬌ 1.9% clicks] Tag Cloud [5.8% fixations ⬌ 3.1% clicks] Subtle differences between passive and active use:
3.4 Passive Behaviour: Active vs. Passive 0% 2% 4% 6% 8% Stage 1 Stage 2 Stage 3 Query Suggestions [3.6% fix. ⬌ 1.9% clicks] Tag Cloud [5.8% fixations ⬌ 3.1% clicks] Recent Queries [3% fix. ⬌ 2% clicks] Subtle differences between passive and active use:
3.4 Passive Behaviour: Active vs. Passive 0% 2% 4% 6% 8% Stage 1 Stage 2 Stage 3 Query Suggestions [3.6% fix. ⬌ 1.9% clicks] Tag Cloud [5.8% fixations ⬌ 3.1% clicks] Recent Queries [3% fix. ⬌ 2% clicks] Subtle differences between passive and active use: Opposite for Category Filters [5% ⬌ 3.8%]
5.4 Passive Behaviour: Wrapup •Fixations & mouse moves • validating active behaviour • subtle differences active and passive use • Could subjective ratings and qualitative feedback provide more insights?
Findings: Perceived Feature Utility perceived usefulness (post-stage & experiment)6
6.2 Perceived Usefulness: post-experiment • Post-experiment questionnaire: • In which stage or stages were SUI features most useful? • Pronounced differences • significant differences for all features 0% 25% 50% 75% 100% Query Box / 
 Results List Category
 Filters Tag 
 Cloud Query 
 Suggestions Recent 
 Queries Saved 
 Results
6.2 Perceived Usefulness: post-experiment • Post-experiment questionnaire: • In which stage or stages were SUI features most useful? • Pronounced differences • significant differences for all features 0% 25% 50% 75% 100% Query Box / 
 Results List Category
 Filters Tag 
 Cloud Query 
 Suggestions Recent 
 Queries Saved 
 Results
6.2 Perceived Usefulness: post-experiment • Post-experiment questionnaire: • In which stage or stages were SUI features most useful? • Pronounced differences • significant differences for all features 0% 25% 50% 75% 100% Query Box / 
 Results List Category
 Filters Tag 
 Cloud Query 
 Suggestions Recent 
 Queries Saved 
 Results
6.3 Perceived Usefulness: Category Filters • “good at the start (…) but later I wanted something more specific” (P.11) • common remarks in 2nd and 3rd stage: • “… could be more specific in its categories” • “…hard to find the category I want” (P. 27)
6.3 Perceived Usefulness: Tag Cloud • at the start: • “…aids exploring the topic” (P.06); • “came up with words that I hadn’t thought of” • later stages: • “doesn’t help to narrow the search much” (P.18) • “in the end seemed to be too general” (P.07)
6.3 Perceived Usefulness: Tag Cloud • at the start: • “…aids exploring the topic” (P.06); • “came up with words that I hadn’t thought of” • later stages: • “doesn’t help to narrow the search much” (P.18) • “in the end seemed to be too general” (P.07) • Post-experiment comments: • “…was good at the beginning, because when you are not exactly sure what you are looking for, it can give inspiration” (P.12) • “… nice to look at what other kinds of ideas [exist] that maybe you didn’t think of. Then one word may spark your interest” (P.15)
6.3 Perceived Utility: Query Suggestions • “…was good at the start but as soon as I got more specific into my topic, that went down” (P.11) • “clicked [it] .. a couple of times .. it gave me sort of serendipitous results, which are useful” (P.24)
6.3 Perceived Utility: Recent Queries • Naturally: “…most useful in the end because I had more searches from before” (P.26) • “The previous searches became more useful ‘as I made them’ because they were there and I could see what I searched before. I was sucking myself in and could work by looking at those.” (P.23) • May aid searchers in 
 their information journey..
6.3 Perceived Utility: Saved Results • “most useful in the end” (P.12) • “At the start [I was] saving a lot of general things about different topics. Later on I went back to the saved ones for the topic I chose and then sort of went on from that and see what else I should search” (P.26) • “I just felt I was organizing my research a little bit” (P.18) • It “helps me to lay out the plans of my research”.
Conclusion towards more dynamic support7
0%! 20%! 40%! 60%! 80%! 100%! Stage 1! Stage 2! Stage 3! Percentageofparticipants! input / informational! control! personalisable! Stage 2! Stage 3! input / informational! control! personalisable! Conclusion: Findings Summary • Informational features highly useful in most stages • Decreasing use of input features • Control features decreasingly useful • likely caused by a user’s evolving domain knowledge • Personalizable features increasingly useful • ‘growing’ with a user’s understanding, task management support SUI features perceived as most useful, per stage
7. Conclusion: theoretical roundup complex information seeking task pre-focus stage: • vague understanding • limited domain knowledge • trouble expressing information need • large amount of new information • explaining prominent role of control features • explore information • filter result set using [Kuhlthau04,Vakkari&Hakkala00,Vakkari01]
7. Conclusion: theoretical roundup complex information seeking task pre-focus stage: • vague understanding • limited domain knowledge • trouble expressing information need • large amount of new information • explaining prominent role of control features • explore information • filter result set focus formulation stage: • more directed search • better understanding • seeking more relevant information, using differentiated criteria • control features become less essential • “not specific enough” • personalizable feat’s more important: may “grow” with emerging understanding using [Kuhlthau04,Vakkari&Hakkala00,Vakkari01]
7. Conclusion: theoretical roundup complex information seeking task pre-focus stage: • vague understanding • limited domain knowledge • trouble expressing information need • large amount of new information • explaining prominent role of control features • explore information • filter result set focus formulation stage: • more directed search • better understanding • seeking more relevant information, using differentiated criteria • control features become less essential • “not specific enough” • personalizable feat’s more important: may “grow” with emerging understanding postfocus stage • specific searches • re-checks additional information • precise expression • low uniqueness, high redundancy of info • long, precise, queries • further decline of control features • frequent use of personalizable features • “see what else to search” using [Kuhlthau04,Vakkari&Hakkala00,Vakkari01]
7. Conclusion: Future Work •Our study: essay writing simulated work task • Extension to other types of complex tasks, user populations •Further research into task-aware search systems • additional features may be useful at different stages • e.g. user hints, assistance • improvement of current features
Towards “stage-aware” Systems prefocus focus formulation postfocus searchersearch
 system search
 interface search 
 stage stage-independent functionalityranking stage-dependent functionality manual or automatic
 detection
• INEX/CLEF Interactive Social Book Search Lab • http://social-book-search.humanities.uva.nl/#/interactive • Gaede, Hall, Huurdeman, Kamps, Koolen, Skov, Toms, Walsh (2015) • Aim: support different stages in the search process: browse, search & review • Joint study across universities, 192 participants Multistage interface: search Multistage interface: browse Multistage interface: review Example: multistage interface
7. Conclusion: towards dynamic SUIs •Most Web search systems converged over static and familiar designs • trialled features often struggled to provide value for searchers • perhaps impeding search [Diriye10] if introduced in simple tasks, or at the wrong moment •Our work provides insights into when SUI features are useful during search episodes • potential responsive and adaptive SUIs for complex tasks
References (1/2) [Ahlberg&Shneiderman94] C. Ahlberg and B. Shneiderman. Visual information seeking: Tight coupling of dynamic query filters with starfield displays. In CHI, pages 313–317. ACM, 1994. 
 [Buscher08] G. Buscher, A. Dengel, and L. van Elst. Eye movements as implicit relevance feedback. In CHI’08 extended abstracts on Human factors in computing systems, pages 2991–2996. ACM, 2008. [Diriye10] A. Diriye, A. Blandford, and A. Tombros. When is system support effective? In Proc. IIiX, pages 55–64. ACM, 2010. [Diriye13] A. Diriye, A. Blandford, A. Tombros, and P. Vakkari. The role of search interface features during information seeking. In TPDL, volume 8092 of LNCS, pages 235–240. Springer, 2013. [Donato10] D. Donato, F. Bonchi, T. Chi, and Y. Maarek. Do You Want to Take Notes?: Identifying Research Missions in Yahoo! Search Pad. In Proc. WWW’10, pages 321–330, 2010. ACM. [GaedeEtAl15] Maria Gäde, Mark Hall, Hugo Huurdeman, Jaap Kamps, Marijn Koolen, Mette Skov, Elaine Toms, and David Walsh. Overview of the SBS 2015 interactive track. In CLEF’15 Working Notes. CEUR-WS, 2015. [Hearst09] M. A. Hearst. Search user interfaces. Cambridge University Press, 2009. [Hearst13] M. A. Hearst and D. Degler. Sewing the seams of sensemaking: A practical interface for tagging and organizing saved search results. In HCIR. ACM, 2013. [Huurdeman&Kamps15] Hugo C. Huurdeman and Jaap Kamps (2015). Supporting the Process: Adapting Search Systems to Search Stages. In: S. Kurbanoğlu, S. Špiranec, J. Boustany, E. Grassian, D. Mizrachi, & L. Roy (Eds.), Information Literacy: Moving towards sustainability, Communication in Computer and Information Science series (Vol. 552, pp. 394-404).
 [Huurdeman&Kamps14] H. C. Huurdeman and J. Kamps. From Multistage Information-seeking Models to Multistage Search Systems. In Proc. IIiX’14, pages 145–154, 2014. ACM [Kuhlthau91] C. C. Kuhlthau. Inside the search process: Information seek- ing from the user’s perspective. JASIS, 42:361–371, 1991. [Kuhlthau04] C. C. Kuhlthau. Seeking Meaning: A Process Approach to Library and Information Services. Libraries Unlimited, 2004. [Kules12] B. Kules and R. Capra. Influence of training and stage of search on gaze behavior in a library catalog faceted search interface. JASIST, 63:114–138, 2012. [LiuBelkin15] J. Liu and N. J. Belkin. Personalizing information retrieval for multi-session tasks. JASIST, 66(1):58–81, Jan. 2015. [Marchionini06] G. Marchionini. Exploratory search: from finding to understanding. CACM, 49(4):41–46, 2006. [Niu14] X. Niu and D. Kelly. The use of query suggestions during information search. IPM, 50:218–234, 2014. [Proulx06] P. Proulx, S. Tandon, A. Bodnar, D. Schroh, W. Wright, D. Schroh, R. Harper, and W. Wright. Avian Flu Case Study with nSpace and GeoTime. In Proceedings of the IEEE Symposium on Visual Analytics Science and Technology (VAST'06). IEEE, 2006.
References (2/2) [Toms11] E. G. Toms. Task-based information searching and retrieval. In Interactive Information Seeking, Behaviour and Retrieval. Facet, 2011. [Rodden08] K. Rodden, X. Fu, A. Aula, and I. Spiro. Eye-mouse coordination patterns on web search results pages. In CHI’08 Extended Abstracts, pages 2997–3002. ACM, 2008. [Shneiderman05] B. Shneiderman and C. Pleasant. Designing the user in- terface: strategies for effective human-computer interaction. Pearson Education, 2005. [Tunkelang09] D. Tunkelang. Faceted search. Synthesis lectures on information concepts, retrieval, and services, 1(1):1–80, 2009. [Vakkari01] P. Vakkari. A theory of the task-based information retrieval process: a summary and generalisation of a longitudinal study. Journal of Documentation, 57:44–60, 2001. [White05] R. W. White, I. Ruthven, and J. M. Jose. A study of factors affecting the utility of implicit relevance feedback. In SIGIR, pages 35–42. ACM, 2005. [White09] R. W. White and R. A. Roth. Exploratory search: Beyond the query-response paradigm. Synthesis Lectures on Information Concepts, Retrieval, and Services, 1:1–98, 2009. [Wilson&schraefel08] M. L. Wilson and m. c. schraefel. A longitudinal study of exploratory and keyword search. In In Proc. JCDL’08, pages 52–56. ACM, 2008. [Wilson99] T. D. Wilson. Models in information behaviour research. Journal of Documentation, 55:249–270, 1999. [Wilson11] M. L. Wilson. Interfaces for information retrieval. In I. Ruthven and D. Kelly, editors. Interactive Information Seeking, Behaviour and Retrieval. Facet, 2011.
From Exploration to Construction
 How to Support the Complex Dynamics of Information Seeking 
 
 Hugo C. Huurdeman University of Amsterdam @TimelessFuture webarchiving.nl

More Related Content

PDF
Active & Passive Utility of Search Interface Features in different Informatio...
TimelessFuture
 
PDF
From multistage information seeking models to multistage search systems (IIiX...
TimelessFuture
 
DOCX
RESUME anup
Anup Sahu
 
PDF
Fukuda fx 7402 orig
Corporativo Medico Castillo
 
PDF
HRM..
Linh Ha
 
PDF
Trading Derivados14/05/2013
BNP Paribas Personal Investors
 
PDF
Nido Marketing Strategies
Muhammad Ali Jehangir
 
PPTX
Lead fx presentation general deck as at feb 22 2016
Lead_FX
 
Active & Passive Utility of Search Interface Features in different Informatio...
TimelessFuture
 
From multistage information seeking models to multistage search systems (IIiX...
TimelessFuture
 
RESUME anup
Anup Sahu
 
Fukuda fx 7402 orig
Corporativo Medico Castillo
 
HRM..
Linh Ha
 
Trading Derivados14/05/2013
BNP Paribas Personal Investors
 
Nido Marketing Strategies
Muhammad Ali Jehangir
 
Lead fx presentation general deck as at feb 22 2016
Lead_FX
 

Similar to From Exploration to Construction
 - How to Support the Complex Dynamics of Information Seeking (20)

PPT
Search & Filter Interface Round Up - Usability Marathon - Cueva
Amy Cueva
 
PDF
Search&Filter Interface Round Up by Amy Cueva, Mad Pow
UIDesign Group
 
PDF
pedersen
Hiroshi Ono
 
PPTX
Information retrieval 2 search behaviour and search process
Vaibhav Khanna
 
PPTX
SharePoint Saturday Belgium 2013 Intranet search fail
BIWUG
 
PPTX
SPSBE14 Intranet Search #fail
Ben van Mol
 
PPT
Search Analytics for Fun and Profit
Louis Rosenfeld
 
PDF
Supporting the Process - Adapting Search Systems To Search Stages (ECIL15)
TimelessFuture
 
PPTX
Cognitive Ability Effects on Effort in Web Search & Navigation by Gwizdka
jacekg
 
PPTX
Devcon 2013: Understanding Search Behavior
Kelley Howell
 
PPT
Search and Filter Interface Round Up - Userability Marathon 2009 - Amy Cueva
Mad*Pow
 
PDF
Search Behavior Patterns
Ramzi Alqrainy
 
PPT
675d614e68cce (5).ppt image retrival for UI
avinashpowar1986
 
PDF
The Multi-Stage Experience: the Simulated Work Task Approach to Studying Info...
TimelessFuture
 
PDF
Social Information Access: A Personal Update
Daqing He
 
PPTX
Chaos&Order: Using visualization as a means to
 explore large heritage collec...
TimelessFuture
 
PDF
The Value of Multistage Search Systems for Book Search
TimelessFuture
 
PDF
Search Solutions 2011: Successful Enterprise Search By Design
Marianne Sweeny
 
PPTX
“From Discovery to Fulfillment: Improving the User Experience at Every Stage.”
Lynn Connaway
 
PDF
A Journey into Evaluation: from Retrieval Effectiveness to User Engagement
Mounia Lalmas-Roelleke
 
Search & Filter Interface Round Up - Usability Marathon - Cueva
Amy Cueva
 
Search&Filter Interface Round Up by Amy Cueva, Mad Pow
UIDesign Group
 
pedersen
Hiroshi Ono
 
Information retrieval 2 search behaviour and search process
Vaibhav Khanna
 
SharePoint Saturday Belgium 2013 Intranet search fail
BIWUG
 
SPSBE14 Intranet Search #fail
Ben van Mol
 
Search Analytics for Fun and Profit
Louis Rosenfeld
 
Supporting the Process - Adapting Search Systems To Search Stages (ECIL15)
TimelessFuture
 
Cognitive Ability Effects on Effort in Web Search & Navigation by Gwizdka
jacekg
 
Devcon 2013: Understanding Search Behavior
Kelley Howell
 
Search and Filter Interface Round Up - Userability Marathon 2009 - Amy Cueva
Mad*Pow
 
Search Behavior Patterns
Ramzi Alqrainy
 
675d614e68cce (5).ppt image retrival for UI
avinashpowar1986
 
The Multi-Stage Experience: the Simulated Work Task Approach to Studying Info...
TimelessFuture
 
Social Information Access: A Personal Update
Daqing He
 
Chaos&Order: Using visualization as a means to
 explore large heritage collec...
TimelessFuture
 
The Value of Multistage Search Systems for Book Search
TimelessFuture
 
Search Solutions 2011: Successful Enterprise Search By Design
Marianne Sweeny
 
“From Discovery to Fulfillment: Improving the User Experience at Every Stage.”
Lynn Connaway
 
A Journey into Evaluation: from Retrieval Effectiveness to User Engagement
Mounia Lalmas-Roelleke
 
Ad

More from TimelessFuture (19)

PDF
Data Visualization via Enhanced Maps in a Digital Humanities Context – a Desi...
TimelessFuture
 
PDF
Webmapping: maps for presentation, exploration & analysis
TimelessFuture
 
PDF
Experiential Interfaces: 

3D reconstructions as entry points for exploration...
TimelessFuture
 
PDF
Step inside the Image: 

Interpretative Interfaces for 
3D Historical Content
TimelessFuture
 
PDF
Supporting the Interpretation of Enriched Audiovisual Sources through Tempora...
TimelessFuture
 
PDF
Op Ontdekkingsreis door het KB Webarchief - Exploratieve Visualisatie in een ...
TimelessFuture
 
PDF
Visualization Lecture - Clariah Summer School 2018
TimelessFuture
 
PDF
Outcomes Visual Navigation Project
TimelessFuture
 
PDF
KNVI 2017: De collectie in een ander licht - Creatieve inzet van nieuwe techn...
TimelessFuture
 
PDF
Workshop: Inspirational Journeys - Challenges and Solutions for Visual Naviga...
TimelessFuture
 
PDF
“More than Meets the Eye” - Analyzing the Success of User Queries in Oria
TimelessFuture
 
PDF
Not available, or not found? Lessons from user queries in the Oria catalog at...
TimelessFuture
 
PDF
Webarchief & Wetenschap (Dutch)
TimelessFuture
 
PDF
Towards Multidimensional Web Archive Access (IIPC 2016)
TimelessFuture
 
PDF
Towards Research Engines: Supporting Search Stages in Web Archives (2015)
TimelessFuture
 
PDF
WebART: hoe maak je webarchieven bruikbaar voor de wetenschap? (Dutch)
TimelessFuture
 
PDF
Finding Pages on the Unarchived Web (DL 2014)
TimelessFuture
 
PDF
WebART - "Data Digging" - eHumanities Group 2013
TimelessFuture
 
PDF
WebART: Facilitating Scholarly Use of Web Archives (IIPC, Apr. 2013)
TimelessFuture
 
Data Visualization via Enhanced Maps in a Digital Humanities Context – a Desi...
TimelessFuture
 
Webmapping: maps for presentation, exploration & analysis
TimelessFuture
 
Experiential Interfaces: 

3D reconstructions as entry points for exploration...
TimelessFuture
 
Step inside the Image: 

Interpretative Interfaces for 
3D Historical Content
TimelessFuture
 
Supporting the Interpretation of Enriched Audiovisual Sources through Tempora...
TimelessFuture
 
Op Ontdekkingsreis door het KB Webarchief - Exploratieve Visualisatie in een ...
TimelessFuture
 
Visualization Lecture - Clariah Summer School 2018
TimelessFuture
 
Outcomes Visual Navigation Project
TimelessFuture
 
KNVI 2017: De collectie in een ander licht - Creatieve inzet van nieuwe techn...
TimelessFuture
 
Workshop: Inspirational Journeys - Challenges and Solutions for Visual Naviga...
TimelessFuture
 
“More than Meets the Eye” - Analyzing the Success of User Queries in Oria
TimelessFuture
 
Not available, or not found? Lessons from user queries in the Oria catalog at...
TimelessFuture
 
Webarchief & Wetenschap (Dutch)
TimelessFuture
 
Towards Multidimensional Web Archive Access (IIPC 2016)
TimelessFuture
 
Towards Research Engines: Supporting Search Stages in Web Archives (2015)
TimelessFuture
 
WebART: hoe maak je webarchieven bruikbaar voor de wetenschap? (Dutch)
TimelessFuture
 
Finding Pages on the Unarchived Web (DL 2014)
TimelessFuture
 
WebART - "Data Digging" - eHumanities Group 2013
TimelessFuture
 
WebART: Facilitating Scholarly Use of Web Archives (IIPC, Apr. 2013)
TimelessFuture
 
Ad

Recently uploaded (20)

PPTX
Tissue processing ( HISTOPATHOLOGICAL TECHNIQUE
mathiasmelwyn7
 
PDF
grade 11-chemistry_fetena_net_5883.pdf teacher guide for all student
banteamlakmebratu738
 
PPTX
human mycosis Human fungal infections are called human mycosis..pptx
shilpa939953
 
PDF
A GUIDE TO GENETICS FOR UNDERGRADUATE MEDICAL STUDENTS
DrBincy A. S
 
PDF
STATICS OF THE RIGID BODIES Hibbelers.pdf
pascualasturiaskaye4
 
PDF
The Lost Whites of Pakistan by Jahanzaib Mughal.pdf
jahanzaibmughal6
 
PDF
Anesthesia in Laparoscopic Surgery in India
mohitsuren827
 
PPTX
GDM (1) (1).pptx small presentation for students
shrawanbpkihs
 
PDF
Chinmaya Tiranga quiz Grand Finale.pdf
Nitin Suresh
 
PPTX
1st Inaugural Professorial Lecture held on 19th February 2020 (Governance and...
kawisojames10
 
PDF
Weekly quiz Compilation Jan -July 25.pdf
Nitin Suresh
 
PPTX
IMMUNITY IMMUNITY refers to protection against infection, and the immune syst...
shilpa939953
 
PDF
OBE - B.A.(HON'S) IN INTERIOR ARCHITECTURE -Ar.MOHIUDDIN.pdf
ArchitectAFMMohiuddi
 
PPTX
master seminar digital applications in india
ananyaray35
 
PDF
2.FourierTransform-ShortQuestionswithAnswers.pdf
Raji Murugan
 
PDF
GENETICS IN BIOLOGY IN SECONDARY LEVEL FORM 3
ElishamichaelSamwel
 
PDF
Computing-Curriculum for Schools in Ghana
abigailanane203
 
PDF
RMMM.pdf make it easy to upload and study
sajedul2
 
PPTX
Pharma ospi slides which help in ospi learning
rameetkumar304
 
PDF
Module 4: Burden of Disease Tutorial Slides S2 2025
Jonathan Hallett
 
Tissue processing ( HISTOPATHOLOGICAL TECHNIQUE
mathiasmelwyn7
 
grade 11-chemistry_fetena_net_5883.pdf teacher guide for all student
banteamlakmebratu738
 
human mycosis Human fungal infections are called human mycosis..pptx
shilpa939953
 
A GUIDE TO GENETICS FOR UNDERGRADUATE MEDICAL STUDENTS
DrBincy A. S
 
STATICS OF THE RIGID BODIES Hibbelers.pdf
pascualasturiaskaye4
 
The Lost Whites of Pakistan by Jahanzaib Mughal.pdf
jahanzaibmughal6
 
Anesthesia in Laparoscopic Surgery in India
mohitsuren827
 
GDM (1) (1).pptx small presentation for students
shrawanbpkihs
 
Chinmaya Tiranga quiz Grand Finale.pdf
Nitin Suresh
 
1st Inaugural Professorial Lecture held on 19th February 2020 (Governance and...
kawisojames10
 
Weekly quiz Compilation Jan -July 25.pdf
Nitin Suresh
 
IMMUNITY IMMUNITY refers to protection against infection, and the immune syst...
shilpa939953
 
OBE - B.A.(HON'S) IN INTERIOR ARCHITECTURE -Ar.MOHIUDDIN.pdf
ArchitectAFMMohiuddi
 
master seminar digital applications in india
ananyaray35
 
2.FourierTransform-ShortQuestionswithAnswers.pdf
Raji Murugan
 
GENETICS IN BIOLOGY IN SECONDARY LEVEL FORM 3
ElishamichaelSamwel
 
Computing-Curriculum for Schools in Ghana
abigailanane203
 
RMMM.pdf make it easy to upload and study
sajedul2
 
Pharma ospi slides which help in ospi learning
rameetkumar304
 
Module 4: Burden of Disease Tutorial Slides S2 2025
Jonathan Hallett
 

From Exploration to Construction
 - How to Support the Complex Dynamics of Information Seeking

  • 1. From Exploration to Construction
 How to Support the Complex Dynamics of Information Seeking 
 
 Hugo C. Huurdeman PhD candidate University of Amsterdam webarchiving.nl
  • 2. Introduction: a paradox • Models of information 
 seeking describe fundamentally different macro- level stages in complex tasks +uncertainty-uncertainty optimism confusion clarity confidence (dis)satisfaction doubt direction FormulationInitiation Selection Exploration Collection Presentation
  • 3. Introduction: a paradox • Models of information 
 seeking describe fundamentally different macro- level stages in complex tasks +uncertainty-uncertainty optimism confusion clarity confidence (dis)satisfaction doubt direction FormulationInitiation Selection Exploration Collection Presentation Search • However, current search systems usually provide a streamlined and static feature set • To what extent do current search approaches support complex tasks?
  • 4. Multistage Information Seeking Models macro perspective1 Read more: Huurdeman & Kamps (2014), From Multistage Information Seeking Models 
 to Multistage Search Systems. Proc. IIiX 2014. http://dx.doi.org/10.1145/2637002.2637020 Huurdeman & Kamps (2015). Supporting the Process - Adapting Search Systems to Search Stages. Proc. ECIL 2015. http://dx.doi.org/10.1007/978-3-319-28197-1_40
  • 5. 1.1 Information Seeking Models • Information seeking modeled in a multitude of ways: • as behavioral patterns (Ellis) • as nonlinear activities (Foster) • as problem-solving (Wilson) • as temporal stages (Kuhlthau), .. • Our main focus: • temporally based IS models • Kuhlthau [1991] (Vakkari [2001]) • cognitively complex (work) tasks • involving learning & construction information search information
 seeking information 
 behavior [Wilson99]
  • 6. 1.2 Kuhlthau: Information Search Process [1991]+uncertainty- feelings thoughts actions vague focused seeking relevant information (exploring) seeking pertinent information (documenting) uncertainty optimism confusion clarity confidence (dis)satisfaction doubt direction FormulationInitiation Selection Exploration Collection Presentation
  • 7. 1.2 Vakkari’s adaptation (in [Vakkari01]) Prefocus Focus formulation Postfocus seeking general background information seeking specific information faceted backgr.
 information relevance hard to judge relevance easier to judge decrease of number of broader terms information
 sought relevance search terms increase of number of search terms, synonyms, narrower terms
  • 8. 1.3 Implications for design of search systems • Observation: good general understanding of macro level inf. seeking stages, but hard to translate to concrete micro level system design choices. macro micro system design inf. seeking stages
  • 9. Search user interfaces supporting seeking micro perspective2
  • 10. 3.1 Search user interfaces supporting seeking • Search User Interface (SUI) design: • no straightforward task to design a UI with a high usability [Shneiderm05] • A (limited) number of available frameworks, guidelines and design pattern libraries for SUIs • e.g. M. Wilson’s framework of SUI features [Wilson11] Search Input Control Informational Personalizable
  • 11. 2.1 SUI approaches: traditional search • Streamlined interfaces • Focus on • query formulation • result list inspection
 • Advantages: [Hearst09] • lower cognitive load • more accessible • more understandable Highly optimized for lookup tasks, less for open-ended queries
  • 12. • Research-based tasks • WebART project • new media researchers • action research setting, structured literature review • Search systems allow for answering new research questions, but also have limitations • lack of transparency • lack of process support 2.2 When traditional search does not work well researcher research activities corpus creation analysis dissemination webarchiving.nl
  • 13. 2.3 SUI approaches: exploratory search • Supporting open- ended inf. seeking • Support learning and investigation activities for complex information problems [Marchionini06] • Many potential exploratory SUI features [White09], e.g. • rapid query refinement, facets (input, control) • leveraging context, visualizations (informational) • histories/workspaces/task management (personalizable)
  • 14. 2.4 SUI approaches: sensemaking & analytics • Support analysis & synthesis in interface • Potential functions facilitating notetaking, hypothesis formulation & collaborative search [Hearst09] • some overlap with exploratory search
  • 16. 2.5 Implications for search stage support (2/2) • Observation: good understanding of search system features at the micro level, but fragmented understanding of how they can support information seeking stages at the macro level macro micro search system features inf. seeking stages
  • 17. Reconciling macro and micro views • Would it be possible to reconcile the macro level and micro level views? macro micro search system features inf. seeking stages
  • 18. “The Utility of SUI Features” Our study: investigating the utility of various 
 SUI features at different macro-level stages From: Huurdeman, Wilson & Kamps (2016), Active and Passive Utility of 
 Search Interface Features in Different Information Seeking Task Stages. 
 Proc. ACM CHIIR 2016. http://dx.doi.org/10.1145/2854946.2854957
  • 19. 3. Setup • User study (26 participants; 24 analyzed) • Undergrads Univ. of Nottingham (6 F, 12 M, 18-25y) • Experimental SUI resembling common Search Engine • Within-participants • Task stage independent variable • Task design: explicit multistage approach
  • 20. 3. Setup: Multistage Task Design sim. work task: writing essay subtask subtask subtask prepare list of 
 3 topics choose topic;
 formulate specific
 question find and select 
 additional
 pages to cite 15 minutes 15 minutes 15 minutes initiation
 topic selection
 exploration focus formulation collecting presenting
  • 21. 3. Setup: Multistage Task Design sim. work task: writing essay subtask subtask subtask prepare list of 
 3 topics choose topic;
 formulate specific
 question find and select 
 additional
 pages to cite 15 minutes 15 minutes 15 minutes initiation
 topic selection
 exploration focus formulation collecting presenting General Assigned Topics (b/o discussions teaching staff) • Autonomous Vehicles (AV) • Virtual Reality (VR)
  • 22. 3. Setup: Protocol Training task Pre- Questionnaire Topic Assignment Introduction system Task Post-task Questionnaire 3x Post-experiment questionnaire Debriefing interview
  • 23. • Experimental system: SearchAssist • Results, Query Corrections, Query Suggestions: Bing Web API • Category Filters: DMOZ • Categorization and analysis: • Max Wilson’s framework of SUI features [Wilson11]
  • 30. 3. Setup: Data / Task details • AV & VR topics invoked comparable behaviours: • analysed as one topic set • Total duration main tasks • Total task time: 32:56 • 36.8% SUI, 33% Task screen, 30.2% Webpages Stage 1: 11:32 Stage 2: 8:24 Stage 3: 12:59
  • 31. Findings: Active Behaviour behaviour directly and indirectly derivable from logs4
  • 32. 4.1 Active Behaviour: Clicks 0 4 8 Sig. clicks on interface 
 features over time Stage 1 Stage 2 Stage 3
  • 33. 4.1 Active Behaviour: Clicks 0 4 8 Sig. clicks on interface 
 features over time Category filters ➡ Stage 1 Stage 2 Stage 3
  • 34. 4.1 Active Behaviour: Clicks 0 4 8 Sig. clicks on interface 
 features over time Category filters ➡ Tag Cloud ➡ Stage 1 Stage 2 Stage 3
  • 35. 4.1 Active Behaviour: Clicks 0 4 8 Sig. clicks on interface 
 features over time Category filters ➡ Tag Cloud ➡ Search button ➡ Stage 1 Stage 2 Stage 3
  • 36. 4.1 Active Behaviour: Clicks 0 4 8 Sig. clicks on interface 
 features over time Category filters ➡ Tag Cloud ➡ Search button ➡ Saved Results Stage 1 Stage 2 Stage 3
  • 37. 4.2 Active Behaviour: Queries •Mean number of queries** (unique): • Stage 1: 9.5 (8.1) ➡ Stage 2: 5.5 (5.1) ➡ Stage 3: 5.9 (5.3) 0 2,5 5 7,5 10 Stage 1 Stage 2 Stage 3 Search Box Query Suggestions Recent Queries
  • 38. 4.3 Active Behaviour: Query words •Mean number of query words**: “virtual reality” (P.02) “impact of virtual reality on society art and culture“ “autonomous vehicles” (P.06) “autonomous vehicles costs
 insurance industry” 0 1,25 2,5 3,75 5 Stage 1 Stage 2 Stage 3 Mean Number of Query words
  • 39. 4.4 Active Behaviour: Visited pages • Visited pages (unique)**: • Stage 1: 8.0 (7.3) • Stage 2: 6.4 (5.9) • dwell time highest • Stage 3: 14.2 (10.8) • Mean rank visited pages • from 3.1 to 6.4 0 4 8 12 16 Stage 1 Stage 2 Stage 3 Results List Saved Results
  • 40. 4.5 Active Behaviour: Wrapup • Clicks: • decreasing for Query Box (input), Category Filters & Tag Cloud (control) • increasing for Saved Results (personalizable) • Queries: • decreasing over time, but more complex • Popularity of certain features and impopularity of others: •Some features used in passive instead of active ways?
  • 41. Findings: Passive Behaviour behaviour not typically caught in interaction logs5 eyetribe.com
  • 42. Passive behaviour: mouse hovers • Mouse movements: • movements to reach a feature, also to aid processing contents [Rodden08] •Focus here on mouse movements not leading to click • Tendencies mostly overlap with active interaction measure 0% 25% 50% 75% 100% 1 2 3
  • 43. Passive behaviour: mouse hovers Category filters** ➡ • Mouse movements: • movements to reach a feature, also to aid processing contents [Rodden08] •Focus here on mouse movements not leading to click • Tendencies mostly overlap with active interaction measure 0% 25% 50% 75% 100% 1 2 3
  • 44. Passive behaviour: mouse hovers Category filters** ➡ Tag Cloud* ➡ • Mouse movements: • movements to reach a feature, also to aid processing contents [Rodden08] •Focus here on mouse movements not leading to click • Tendencies mostly overlap with active interaction measure 0% 25% 50% 75% 100% 1 2 3
  • 45. Passive behaviour: mouse hovers Category filters** ➡ Tag Cloud* ➡ Query Box** ➡ • Mouse movements: • movements to reach a feature, also to aid processing contents [Rodden08] •Focus here on mouse movements not leading to click • Tendencies mostly overlap with active interaction measure 0% 25% 50% 75% 100% 1 2 3
  • 46. Passive behaviour: mouse hovers Category filters** ➡ Tag Cloud* ➡ Query Box** ➡ Results List* ⤻ • Mouse movements: • movements to reach a feature, also to aid processing contents [Rodden08] •Focus here on mouse movements not leading to click • Tendencies mostly overlap with active interaction measure 0% 25% 50% 75% 100% 1 2 3
  • 47. 5.2 Passive Behaviour: eye fixations Stage 1 (exploration) Stage 2 (focus formulation) Stage 3 (postfocus, collection) • Overview of eye movement via heatmaps:
  • 48. Passive behaviour: eye tracking eye tracking fixations 0 25 50 75 100 1 2 3 • Further insights via eye tracking fixation counts • fixations > 80 ms, similar to e.g. [Buscher08]
  • 49. Passive behaviour: eye tracking eye tracking fixations 0 25 50 75 100 1 2 3 • Further insights via eye tracking fixation counts • fixations > 80 ms, similar to e.g. [Buscher08] Query Suggestions* ➡
  • 50. Passive behaviour: eye tracking eye tracking fixations 0 25 50 75 100 1 2 3 Tag Cloud* ➡ • Further insights via eye tracking fixation counts • fixations > 80 ms, similar to e.g. [Buscher08] Query Suggestions* ➡
  • 51. Passive behaviour: eye tracking eye tracking fixations 0 25 50 75 100 1 2 3 Category filters** ➡ Tag Cloud* ➡ • Further insights via eye tracking fixation counts • fixations > 80 ms, similar to e.g. [Buscher08] Query Suggestions* ➡
  • 52. Passive behaviour: eye tracking eye tracking fixations 0 25 50 75 100 1 2 3 Category filters** ➡ Tag Cloud* ➡ Query Box** ➡ • Further insights via eye tracking fixation counts • fixations > 80 ms, similar to e.g. [Buscher08] Query Suggestions* ➡
  • 53. Passive behaviour: eye tracking eye tracking fixations 0 25 50 75 100 1 2 3 Category filters** ➡ Tag Cloud* ➡ Query Box** ➡ Results List* ⤻ • Further insights via eye tracking fixation counts • fixations > 80 ms, similar to e.g. [Buscher08] Query Suggestions* ➡
  • 54. 3.4 Passive Behaviour: Active vs. Passive 0% 2% 4% 6% 8% Stage 1 Stage 2 Stage 3 Subtle differences between passive and active use:
  • 55. 3.4 Passive Behaviour: Active vs. Passive 0% 2% 4% 6% 8% Stage 1 Stage 2 Stage 3 Tag Cloud [5.8% fixations ⬌ 3.1% clicks] Subtle differences between passive and active use:
  • 56. 3.4 Passive Behaviour: Active vs. Passive 0% 2% 4% 6% 8% Stage 1 Stage 2 Stage 3 Query Suggestions [3.6% fix. ⬌ 1.9% clicks] Tag Cloud [5.8% fixations ⬌ 3.1% clicks] Subtle differences between passive and active use:
  • 57. 3.4 Passive Behaviour: Active vs. Passive 0% 2% 4% 6% 8% Stage 1 Stage 2 Stage 3 Query Suggestions [3.6% fix. ⬌ 1.9% clicks] Tag Cloud [5.8% fixations ⬌ 3.1% clicks] Recent Queries [3% fix. ⬌ 2% clicks] Subtle differences between passive and active use:
  • 58. 3.4 Passive Behaviour: Active vs. Passive 0% 2% 4% 6% 8% Stage 1 Stage 2 Stage 3 Query Suggestions [3.6% fix. ⬌ 1.9% clicks] Tag Cloud [5.8% fixations ⬌ 3.1% clicks] Recent Queries [3% fix. ⬌ 2% clicks] Subtle differences between passive and active use: Opposite for Category Filters [5% ⬌ 3.8%]
  • 59. 5.4 Passive Behaviour: Wrapup •Fixations & mouse moves • validating active behaviour • subtle differences active and passive use • Could subjective ratings and qualitative feedback provide more insights?
  • 60. Findings: Perceived Feature Utility perceived usefulness (post-stage & experiment)6
  • 61. 6.2 Perceived Usefulness: post-experiment • Post-experiment questionnaire: • In which stage or stages were SUI features most useful? • Pronounced differences • significant differences for all features 0% 25% 50% 75% 100% Query Box / 
 Results List Category
 Filters Tag 
 Cloud Query 
 Suggestions Recent 
 Queries Saved 
 Results
  • 62. 6.2 Perceived Usefulness: post-experiment • Post-experiment questionnaire: • In which stage or stages were SUI features most useful? • Pronounced differences • significant differences for all features 0% 25% 50% 75% 100% Query Box / 
 Results List Category
 Filters Tag 
 Cloud Query 
 Suggestions Recent 
 Queries Saved 
 Results
  • 63. 6.2 Perceived Usefulness: post-experiment • Post-experiment questionnaire: • In which stage or stages were SUI features most useful? • Pronounced differences • significant differences for all features 0% 25% 50% 75% 100% Query Box / 
 Results List Category
 Filters Tag 
 Cloud Query 
 Suggestions Recent 
 Queries Saved 
 Results
  • 64. 6.3 Perceived Usefulness: Category Filters • “good at the start (…) but later I wanted something more specific” (P.11) • common remarks in 2nd and 3rd stage: • “… could be more specific in its categories” • “…hard to find the category I want” (P. 27)
  • 65. 6.3 Perceived Usefulness: Tag Cloud • at the start: • “…aids exploring the topic” (P.06); • “came up with words that I hadn’t thought of” • later stages: • “doesn’t help to narrow the search much” (P.18) • “in the end seemed to be too general” (P.07)
  • 66. 6.3 Perceived Usefulness: Tag Cloud • at the start: • “…aids exploring the topic” (P.06); • “came up with words that I hadn’t thought of” • later stages: • “doesn’t help to narrow the search much” (P.18) • “in the end seemed to be too general” (P.07) • Post-experiment comments: • “…was good at the beginning, because when you are not exactly sure what you are looking for, it can give inspiration” (P.12) • “… nice to look at what other kinds of ideas [exist] that maybe you didn’t think of. Then one word may spark your interest” (P.15)
  • 67. 6.3 Perceived Utility: Query Suggestions • “…was good at the start but as soon as I got more specific into my topic, that went down” (P.11) • “clicked [it] .. a couple of times .. it gave me sort of serendipitous results, which are useful” (P.24)
  • 68. 6.3 Perceived Utility: Recent Queries • Naturally: “…most useful in the end because I had more searches from before” (P.26) • “The previous searches became more useful ‘as I made them’ because they were there and I could see what I searched before. I was sucking myself in and could work by looking at those.” (P.23) • May aid searchers in 
 their information journey..
  • 69. 6.3 Perceived Utility: Saved Results • “most useful in the end” (P.12) • “At the start [I was] saving a lot of general things about different topics. Later on I went back to the saved ones for the topic I chose and then sort of went on from that and see what else I should search” (P.26) • “I just felt I was organizing my research a little bit” (P.18) • It “helps me to lay out the plans of my research”.
  • 71. 0%! 20%! 40%! 60%! 80%! 100%! Stage 1! Stage 2! Stage 3! Percentageofparticipants! input / informational! control! personalisable! Stage 2! Stage 3! input / informational! control! personalisable! Conclusion: Findings Summary • Informational features highly useful in most stages • Decreasing use of input features • Control features decreasingly useful • likely caused by a user’s evolving domain knowledge • Personalizable features increasingly useful • ‘growing’ with a user’s understanding, task management support SUI features perceived as most useful, per stage
  • 72. 7. Conclusion: theoretical roundup complex information seeking task pre-focus stage: • vague understanding • limited domain knowledge • trouble expressing information need • large amount of new information • explaining prominent role of control features • explore information • filter result set using [Kuhlthau04,Vakkari&Hakkala00,Vakkari01]
  • 73. 7. Conclusion: theoretical roundup complex information seeking task pre-focus stage: • vague understanding • limited domain knowledge • trouble expressing information need • large amount of new information • explaining prominent role of control features • explore information • filter result set focus formulation stage: • more directed search • better understanding • seeking more relevant information, using differentiated criteria • control features become less essential • “not specific enough” • personalizable feat’s more important: may “grow” with emerging understanding using [Kuhlthau04,Vakkari&Hakkala00,Vakkari01]
  • 74. 7. Conclusion: theoretical roundup complex information seeking task pre-focus stage: • vague understanding • limited domain knowledge • trouble expressing information need • large amount of new information • explaining prominent role of control features • explore information • filter result set focus formulation stage: • more directed search • better understanding • seeking more relevant information, using differentiated criteria • control features become less essential • “not specific enough” • personalizable feat’s more important: may “grow” with emerging understanding postfocus stage • specific searches • re-checks additional information • precise expression • low uniqueness, high redundancy of info • long, precise, queries • further decline of control features • frequent use of personalizable features • “see what else to search” using [Kuhlthau04,Vakkari&Hakkala00,Vakkari01]
  • 75. 7. Conclusion: Future Work •Our study: essay writing simulated work task • Extension to other types of complex tasks, user populations •Further research into task-aware search systems • additional features may be useful at different stages • e.g. user hints, assistance • improvement of current features
  • 76. Towards “stage-aware” Systems prefocus focus formulation postfocus searchersearch
 system search
 interface search 
 stage stage-independent functionalityranking stage-dependent functionality manual or automatic
 detection
  • 77. • INEX/CLEF Interactive Social Book Search Lab • http://social-book-search.humanities.uva.nl/#/interactive • Gaede, Hall, Huurdeman, Kamps, Koolen, Skov, Toms, Walsh (2015) • Aim: support different stages in the search process: browse, search & review • Joint study across universities, 192 participants Multistage interface: search Multistage interface: browse Multistage interface: review Example: multistage interface
  • 78. 7. Conclusion: towards dynamic SUIs •Most Web search systems converged over static and familiar designs • trialled features often struggled to provide value for searchers • perhaps impeding search [Diriye10] if introduced in simple tasks, or at the wrong moment •Our work provides insights into when SUI features are useful during search episodes • potential responsive and adaptive SUIs for complex tasks
  • 79. References (1/2) [Ahlberg&Shneiderman94] C. Ahlberg and B. Shneiderman. Visual information seeking: Tight coupling of dynamic query filters with starfield displays. In CHI, pages 313–317. ACM, 1994. 
 [Buscher08] G. Buscher, A. Dengel, and L. van Elst. Eye movements as implicit relevance feedback. In CHI’08 extended abstracts on Human factors in computing systems, pages 2991–2996. ACM, 2008. [Diriye10] A. Diriye, A. Blandford, and A. Tombros. When is system support effective? In Proc. IIiX, pages 55–64. ACM, 2010. [Diriye13] A. Diriye, A. Blandford, A. Tombros, and P. Vakkari. The role of search interface features during information seeking. In TPDL, volume 8092 of LNCS, pages 235–240. Springer, 2013. [Donato10] D. Donato, F. Bonchi, T. Chi, and Y. Maarek. Do You Want to Take Notes?: Identifying Research Missions in Yahoo! Search Pad. In Proc. WWW’10, pages 321–330, 2010. ACM. [GaedeEtAl15] Maria Gäde, Mark Hall, Hugo Huurdeman, Jaap Kamps, Marijn Koolen, Mette Skov, Elaine Toms, and David Walsh. Overview of the SBS 2015 interactive track. In CLEF’15 Working Notes. CEUR-WS, 2015. [Hearst09] M. A. Hearst. Search user interfaces. Cambridge University Press, 2009. [Hearst13] M. A. Hearst and D. Degler. Sewing the seams of sensemaking: A practical interface for tagging and organizing saved search results. In HCIR. ACM, 2013. [Huurdeman&Kamps15] Hugo C. Huurdeman and Jaap Kamps (2015). Supporting the Process: Adapting Search Systems to Search Stages. In: S. Kurbanoğlu, S. Špiranec, J. Boustany, E. Grassian, D. Mizrachi, & L. Roy (Eds.), Information Literacy: Moving towards sustainability, Communication in Computer and Information Science series (Vol. 552, pp. 394-404).
 [Huurdeman&Kamps14] H. C. Huurdeman and J. Kamps. From Multistage Information-seeking Models to Multistage Search Systems. In Proc. IIiX’14, pages 145–154, 2014. ACM [Kuhlthau91] C. C. Kuhlthau. Inside the search process: Information seek- ing from the user’s perspective. JASIS, 42:361–371, 1991. [Kuhlthau04] C. C. Kuhlthau. Seeking Meaning: A Process Approach to Library and Information Services. Libraries Unlimited, 2004. [Kules12] B. Kules and R. Capra. Influence of training and stage of search on gaze behavior in a library catalog faceted search interface. JASIST, 63:114–138, 2012. [LiuBelkin15] J. Liu and N. J. Belkin. Personalizing information retrieval for multi-session tasks. JASIST, 66(1):58–81, Jan. 2015. [Marchionini06] G. Marchionini. Exploratory search: from finding to understanding. CACM, 49(4):41–46, 2006. [Niu14] X. Niu and D. Kelly. The use of query suggestions during information search. IPM, 50:218–234, 2014. [Proulx06] P. Proulx, S. Tandon, A. Bodnar, D. Schroh, W. Wright, D. Schroh, R. Harper, and W. Wright. Avian Flu Case Study with nSpace and GeoTime. In Proceedings of the IEEE Symposium on Visual Analytics Science and Technology (VAST'06). IEEE, 2006.
  • 80. References (2/2) [Toms11] E. G. Toms. Task-based information searching and retrieval. In Interactive Information Seeking, Behaviour and Retrieval. Facet, 2011. [Rodden08] K. Rodden, X. Fu, A. Aula, and I. Spiro. Eye-mouse coordination patterns on web search results pages. In CHI’08 Extended Abstracts, pages 2997–3002. ACM, 2008. [Shneiderman05] B. Shneiderman and C. Pleasant. Designing the user in- terface: strategies for effective human-computer interaction. Pearson Education, 2005. [Tunkelang09] D. Tunkelang. Faceted search. Synthesis lectures on information concepts, retrieval, and services, 1(1):1–80, 2009. [Vakkari01] P. Vakkari. A theory of the task-based information retrieval process: a summary and generalisation of a longitudinal study. Journal of Documentation, 57:44–60, 2001. [White05] R. W. White, I. Ruthven, and J. M. Jose. A study of factors affecting the utility of implicit relevance feedback. In SIGIR, pages 35–42. ACM, 2005. [White09] R. W. White and R. A. Roth. Exploratory search: Beyond the query-response paradigm. Synthesis Lectures on Information Concepts, Retrieval, and Services, 1:1–98, 2009. [Wilson&schraefel08] M. L. Wilson and m. c. schraefel. A longitudinal study of exploratory and keyword search. In In Proc. JCDL’08, pages 52–56. ACM, 2008. [Wilson99] T. D. Wilson. Models in information behaviour research. Journal of Documentation, 55:249–270, 1999. [Wilson11] M. L. Wilson. Interfaces for information retrieval. In I. Ruthven and D. Kelly, editors. Interactive Information Seeking, Behaviour and Retrieval. Facet, 2011.
  • 81. From Exploration to Construction
 How to Support the Complex Dynamics of Information Seeking 
 
 Hugo C. Huurdeman University of Amsterdam @TimelessFuture webarchiving.nl