Discovering Common Motifs in Cursor Movement Data
Download as PPTX, PDF1 like1,323 views
The document discusses research on discovering common motifs in mouse cursor movement data. It summarizes prior work on modeling post-click user behavior on search result pages. The researchers aim to automatically discover meaningful patterns (motifs) in cursor movement data without pre-defining complex features. They describe a pipeline to generate motif candidates, find frequent candidates, de-duplicate motifs, and apply various optimizations. Experimental results show motifs can improve relevance prediction and search result ranking. Motifs are also useful for characterizing attention patterns and predicting cognitive impairment.
![4
The Importance of Online User Attention
• “Eye-mind Hypothesis”
[Just and Carpenter, 1980]
• “When a subject looks at a
word or object, he or she
also thinks about (process
cognitively), and for
exactly as long as the
recorded fixation.”](https://image.slidesharecdn.com/yandex-motifs-2014-v2-140325023702-phpapp02/85/Discovering-Common-Motifs-in-Cursor-Movement-Data-4-320.jpg)
![Applications
Examination Strategies
[Buscher et al.]
Web Page Re-Design
[Leiva et al.]
Behavior Biased
Summaries
[Ageev et al.]
Query-Expansion &
Relevance Feedback
[Buscher et al.]
Parkinson, ADHD, FASD
[Tseng et al.]
Prediction of Cognitive
Impairment
[Zola et al.]
Search Relevance
[Guo & Agichtein]
Search Abandonment
[Huang et al.]
7](https://image.slidesharecdn.com/yandex-motifs-2014-v2-140325023702-phpapp02/85/Discovering-Common-Motifs-in-Cursor-Movement-Data-7-320.jpg)
![Applications
Examination Strategies
[Buscher et al.]
Web Page Re-Design
[Leiva et al.]
Behavior Biased
Summaries
[Ageev et al.]
Query-Expansion &
Relevance Feedback
[Buscher et al.]
Parkinson, ADHD, FASD
[Tseng et al.]
Prediction of Cognitive
Impairment
[Zola et al.]
Search Relevance
[Guo & Agichtein]
Search Abandonment
[Huang et al.]
8
Our focus](https://image.slidesharecdn.com/yandex-motifs-2014-v2-140325023702-phpapp02/85/Discovering-Common-Motifs-in-Cursor-Movement-Data-8-320.jpg)
![Prior Work:
Cursor Movement on Landing Pages
• Post Click Behavior Model [Guo and Agichtein, WWW 2012]
• Two basic patterns: “Reading” and “Scanning”
Reading Scanning
“Reading”: consuming or verifying when
(seemingly) relevant information is found
“Scanning”: not yet found the
relevant information, still in the
process of visually searching
13](https://image.slidesharecdn.com/yandex-motifs-2014-v2-140325023702-phpapp02/85/Discovering-Common-Motifs-in-Cursor-Movement-Data-13-320.jpg)
![Post-Click Behavior (PCB) Data Improves Ranking
• PCB and PCB_User consistently outperform
DTR (baseline)
14
[Guo & Agichtein, WWW 2012]
[Guo , Lagun & Agichtein, CIKM 2012]
DTR = Dwell time + Rank
NDCG](https://image.slidesharecdn.com/yandex-motifs-2014-v2-140325023702-phpapp02/85/Discovering-Common-Motifs-in-Cursor-Movement-Data-14-320.jpg)
![Mouse Cursor Data: Challenges
Different users examine web pages
with different speed, hence move
mouse slower/faster.
[Flexible Distance Metric, DTW]
Similar type of movements can
appear in different parts of a web
page (top vs. bottom).
[Location Invariance: normalize
subsequence position]
18](https://image.slidesharecdn.com/yandex-motifs-2014-v2-140325023702-phpapp02/85/Discovering-Common-Motifs-in-Cursor-Movement-Data-18-320.jpg)
![Optimizations in Similarity Search
• Early stopping
– in DTW computation (takes O(n^2) time)
– in lower bound computation (takes O(n) time)
[Keogh et al.]
• Parallel Computation
– No dependency in distance computation
use multiple cores
• Distance Metric Learning
• Spatial Indexing
25](https://image.slidesharecdn.com/yandex-motifs-2014-v2-140325023702-phpapp02/85/Discovering-Common-Motifs-in-Cursor-Movement-Data-25-320.jpg)
![Motifs for Relevance Prediction
• Baselines
– Cursor Hover (on the search result page)
[Huang et al., CHI 2011]
– Post Click Behavior Model
[Guo & Agichtein, WWW, 2012]
• Dwell time
• Statistics of cursor movements: max, min, range, etc.
• Statistics of scrolling activity: max, min, range, etc.
33
Reading Scanning](https://image.slidesharecdn.com/yandex-motifs-2014-v2-140325023702-phpapp02/85/Discovering-Common-Motifs-in-Cursor-Movement-Data-33-320.jpg)
![Dynamics in User Generated Content
Wikipedia
Major events (e.g., natural disasters, sports) affect the content change in Wikipedia articles.
Use content change for ranking:
• Words used in early revisions of the documents are more essential and important to
the documents.
• Words used during a major event may reflect relevance change between words and
documents
Twitter
Topic transitions in Tweet streams:
• What you’ve tweeted before may affect what you will tweet in the near feature.
Sentiment change in Twitter during major events:
• People respond differently to the same event since they could hold different prior
opinions. (e.g., conservatives vs. liberals)
Yu Wang (Ph.D. expected 2014)
[CIKM 2010, KDD 2012, CIKM 2013]](https://image.slidesharecdn.com/yandex-motifs-2014-v2-140325023702-phpapp02/85/Discovering-Common-Motifs-in-Cursor-Movement-Data-57-320.jpg)
![Community Question Answering (CQA)
1. What are the factors influencing answer
contributions in CQA Systems?
– Analyzing answerer behavior [ECIR 2011]
2. What kind of searches benefit most from CQA
services and archives?
– Understanding how searchers become askers [SIGIR 2011]
3. How to improve search quality with CQA data?
– Predicting searcher satisfaction with CQA data [SIGIR 2012]
Qiaoling Liu,
Ph.D. expected: 2014](https://image.slidesharecdn.com/yandex-motifs-2014-v2-140325023702-phpapp02/85/Discovering-Common-Motifs-in-Cursor-Movement-Data-58-320.jpg)
Discovering Common Motifs in Cursor Movement Data
- 1. Discovering Common Motifs in Cursor Movement Data Dmitry Lagun, 2014 Emory University 1
- 2. Thank you! 2 Mikhail Ageev Qi Guo Eugene Agichtein
- 3. 3 The Importance of Online User Attention • “Attention is focused mental engagement on a particular item of information.” (Davenport & Beck 2001, p. 20) Abundance of information Scarcity of attention
- 4. 4 The Importance of Online User Attention • “Eye-mind Hypothesis” [Just and Carpenter, 1980] • “When a subject looks at a word or object, he or she also thinks about (process cognitively), and for exactly as long as the recorded fixation.”
- 5. 5 The Importance of Online User Attention • Attention is critical for science of cognition (vision, language, memory) • Many industry applications: – Web search intent, quality, presentation, s atisfaction – UI usability testing – Display advertising, customer engagement, branding
- 6. Measurement of Attention • Eye Tracking – Based on corneal reflection of infra-red light Infra-red cameras Users spend most of the time on top search results 6
- 7. Applications Examination Strategies [Buscher et al.] Web Page Re-Design [Leiva et al.] Behavior Biased Summaries [Ageev et al.] Query-Expansion & Relevance Feedback [Buscher et al.] Parkinson, ADHD, FASD [Tseng et al.] Prediction of Cognitive Impairment [Zola et al.] Search Relevance [Guo & Agichtein] Search Abandonment [Huang et al.] 7
- 8. Applications Examination Strategies [Buscher et al.] Web Page Re-Design [Leiva et al.] Behavior Biased Summaries [Ageev et al.] Query-Expansion & Relevance Feedback [Buscher et al.] Parkinson, ADHD, FASD [Tseng et al.] Prediction of Cognitive Impairment [Zola et al.] Search Relevance [Guo & Agichtein] Search Abandonment [Huang et al.] 8 Our focus
- 9. emory math and cs 9 Search
- 13. Prior Work: Cursor Movement on Landing Pages • Post Click Behavior Model [Guo and Agichtein, WWW 2012] • Two basic patterns: “Reading” and “Scanning” Reading Scanning “Reading”: consuming or verifying when (seemingly) relevant information is found “Scanning”: not yet found the relevant information, still in the process of visually searching 13
- 14. Post-Click Behavior (PCB) Data Improves Ranking • PCB and PCB_User consistently outperform DTR (baseline) 14 [Guo & Agichtein, WWW 2012] [Guo , Lagun & Agichtein, CIKM 2012] DTR = Dwell time + Rank NDCG
- 15. Post-Click Behavior (PCB) Model Features • Average cursor position, cursor speed, direction • Travelled distance, horizontal and vertical ranges • Max/Min cursor positions on the screen • Scroll speed, frequency and scroll distance • Cursor position in a region-of-interest Can we automatically discover meaningful features of cursor trajectory? 15
- 16. Our Approach: Cursor Motif Mining Instead of engineering complex features, discover common subsequences (motifs) Motif is a frequently occurring sequence of cursor movements. Similar 16
- 17. Mouse Cursor Data: Challenges Different users examine web pages with different speed, hence move mouse slower/faster. Similar of movements can appear in different parts of a web page (top vs. bottom). 17
- 18. Mouse Cursor Data: Challenges Different users examine web pages with different speed, hence move mouse slower/faster. [Flexible Distance Metric, DTW] Similar type of movements can appear in different parts of a web page (top vs. bottom). [Location Invariance: normalize subsequence position] 18
- 19. Motif Discovery Pipeline Generate Motif Candidates Discover Frequent Candidates De-duplicate / Output Motifs Distance Measure 19
- 20. Candidate Generation window size sliding window Motif candidates 20
- 21. Distance Measure • Which time series are similar? • Popular Choices: – Euclidian Distance (ED) – Dynamic Time Warping (DTW) 21
- 22. Frequent Motif Mining • Similarity Search – How many subsequences in the dataset are similar to the given candidate subsequence? motif candidates motifcandidates dist(i,j) – how similar i-th candidate to the j-th motif candidate. Algorithm Parameters: max_dist – distance when two subsequences are considered “similar” min_count – minimal frequency of motif candidate 22 Brute force search is computationally expensive
- 23. De-Duplication (only keep cluster centroids) • Similarity search can generate a lot of frequent candidates that are similar between each other (due to redundancy in motif candidate generation) 23
- 24. Motif Discovery Pipeline Generate Motif Candidates Discover Frequent Candidates De-duplicate / Output Motifs Distance Metric 24
- 25. Optimizations in Similarity Search • Early stopping – in DTW computation (takes O(n^2) time) – in lower bound computation (takes O(n) time) [Keogh et al.] • Parallel Computation – No dependency in distance computation use multiple cores • Distance Metric Learning • Spatial Indexing 25
- 26. Distance Measure Learning • Goal: Fast pruning of not-promising candidates in similarity search Features (x_max, y_max, …, feature_k) Features (x_max, y_max, …, feature_k) 26 Tune the weights with Gradient based method (e.g. SGD)
- 27. Spatial Indexing • Goal: Fast pruning of not-promising candidates in similarity search • Indexes motif candidates in weighted feature space • Improves asymptotic time for similarity search 27
- 29. Example of Discovered Motif discovered motif 29 eye gaze mouse cursor matching subsequence
- 30. Motifs Discovery: Examples On Search Engine Result Pages (SERPs) On “Landing” Pages (non-SERPs) 30
- 31. Discovered motifs have many uses • Summarize typical mouse cursor usages – E.g. create dictionary of typical cursor usages • Compact (task-free) representation – Characterize entire cursor trajectory based on which motifs appear in it • For classification/regression: – Compute whether particular motifs appears in a given mouse cursor trajectory 31
- 32. Using motifs as features for Classification/Regression • We can measure how similar is mouse movement trajectory to each of the discovered motifs window size sliding window 32 motif
- 33. Motifs for Relevance Prediction • Baselines – Cursor Hover (on the search result page) [Huang et al., CHI 2011] – Post Click Behavior Model [Guo & Agichtein, WWW, 2012] • Dwell time • Statistics of cursor movements: max, min, range, etc. • Statistics of scrolling activity: max, min, range, etc. 33 Reading Scanning
- 34. Dataset • User study (21 users) – mostly informational search tasks – 566 search queries – 1340 page views – 854 relevance judgments 34
- 35. Motifs are Better than Previous Models (PCB, Hover) 35 Feature Group Pearson Correlation Cursor Hover 0.120 Post Click Behavior 0.392 Motifs 0.394 (+0.5%) Post Click Behavior + Motifs 0.468 (+19.4%)
- 36. Motifs are Helpful for Web Search Result Ranking 36
- 37. Conclusions • It is possible to automatically discover meaningful motifs from mouse cursor data • Motifs are helpful for relevance prediction & ranking • Cursor motifs provide compact (task free) representation for the entire cursor trajectory 37
- 38. Applications of Gaze/Mouse Cursor Tracking in Medical Domain 38
- 39. Background: Mild Cognitive Impairment (MCI) and Alzheimer’s Disease • Alzheimer’s disease (AD) affects more than 5M Americans, expected to grow in the coming decade • Memory impairment (aMCI) indicates onset of AD (affects hippocampus first) • Visual Paired Comparison (VPC) task: promising for early diagnosis of both MCI and AD before it is detectable by other means 39
- 40. VPC Task: Eye Tracking Equipment 40
- 41. Impaired Subjects spent 50% on Novel Image after Long Delay 41
- 42. VPC Task: Eye Tracking 42
- 43. Exploiting Eye Gaze Movement Data Novelty Preference fixation duration distribution + 43
- 44. Shapelets are Helpful for Prediction of Cognitive Decline • Shapelets – “class specific” motifs 44
- 45. Shapelets are Helpful for Prediction of Cognitive Decline • Shapelets – “class specific” motifs Baseline AUC = 0.892 ± 0.003 Shapelets AUC = 0.916 ± 0.006 45
- 46. User Attention on Web Pages 46
- 47. Cross-Domain User Study • Research Question – Does web page content affect user attention? • Domains – Search (Google), Wikipedia, Shopping (Amazon), Social (Twitter), News (CNN ) • 20 users (4 + 20 tasks per user) • 400 tasks, 1700 page views • 500K gaze/cursor measurements (sampled every 50 ms) 47 ? search domain X
- 52. Conclusions • It is possible to automatically discover meaningful motifs from mouse cursor data • Motifs are helpful for relevance prediction, ranking and prediction of cognitive impairment • Attention patterns vary significantly across search interfaces 52
- 53. Thank You! • This work was supported by 53
- 54. Emory IR Lab: Research Areas Modeling collaborative content creation for information organization, indexing, and search 54 • Mining search behavior data to improve information finding. Medical applications of Search, NLP, behavior modeling.
- 55. UFindIt: Remote Search Behavior Studies 55 Misha Ageev (MGU & Yandex), Dmitry Lagun (Emory), Denis Savenkov (Emory) SIGIR 2011 (best paper award), SIGIR 2013, EMNLP 2013
- 56. Search behavior models for Touch Screens Ongoing project, looking for students 56 Guo et al., SIGIR 2013
- 57. Dynamics in User Generated Content Wikipedia Major events (e.g., natural disasters, sports) affect the content change in Wikipedia articles. Use content change for ranking: • Words used in early revisions of the documents are more essential and important to the documents. • Words used during a major event may reflect relevance change between words and documents Twitter Topic transitions in Tweet streams: • What you’ve tweeted before may affect what you will tweet in the near feature. Sentiment change in Twitter during major events: • People respond differently to the same event since they could hold different prior opinions. (e.g., conservatives vs. liberals) Yu Wang (Ph.D. expected 2014) [CIKM 2010, KDD 2012, CIKM 2013]
- 58. Community Question Answering (CQA) 1. What are the factors influencing answer contributions in CQA Systems? – Analyzing answerer behavior [ECIR 2011] 2. What kind of searches benefit most from CQA services and archives? – Understanding how searchers become askers [SIGIR 2011] 3. How to improve search quality with CQA data? – Predicting searcher satisfaction with CQA data [SIGIR 2012] Qiaoling Liu, Ph.D. expected: 2014
- 59. • Emory IR Lab is looking for a few good Ph.D. students to start September 2015 • Information retrieval and web search: search behavior, ranking, user interfaces, content analysis, Question Answering • Social media and social network mining applications: political science, public health, advertising • Psychology, Neuroscience, Medicine applications: computational attention, memory, cognition, language Contact: Eugene Agichtein Associate Professor eugene@mathcs.emory.edu www.mathcs.emory.edu/~eugene/ 59 http://www.mathcs.emory.edu/programs-grad/ Computer Science Ph.D. Program information and application process:
- 60. 60 Atlanta, GA
Editor's Notes
- #4: Attention, Interest, Desire and Action
- #5: Just MA, Carpenter PA (1980) A theory of reading: from eye fixation to comprehension. Psychol Rev 87:329–354
- #13: Mouse movement allows how users read the web pagesWhy dwell time is not enough?
- #14: I
- #17: Motif is defined as …Show example of several similar motifs in a plot
- #21: We represent mouse movement trajectory with two 2-dimensional time series
- #22: Replace the explanation for DTW
- #24: Add definition of motif
- #26: Fix Showkeogh year
- #27: May be shorten to one slide (very distracting)
- #31: We can study differences in cursor usages on various web pages
- #33: Similar to binary version of bag of words
- #36: Emphasize similar performance for automatic vs. manual
- #57: Even when the interactions are available, the device formfactor requires re-intepretationUsers are increasingly expecting touch interactions