Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web
Download as PPTX, PDF1 like1,015 views
The document discusses the COPE project, which aims to identify coping strategies used by visually disabled users while interacting with the web and develop methods for automatic detection of these strategies. It proposes a four-step methodology involving observation, implementation of detection algorithms, deployment into the real world, and user studies to gather data and improve web accessibility. The findings indicate an overlap in the problems faced by users with disabilities and mainstream users, emphasizing the need for better accessibility solutions.
Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web
- 1. Detecting Problematic Interactions on the Web { The COPE Project: Coping strategies on the Web markel.vigo@{ehu.es | manchester.ac.uk} @markelvigo Donostia, 18/11/2013
- 2. Motivation The Web is a complex environment
- 3. Motivation Ceaparu,I., Lazar,J., Bessiere,K., Robinson,J., and Shneiderman,B. (2004) Determining causes and severity of end-user frustration. International Journal of Human-Computer Interaction 17(3), 333–356.
- 4. Problems Design Unfamiliar conventions Lack of skills
- 5. Detecting problems Post-hoc analysis: 87% detection accuracy (Feild, 2010) - Self-reported frustration - Usage metrics: query length, task time. Feild, H.A., Allan, J. and Jones, R. (2010) Predicting searcher frustration. ACM SIGIR ’10, 34–41.
- 6. Detecting problems Task-driven vs. emergent tasks Top-down vs. bottom up Constrained vs. ecological SERP vs. navigation and more
- 7. Reactions to problems Reactions are cognitive markers that indicate problems Ask for help (Novick, 2007) Other means (Ceaparu, 2004) - "I found an alternative solution" "I figure out how to fix myself" Novick, D. G., Elizalde, E., and Bean, N. (2007) Toward a more accurate view of when and how people seek help with computer applications. ACM SIGDOC '07, 95–102. Ceaparu, I., Lazar, J., Bessiere, K., Robinson, J., and Shneiderman, B. (2004) Determining causes and severity of end-user frustration. International Journal of Human-Computer Interaction 17(3), 333–356.
- 8. Identifying strategies Reactions are cognitive markers that indicate problems If we are able to automatically detect coping we can provide the means to overcome the situation
- 9. Identifying strategies Challenges - Low frequency Non-overtly manifested Ecological setting (get out of the lab!) Consequences - - Significant amount of observations into the wild Time and personnel++
- 10. Hypothesis We can transfer strategies from populations who cope more frequent and overtly to general audiences
- 11. Evidence Overlap of problems between: - - The constrains imposed by reduced screens and keyboards (situational impairments) Problems encountered by motor impaired desktop users Same scope, different magnitude Yesilada,Y., Harper,S., Chen,T., Trewin,S. (2010) Small-device users situationally impaired by input. Computers in Human Behavior 26 (3), 427–435. Simon Harper, Yeliz Yesilada, Tianyi Chen (2011) Mobile device impairment...similar problems, similar solutions? Behaviour & IT 30(5),673-690
- 12. Hypothesis There is an accessibility continuum with overlapping - Problems Strategies Solutions SCOPE lower access end people with disabilities MAGNITUDE situational impairment able bodied users higher access end
- 13. Hypothesis MAGNITUDE Yesilada et al. (2010) Harper et al. (2011) SCOPE visuallyphysically mobile disabledimpaired users able bodied users lower access end higher access end Our goal is more ambitious: explore the commonalities of distant populations. - Visually disabled users who cope Able bodied users encountering difficulties
- 14. Our proposal 4 step method: 1. 2. 3. 4. Observation and identification of strategies Implementation of algorithms to detect strategies Deployment into the wild Run user studies Go to step 2 until obtaining reliable results Vigo, M. and Harper, S. (2013) Considering people with disabilities as überusers for eliciting generalisable coping strategies on the web. ACM Web Science, 441-444.
- 15. Step 1. Observation and identification of strategies 2 independent ethnographic studies and user tests 24 screen reader and screen magnifier users 8 coping strategies (17 implementations) were identified Vigo, M and Harper, S. (2013) Coping tactics employed by visually disabled users on the web. International Journal of Human-Computer Studies 71(11), 1013–1025.
- 16. Step 1. Observation and identification of strategies T1. Asking for assistance T2. Impulsive clicking - - I21. Deliberately clicking on lowscented links I22. Clicking on any link I23. Clicking on any link as long as it is accessible T3. Exploration tactics - - - I31. Persevering I32. Escaping from content by moving down I33. Move down without listening to content I34. Move around T5. Gaining orientation - I51. Gaining local orientation I52. Gaining global orientation T6. Redoing - I61. Re-checking I62. Re-typing I63. Re-tracing I64. Re-starting T7. Not operating - - I71. Delegating on assistive technology I72. Swapping assistive technology I73. Waiting T8. Giving up T4. Narrowing down search Vigo, M and Harper, S. (2013) Coping tactics employed by visually disabled users on the web. International Journal of Human-Computer Studies 71(11), 1013–1025.
- 17. Step 1. Further analysis I We analyse the skill acquisition process of visually disabled users over time A decrease in last resort tactics is observed across sessions In parallel, users exhibit more exploration tactics and explore more websites 1.0 2.5 0.8 2.0 tactics tactics tactics ASK GUP NOP 0.4 0.2 1.5 EXP tactics 0.6 NDO ORI 1.0 RED 0.5 0.0 0.0 1 2 3 sessions 4 5 1 2 3 4 5 sessions Vigo, M and Harper, S. (under review) A snapshot of the first encounters of visually disabled users with the Web.
- 18. Step 1. Further analysis I While effectiveness increases, coping success does not Misuse of tactics may be indicative of a larger repertoire or increased confidence. 1.0 performance 0.8 0.6 Performance effectiveness coping success 0.4 0.2 0.0 1 2 3 4 5 sessions Vigo, M and Harper, S. (under review) A snapshot of the first encounters of visually disabled users with the Web.
- 19. Step 1. Further analysis II Users are not always driven by information scent Navigation is driven by the need to overcome an interaction problem Users escape from the current web patch - - Users click on misleading links if by doing this problems are averted. Low satisfacing levels: in extraordinary situations any web patch is “good enough”. Vigo, M and Harper, S. (2013) Challenging information foraging theory: screen reader users are not always driven by information scent. ACM Conference on Hypertext and Social Media, 60-68
- 20. Our proposal 4 step method: 1. 2. 3. 4. Observation and identification of strategies ✔ Implementation of algorithms to detect strategies Deployment into the wild Run user studies Go to step 2 until obtaining reliable results Vigo, M. and Harper, S. (2013) Considering people with disabilities as überusers for eliciting generalisable coping strategies on the web. ACM Web Science, 441-444.
- 21. Step 2. Implementation of algorithms to detect strategies I61. Re-checking: quick revisitations webpagei→wpj→wpiwpj I63. Re-tracing: users retrace the steps in a sequence of pages. wpj→wpk→wpl→wpm→wpl→wpk→wpj→wpk→wpl→wpm
- 22. Our proposal 4 step method: 1. 2. 3. 4. Observation and identification of strategies ✔ Implementation of algorithms to detect strategies ✔ Deployment into the wild Run user studies Go to step 2 until obtaining reliable results Vigo, M. and Harper, S. (2013) Considering people with disabilities as überusers for eliciting generalisable coping strategies on the web. ACM Web Science, 441-444.
- 23. Step 3. Deployment into the wild WebTactics 1.1 Detection t1(){ algorithms are .. injected onto web } pages. ti(){ .. } tn(){ .. } 2. When the use of a tactic is detected a notification is triggered. 3. This event is asynchronously sent to a remote location. 4. Reports from users are viewed by interested parties. DB User Website Manager /Researcher 1.2 Each algorithm keeps track of determined sequences of events and actions. id timestamp URL tactic ccgu1331569030153 1333922552190 http://www.bbc.co.uk/sport/ t5 ccgu1331569030153 1333922556391 http://www.bbc.co.uk/sport/ t1 Vigo, M. and Harper, S. (2013) Evaluating Accessibility-in-use. W4A '13, article 7.
- 24. Step 3. Deployment into the wild Vigo, M. and Harper, S. (2013) Evaluating Accessibility-in-use. W4A '13, article 7.
- 25. Our proposal 4 step method: 1. 2. 3. 4. Observation and identification of strategies ✔ Implementation of algorithms to detect strategies ✔ Deployment into the wild ✔ Run user studies Go to step 2 until obtaining reliable results Vigo, M. and Harper, S. (2013) Considering people with disabilities as überusers for eliciting generalisable coping strategies on the web. ACM Web Science, 441-444.
- 26. Step 4. Run user studies 18 participants, 10 days 126 retraces and 67 rechecks Tabbed browsing was interfering Feedback on false positives: - “I’m browsing across tabs” “I’m comparing different web pages” “I’m navigating through different tabs” Vigo, M. and Harper, S. (in progress) Detecting navigation tactics to cope on the World 'Wild' Web. Vigo, M. and Harper, S. (2013) Considering people with disabilities as überusers for eliciting generalisable coping strategies on the web. ACM Web Science, 441-444.
- 27. Our proposal 4 step method: 1. 2. 3. 4. Observation and identification of strategies ✔ Implementation of algorithms to detect strategies ✔ Deployment into the wild ✔ Run user studies ✔ Go to step 2 until obtaining reliable results Vigo, M. and Harper, S. (2013) Considering people with disabilities as überusers for eliciting generalisable coping strategies on the web. ACM Web Science, 441-444.
- 28. Step 2. Implementation of algorithms to detect strategies I61. Re-checking: quick revisitations webpagei→wpj→wpiwpj NONTABBED NONTABBED NONTABBED I63. Re-tracing: users retrace the steps in a sequence of pages. wpj→wpk→wpl→wpm→wpl→wpk→wpj→wpk→wpl→wpm NONTABBED NONTABBED NONTABBED NONNONNONTABBED TABBED TABBED NONTABBED NONTABBED NONTABBED
- 29. Our proposal 4 step method: 1. 2. 3. 4. Observation and identification of strategies ✔ Implementation of algorithms to detect strategies ✔✔ Deployment into the wild ✔✔ Run user studies ✔ Go to step 2 until obtaining reliable results Vigo, M. and Harper, S. (2013) Considering people with disabilities as überusers for eliciting generalisable coping strategies on the web. ACM Web Science, 441-444.
- 30. Step 4. Run user studies 20 participants, 10 days 24 retraces and 16 rechecks 1.0 1.0 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0.0 study 1 study 2 I63. Retracing 0.0 study 1 study 2 I61. Rechecking Vigo, M. and Harper, S. (in progress) Detecting navigation tactics to cope on the World 'Wild' Web. Vigo, M. and Harper, S. (2013) Considering people with disabilities as überusers for eliciting generalisable coping strategies on the web. ACM Web Science, 441-444.
- 31. Conclusions There is an overlap on problems and strategies More evidence supporting the accessibility continuum Implications for accessibility-inuse
- 32. Future work Ongoing work 4th iteration with users - Testing with visually disabled users 2nd stage of the COPE project - - - Intervening in the interface Long term objective Triangulation
- 33. Detecting Problematic Interactions on the Web { questions? markel.vigo@{ehu.es | manchester.ac.uk} @markelvigo Donostia, 18/11/2013
Editor's Notes
- #5: Design-> accessibility, usability, information architecture | Guidelines not able to catch user experience due subjective dimensionsConventions -> cultural or linked to mental models | not generalisableLack of skills -> we don't know how to measure learniability