![Context vs. Intention?
Context is any
information that can
be used to
characterize the
situation of an
entity. An entity is a
person, […]
noun
(1) thing intended; an
aim or plan
[…]](https://image.slidesharecdn.com/mlux-cbmi-130618040032-phpapp02/85/CBMI-2013-Presentation-User-Intentions-in-Multimedia-18-320.jpg)
![Crowd-Sourcing
• It’s hard to judge intentions of others
– That makes it error prone
“a reminder of the beautiful Island
were [sic] my father came from”
o Recall situation
o Preserve good feelings
o Publish online
o Show to family & friends
o Support task of mine
o Preserve bad feeling](https://image.slidesharecdn.com/mlux-cbmi-130618040032-phpapp02/85/CBMI-2013-Presentation-User-Intentions-in-Multimedia-37-320.jpg)
![turkers
Recall situation 2 2 0 1 0 2
Preserve good feeling 2 -2 1 0 0 1
Publish online 2 0 0 0 1 2
Show to family & friends 2 1 2 1 1 0
Support task of mine 0 -2 1 1 1 -2
Preserve bad feeling -2 0 -2 0 -2 -2
“a reminder of the beautiful Island
were [sic] my father came from”
o Recall situation
o Preserve good feelings
o Publish online
o Show to family & friends
o Support task of mine
o Preserve bad feeling](https://image.slidesharecdn.com/mlux-cbmi-130618040032-phpapp02/85/CBMI-2013-Presentation-User-Intentions-in-Multimedia-38-320.jpg)
CBMI 2013 Presentation: User Intentions in Multimedia
- 1. User Intentions in Visual Information Retrieval & Multimedia Information Systems Mathias Lux
- 2. User Intentions in Visual Information Retrieval & Multimedia Information Systems Mathias Lux
- 4. Query By Example • User has particular information need • Need reflected by example image • Query is expressed visually
- 5. We all know that ... • Some features work better than others • Features have different characteristics • Some features work out well for some domains, while others don’t
- 11. PHOG & Birds on the Water
- 13. Which one is right? • How to determine the right feature? • What are the necessary characteristics? • How do I define visual similarity within the domain? • What is visual similarity for the user?
- 14. Why is there a different ranking?
- 15. Users in Context
- 16. Definition: Context “Context is any information that can be used to characterize the situation of an entity. An entity is a person, place, or object that is considered relevant to the interaction between a user and an application, including the user and applications themselves.” Ref. G. Abowd et al., “Towards a better understanding of context and context-awareness. In Handheld and Ubiquitous Computing, vol. 1707 LNCS, 1999.
- 17. Definition: Intention noun (1) thing intended; an aim or plan (2) Medicine the healing process of a wound (3) (intentions) Logic conceptions formed by directing the mind towards an object
- 18. Context vs. Intention? Context is any information that can be used to characterize the situation of an entity. An entity is a person, […] noun (1) thing intended; an aim or plan […]
- 19. A User’s Intention is • part of a user’s context • of manageable size (verb & frame) • related to the information need in search Examples • I want to download a new background for my mobile. • I want to share the first laugh of my daughter. • I want to see what a Lancia Lyra looks like.
- 20. User Intentions in the Web Underlying goals of web searches • Informational – to learn / know something • Navigational – to go to a specific place (on the web) • Transactional – to go somewhere to ultimately buy sth. Ref. Broder: A Taxonomy of Web Search. SIGIR Forum, 2002
- 21. User Intentions in the Web Ref. Rose & Levinson: Understanding user goals in web search, WWW 2004 Broder Survey Broder Log Analysis R&L St udy 1 R&L St udy 2 R&L St udy 3 24,5 20 14,7 11,7 13,5 39 48 60,9 61,3 61,5 36 30 24,3 27 25 Navigat ional Inf ormat ional Transact ional
- 22. User Intentions n Multimedia • Search • Production • Sharing • Archiving • Image • Video • Audio • Multiple modalities
- 23. Hand-picked Examples Right now there is no all-in-one publication on user intentions in multimedia …
- 24. User Intentions in Image Search Ref. Lux, Kofler & Marques: A classification scheme for user intentions in image search, CHI 2010 beautiful sunset for the background of my mobile old vs. new Starbucks logo my friend’s new car on flickr.com Latest “explore” photos
- 25. Do queries help with the search intention? User information need vs. query formulation in video search. • How to support users with video indexing and search methods? • Search goal failure is (partially) predictable – Based on keywords and – Based on natural language Ref. Kofler, Larson & Hanjalic: To Seek, Perchance to Fail: Expressions of User Needs in Internet Video Search, ECIR 2011
- 26. Asking for the “Why?” behind the “What?” Ref. Hanjalic, Kofler & Larson: Intent and its Discontents: The User at the Wheel of the Online Video Search Engine, ACM MM 2012 Hear others singing … Learn to sing …
- 27. Asking for the “Why?” behind the “What?” • Information – news, commercial, advertisement, documentary, science, commentary, education, learning, … • Experience – tutorial, how-to, advise, help, training • Affect – books, podcast, music, comedy, series, art, movie, action, gaming, film, episode, entertainment, … Ref. Hanjalic, Kofler & Larson: Intent and its Discontents: The User at the Wheel of the Online Video Search Engine, ACM MM 2012
- 28. Helping with clever Uis? Ref. Lagger, Lux & Marques: An Adaptive Video Retrieval System Based On Recent Studies On User Intentions While Watching Videos Online. ACM CIE, online.
- 29. Who are the Users in a Video Search System? • Study on users of – YouTube – BBC iPlayer – Uitzending Gemist Ref. Kemman, Kleppe& Beunders: Who are the users of a video search system? Classifying a heterogeneous group with a profile matrix, WIAMIS 2012
- 30. Who are the Users in a Video Search System? Ref. Kemman, Kleppe & Beunders: Who are the users of a video search system? Classifying a heterogeneous group with a profile matrix, WIAMIS 2012
- 31. Why do People make Videos? • Study on four main goals: – Affection, Function, Sharing & Preservation. Ref. Lux & Huber: Why did you record this video? WIAMIS 2012 Sharing Af f ect ion Funct ion - 0,59 - 0,7 8 - 0,36 - 0,05 - 0,26 - 0,36 0,39 0,84 0,55 - 0,50 - 0,93 0,25 - 0,07 0,46 0,21 - 0,43 - 0,21 0,47 Preservat ion Sharing Af f ect ion
- 32. Finding User Intentions & Goals is a hard task ….
- 33. Demand Media – The Answer Factory • Demand mined from search queries • Requests for content put on auction • Contractors create content • Crowd does quality control see i.e. eHow.com The Answer Factory: Demand Media and the Fast, Disposable, and Profitable as Hell Media Model, http://www.wired.com/magazine/2009/10/ff_demandmedia/
- 36. Human Computation • Is crowd-sourcing of any help here? – cp. ACM MM & work of Kofler, Larson & Hanjalic
- 37. Crowd-Sourcing • It’s hard to judge intentions of others – That makes it error prone “a reminder of the beautiful Island were [sic] my father came from” o Recall situation o Preserve good feelings o Publish online o Show to family & friends o Support task of mine o Preserve bad feeling
- 38. turkers Recall situation 2 2 0 1 0 2 Preserve good feeling 2 -2 1 0 0 1 Publish online 2 0 0 0 1 2 Show to family & friends 2 1 2 1 1 0 Support task of mine 0 -2 1 1 1 -2 Preserve bad feeling -2 0 -2 0 -2 -2 “a reminder of the beautiful Island were [sic] my father came from” o Recall situation o Preserve good feelings o Publish online o Show to family & friends o Support task of mine o Preserve bad feeling
- 39. Crowd-Sourcing • Turkers disagreed with original publishers. • But pretests had better inter-rater agreements. Ref. Lux, Taschwer & Marques: A Closer Look at Photographers’ Intentions: a Test Dataset, CrowdMM WS at ACM MM 2012 Int ent ions Ot her t urkers 0,147 0,232 pret est s 0,57 1 0,510
- 40. Human Computation • How about motivating people, i.e. with fun & rewarding experience?
- 41. Games with a additional Purpose
- 42. Games with a additional Purpose • Tag a Tune • Popvideo • Matchin • Flip It • Verbosity
- 43. Games with a additional Purpose • How to go beyond annotation? – classical applications are focused on annotation • How to infer user intentions? – proves to be hard to “guess” intentions of others – even “own” intentions may not be explicit • How to leverage user intentions? – finding which intentions can be leveraged and which goals can be supported
- 45. Where did we go? • CBIR & QBE • User Intentions – Search – Production – Sharing • Games with additional Purpose
- 46. What is left? • Lots of loose ends & open grounds for research … … let me propose four different PhD theses …
- 47. Open PhD Theses I • General Model for User Intentions & Goals in Multimedia. – Is there a unified model? – What are the class cardinalities? – How to map production, archiving, search and sharing intentions?
- 48. Open PhD Theses II • GWAP, HC & UIs for determining & inferring & utilizing User Intentions & Goals – Which UI elements, game mechanics and HC mechanics help in this scenario? – What are appropriate design patterns and scenarios? – What is an appropriate research methodology and how to (easily) evaluate?
- 49. Open PhD Theses III • Bringing Context to the Query in Multimedia Information Systems. – How to utilize Intentions & Goals within search and indexing methodology? – Building MMIS around a model for user intentions.
- 50. Open PhD Theses IV • Adaptable Applications – How to adapt an application to users’ intentions? – Which elements & process to display, etc.?
- 51. Thanks for listening … • Mathias Lux • Klagenfurt University, AT • mlux@itec.uni-klu.ac.at