Interactive informationretrieval 토인모_201202
5 likes3,763 views
The document summarizes research on interactive information retrieval, including models of information seeking behavior, evaluation of IR systems with users, and the effect of search engine brand awareness on evaluation of search results. It discusses models like Berrypicking, sense-making theory, Ellis's information seeking process, and Kuhlthau's search process model. It also covers standard IR evaluation methods and usability testing techniques. Finally, it summarizes a study that showed people's quality assessments of the same search results can differ depending on the search engine brand.
![Models of the Information Seeking (4)
§ Ingwersen's cognitive model
§ 'traditional' model
§ represents IR as a two prong set (system and user) of elements and processes converging
on comparison or matching...';
§ Ingwersen's cognitive model [27], (cognition)
§ concentrates on identifying processes of cognition which may occur in all the
information processing elements involved.](https://image.slidesharecdn.com/interactiveinformationretrieval201202-120318090932-phpapp01/85/Interactive-informationretrieval-_201202-16-320.jpg)
![Trapit, Discovery Engine
[Curation]
• 인간의 요소, 인간만이 - 패턴을 인식하는 인간 고유의 능력 @ <큐레이션, 인간을 지향하다>
"프로그래머와 큐레이터로서 인간의 역할이 사라지는 일은 일어나지 않을 거에요. 컴퓨터가 절대로 따라올 수
없는 부분이 있으니까요. 그게 바로 인간의 요소, 인간만이 떠맡을 수 있는 부분이죠"](https://image.slidesharecdn.com/interactiveinformationretrieval201202-120318090932-phpapp01/85/Interactive-informationretrieval-_201202-53-320.jpg)
Interactive informationretrieval 토인모_201202
- 1. 토요인지모임 2012 2월 발표 Interactive Information Retrieval 2012.2.18 박정아 sunseed9@gmail.com http://twitter.com/sunseed9
- 2. Contents § Interactive Information Retrieval § Models of Information Seeking Behavior § Evaluation & Relevance § Search User Interfaces § Towards User-centered Search
- 4. Background - 컴퓨터과학 (Computer Science) + 국어국문학 - 자연어처리 (Natural Language Processing) - 인지과학 (Cognitive Science) - 정보검색 (Information Retrieval) Interacti Human Computer
- 5. 검색
- 8. Human centered approach in Information Retrieval § Cognitive viewpoint in information retrieval § Traditional IR model concentrates on matching not user side & interaction § interactive IR § more than simply developing interfaces for searching § Shneiderman, Byrd, & Croft, 1998 § the strength of good research in IIR comes not only from a technical knowledge of interactive systems development but also from a knowledge of people’s search behavior and search context, including the environmental factors that influence behavior § Fidel & Pejtersen, 2004 § HCIR (Human Computer Information Retrieval) § Human-Computer Interaction + Information Retrieval § HCI & IR § HCI and IR come from different traditions; § HCI, for example, places more emphasis on the published literature on usability § whereas IR emphasizes effectiveness.
- 9. Interactive information retrieval (IIR) § “ Interactive information retrieval (IIR) is, of its nature, cognitive. One important goal for IIR research is to study user interaction with a search system to learn about the user’s search intent and when they encounter relevant documents. Taking account of the user and her context has potential to improve understanding of the search process and the user’s intent. A search system that “knows” this information can improve its performance in retrieving documents that satisfy user’s needs. Awareness of demands imposed on user’s cognitive processing and levels of user’s knowledge can also contribute to improvements in system performance” § from "Inferring Cognitive States from Multimodal Measures in Information Science" § interactive IR is more than simply developing interfaces for searching (Shneiderman, Byrd, & Croft, 1998) and that the strength of good research in IIR comes not only from a technical knowledge of interactive systems development but also from a knowledge of people’s search behavior and search context, including the environmental factors that influence behavior (Fidel & Pejtersen, 2004).
- 10. Interactive Information Research problems § IIR research addresses three major problem areas § (1) understanding information seeking needs and behaviors; § (2) developing retrieval systems that respond to information needs and support information seeking behaviors and interactions; § (3) developing methods and measures to study and evaluate behaviors, interactions and systems. § issues § information seeking behavior related information needs and query intent § Models of the Information Seeking Process § Design of Search User Interfaces § Presentation of Search Results § includes document surrogates, properties of results listings, summaries (snippets) as used in search results § How people evaluate IR systems / Search Quality
- 11. Models of information seeking behavior
- 12. Models of the Information Seeking Behavior § Bates’ berrypicking – acts in searching § Dervin's sense-making theory – gap, bridge § Ellis’s Information Seeking Process § Kuhlthau’s information search process § Ingwersen's cognitive model § Wilson's information-seeking behaviour model § Saracevic's model of stratified interaction
- 13. Models of the Information Seeking (1) § Bates' Berrypicking - “dynamic” not “static”
- 14. Models of the Information Seeking (2) § Dervin's sense-making theory questioning that can reveal the nature of a problematic situation, the extent to which information serves to bridge the gap of uncertainty, confusion, or whatever, and the nature of the outcomes from the use of information. Figure 3: Dervin's 'sense-making' § A problem-solving mode § The solution of the problem, the resolution of the discrepancy, the advance from uncertainty to certainty
- 15. Models of the Information Seeking (3) § Ellis’s Information Seeking Process Figure 5: A process model based on Ellis's 'characteristics' § Kuhlthau’s information search process Figure 2.2: Kuhlthau's model of the search process
- 16. Models of the Information Seeking (4) § Ingwersen's cognitive model § 'traditional' model § represents IR as a two prong set (system and user) of elements and processes converging on comparison or matching...'; § Ingwersen's cognitive model [27], (cognition) § concentrates on identifying processes of cognition which may occur in all the information processing elements involved.
- 17. Models of the Information Seeking (5) § Saracevic's model of stratified interaction § Saracevic then goes on to propose what he calls a 'stratified interaction model' developed within an overall framework of an 'acquisition-cognition-application' model of information use. The levels or strata posited by Saracevic are simplified (in his words) to three: 1. surface, or the level of interaction between the user and the system interface; 2. cognition, or the level of interaction with the texts or their representation 3. situation, or the context that provides the initial problem at hand. Figure 9: Saracevic's model of stratified
- 19. Evaluation of interactive information retrieval systems with users § THE EVALUATION OF SEARCH USER INTERFACES § Measure § search interfaces are usually evaluated in terms of three main aspects of usability: effectiveness, efficiency, and satisfaction, which are defined by ISO 9241-11, 1998 as: § Effectiveness § Accuracy and completeness with which users achieve specified goals. § Efficiency § Resources expended in relation to the accuracy and completeness with which users achieve goals. § Satisfaction § Freedom from discomfort, and positive attitudes towards the use of the product.
- 20. Evaluation of interactive information retrieval systems with users § STANDARD INFORMATION RETRIEVAL EVALUATION § Text REtrieval Conference (TREC), run by the U.S. National Institute of Standards (NIST) for more than 15 years (Voorhees and Harman, 2000) § The most common evaluation measures used for assessing ranking algorithms § Precision, Recall, F-measure, Mean Average Precision (MAP). § The TREC evaluation method has been enormously valuable for comparison of competing ranking algorithms. § discounted cumulative gain (DCG) (Järvelin and Kekäläinen, 2000, Kekäläinen, 2005)
- 21. USABILITY TESTING - the results of an eye-tracking study Arrows indicate dominant directions of eye movement; “hotter” colors indicate more frequent eye fixations, and X's indicate locations of clicks. From Search Engine Results: 2010, by Enquiro Research
- 22. The Dynamics of relevance judgment and its order effect § dynamics of relevance § change-in-meaning hypothesis => direct impression § subject's relevance judgments would vary as a consequence of the order of presentation § first received information affected participants' impression of the following information § learning effect (Harter, 1992) § fatigue effects ( clancy & wachsler 1971) § information item이 어느정도로 증가하면 subject는 너무 fatigue해서 carefully하게 반응 어려움. § the dynamics of relevance judgment § (a) the changing external task situation and demand that modify a user’s information need (Cuadra & Katter, 1967; Park, 1993); (b) the changing cognitive state of the user as a result of encountering relevant documents (Harter, 1992; Xu, 2007b); (c) the different modes of document presentation § such as showing only the title, the abstract, bibliographic information, or the whole document content (Ingwersen & Järvelin, 2005). § belief-adjustment model § Hogarth & Einhorn 1992
- 23. Summary styles compared in an experiment by Aula, 2004.
- 24. Brand Awareness and the Evaluation of Search Results 검색 엔진 브랜드가 검색 결과 품질에 미치는 영향 (The Effect of Brand Awarenes On the Evaluation of Search Engine Results) • 연구 내용 요약 ◦ 사람들에게 같은 검색 결과가 제시되어도 검색 엔진이 무엇이냐에 따라 검색 품질 평가가 달라진다는 것을 재미 있는 실험을 통해 증명함으로써, 검색 엔진 브랜드가 검색 품질 평가의 한 요소임을 밝힌 연구 • 실험 방법 및 결과 ◦ 사용한 쿼리 ■ camping mexico, laser removal, manufactured home, techo music ■ 150만개 가량의 e-commerce 검색 로그로 부터 분야별로 4개의 쿼리 선정 ◦ 검색 결과 ■ 선정된 4개의 검색 쿼리를 구글 검색에 던져서 검색 결과 저장 ◦ 검색 로고 ■ 각각 Google, MSN, Yahoo의 로고들을 캡쳐하여 사용하고, AI2RS라는 새로운 검색 브랜드 로고 추가로 생성하여 사용 • 실험 대상 ◦ 18~25세 사이의 미국 대학생 32명( 남자 24명, 여자 8명) • 실험 결과 ■ 결과적으로 이름 없는 AI2RS 는 평균적으로 10% 떨어지는 평가를 받았다. ■ Yahoo는 4개 쿼리 모두 평균 이상의 평가를 받으며 높은 브랜드 인지도를 나타내었다. • 결론 ◦ 검색 성능(품질) 평가에서 검색 엔진 브랜드의 영향을 살펴본 결과, 동일 한 검색 결과라도 검색 엔진 브랜드에 대한 인식에 따라 검색 품질 평가에 상당한 영향을 미칠 수 있다는 것을 알 수 있다.
- 26. 적합성 (Relevance) • 시스템 중심 적합성 vs 사용자 중심 적합성 시스템 중심 사용자 중심 논리적 (Cooper, 1971) 심리적 (Wilson, 1973) 주제적 (Cooper, 1971; Park, 1994) 상황적 (Wilson, 1973; Harter, 1993) 객관적 (Swanson, 1986; Howard, 1994) 주관적 (Swanson, 1986; Howard, 1994) 적합성 분류 (출처. Maglaughlin & Sonnenwald, 2002) • 시스템 중심 적합성 – 벡터 스페이스 적합성(vector space relevance), 확률 적합성 (probabilistic relevance), 불린 적합성(Boolean relevance) • Borlund, 2003
- 27. 사용자 중심 적합성 • 사용자 중심 적합성 연구 활발 • 특히 90년대 연구 집중 – Froehlich, 1994; Green, 1995; Harter, 1992; Janes, 1994; Mizzaro, 1998; Park, 1994; Saracevic, 1996; Schamber, Eisenberg, & Nilan, 1990 • 사용자 중심 적합성에 대한 정의 – “적합성이란 다차원의 인지적 개념으로써 사용자의 정보 인식과 정 보 이용자의 정보 요구 상황에 상당 부분 의존한다” (Borlund, 2003) – 상황 적합성(Schamber, 1990), 심리적 적합성(Harter, 1992), 과제 기반 적합성 (Cosijn, 2000; Reid 1999) 등
- 28. Saracevic의 적합성 분류(1996) • 시스템 중심 적합성 – 시스템 적합성 • 검색어와 문서간의 유사도 • 사용자 중심 적합성 – 주제 적합성 • 검색어와 문서의 주제 – 인지 적합성 • 문서가 정보 이용자의 지식 상태와 인지적인 정보 요구에 얼마나 잘 부합하는지 – 상황 적합성 • 문서가 상황이나 현재 문제에 얼마 나 잘 적합한지
- 29. Saracevic’s stratified model of IR interaction Context social, cultural Situational tasks; work context... TA Affective RA intent; motivation ... ST tion R Cognitive USE pta knowledge; structure... Ada Query CE characteristics … N VA Surface level LE RE INTERFACE tion N& a TIO orm Engineering AC f of in ER hardware; connections... INT ion R Processing Use UTE ptat software; algorithms … MP Ada Content CO inf. objects; representations... Tefko Saracevic 28
- 30. 사용자 적합성 판단 기준 연구들 출처 도메인 참가자 판단 기준 개수 Schamber, 1991 날씨정보검색 직장인 30명 10 Park, 19913 석사논문연구주제 검색 대학원생 11명 22 Barry, 1994 온라인 정보 검색 학생 18명 23 Spink et al. 1998 연구 정보 검색 교수와 학생 11명 27 Bateman, 1998 논문검색 대학원생 35명 40 Wang & Sorel, 1998 연구 프로젝트 검색 대학원생 25명 11 Tang & Solomon, 1998 기말 논문 검색 대학원생 1명 10 Hirsh, 1999 스포츠 검색 초등학생 10명 14 Maglaughlin & 논문검색 대학원생 12명 29 Sonnenwald, 2002 Choi & Rasumuseen, 이미지 검색 학생 38명 9 2002 Savolainen & Kari, 2006 웹 검색 학생 9명 18
- 31. My Dissertation 정보 검색에서의 사용자 중심 적합성 판단 모형 개발 및 평가 • 개요 (Overview) – 한국 통합 검색 환경에서의 사용자 적합성 판단에 관한 연구 – 정보 검색 과제 별 사용자가 적합성을 판단하는 기준과 적합성 유형과의 관계 • 연구1. 사용자 적합성 판단 기준에 관한 탐색적 연구 – 한국 통합 검색 환경 – 반구조(semi-structured) 인터뷰 • 연구2. 사용자 적합성 판단 모형 개발 – 정보 검색 과제별 적합성 판단 기준과 적합성 유형의 관계 – Xu & Chen 적합성 판단 기준 정량적 연구 기반 – 반제어(semi – controlled) 설문 – “적합성 유형”별, “정보 검색 과제”별 • 연구 3. 정보 검색 과제별 동적 검색 랭킹 모델 구현 및 검증 – 사용자 적합성 판단 기준 랭킹 요소로 정보 검색 시스템 반영 – 사용자 평가 비교 실험 • 정적 검색 랭킹 모델 vs 동적 검색 랭킹 모델
- 32. 정보 검색 과제 • 정보 검색 과제에 따른 사용자 행동 또는 반응 기존 연구 Navarro-Prieto, White, Jose, Kelly et al. Limberg, L. Scaife, & Ruthven Freund (2008) This research (2002) (1999) & Rogers (1999) (2003) Fact finding Fact-finding fact search Fact finding 사실 검색 (fact question) ex) a named Fact finding person’ s current Ex)“How long does email address it take to get a passport? Procedural Understanding b a c k g r o u n d Learning 문제 해결 검색 a topic search (task question) ex) dust allergies How to Ex) “How do I get a passport?” Solve a problem Exploratory decision search Make a decision 의사 결정 검색 assessing an ex) find Rome’ s issue a n d best museum reaching a f o r decision impressionist art
- 35. Presentation of Search Results (Ranked List) Search results listings from Infoseek in 1997 (left) and Google in 2007 (right), courtesy Jan Pedersen.
- 36. Designing Search for Humans - Provide Memory Aids Suggest the Search Action in or near the Query Form www.yelp.com, www.powerset.com
- 37. Memory Aids Provide Access to Recent Actions PubMed amazon.com Dumais et al., Stuff I’ve Seen, SIGIR 2003
- 38. Memory Aids; Anchoring Aids Dynamic Query Suggestions http://www.daum.net http://google.com 38
- 39. Suggest Next Steps: Query suggestions Show suggestions after the query has been issued. http://yahoo.com http://bing.com 39
- 40. Suggest Next Steps: Query suggestions PubMed 40 http://nextbio.com
- 41. Putting It All Together: Faceted Navigation § Suggests next steps § Helps with Vocabulary Problem and Anchoring Problem § Promotes Flow § Show users structure as a starting point, rather than requiring them to generate queries § Organize results into a recognizable structure § Eliminates empty results sets 41
- 42. A New Development: Faceted Breadcrumbs Nudelman, http://www.boxesandarrows.com/view/faceted-finding-with 42
- 44. Search is not easy Frowns, Sighs, and Advanced Queries -- How does search behavior change as search becomes more difficult?
- 45. Instant Search 43
- 47. Interface in Mobile (1) 청각적(voice) 정보를 이용한 검색 시각적(visual) 정보를 이용한 검색
- 48. Interface in Mobile (2) 초성검색 통합웹 컷오프 모바일 화면의 제약으로 인한 컴팩트한 검색 결과 제공
- 50. 'Natural' Search User Interfaces § 'Natural' Search User Interfaces | November 2011 | Communications of the ACM § 사람들은 자연스러운(natural) 인터페이스 - 타이핑 입력 보다 말하기를, 텍스트 읽기 보다 동영상 보기 - 를 원하 고, 혼자가 아닌 함께(social)를 원한다. § "Users will speak rather than type, watch video rather than read, and use technology socially rather than alone" § 따라서 검색 인터페이스는 natural 과 social 을 지원하는 방향으로 발전해 가야한다. § Siri / Social Search
- 51. Siri, artificially intelligent voice search assistant
- 52. 소셜 검색 (What is Social search?)
- 53. Trapit, Discovery Engine [Curation] • 인간의 요소, 인간만이 - 패턴을 인식하는 인간 고유의 능력 @ <큐레이션, 인간을 지향하다> "프로그래머와 큐레이터로서 인간의 역할이 사라지는 일은 일어나지 않을 거에요. 컴퓨터가 절대로 따라올 수 없는 부분이 있으니까요. 그게 바로 인간의 요소, 인간만이 떠맡을 수 있는 부분이죠"
- 54. Beyond Search “Contextual discovery will take data gathered from people’s browsing data and location profiles and use it to serve up interesting and relevant results – without the search”
- 55. Thank You! sunseed9@gmail.com http://twitter.com/sunseed9