INTRODUCTION TO INFORMATION RETRIEVALChapter 1-IR.ppt

Introduction to Information Retrieval
Demamu.M(MSC)

Course Overview
• What the course is about
– How people search and find information
– How computers store and retrieve information
– How computer systems are designed to help people to find
information they need
Demamu.M (MSC)

Course Overview
• What the course emphasize on
– understanding on theories, tools (lexical analyzers,
stemmers, etc.), algorithms (ranking, matching, clustering,
etc.) and evaluation on information retrieval systems
Demamu.M (MSC)

Knowledge Useful for the Course
• Mathematics (set theory, probability, vector algebra)
• Data structure
• File structure
• Linguistics (read papers on Linguistics in Information Science)
• System A & D
• Programming in higher level languages such as C, C++, Java,
VB, etc.
Demamu.M (MSC)

Chapter 1: Agenda
• Introduction (Motivation, Definitions of IR & IR Systems,
challenges of IR)
• Data Retrieval Vs Information Retrieval
• The Retrieval Process
• The structure of IR System
• List of important concepts and terms
Demamu.M (MSC)

Motivational Factors
1. Major observation on computers capability
• Computers are able to scan whole documents and decide on
whether they were relevant or not
• Question: What will happen if computers are not able
to do this?
• IR systems, since their inception, are in place to reduce a
user’s workload in searching through the store of documents
to find relevant ones
• The first IR systems, obviously, were very basic and were
not very effective
Demamu.M (MSC)

2. Information explosion
– Why information Explosion?
– How do you relate information overload with IR?
Demamu.M (MSC)

• 2 (cont’d)
– In IR we talk about information explosion/overload
• The rapid growth in the amount of information
• Finding a needle in haystack
– This helps to appreciate IR systems
– The growth in information and the retrieval mechanisms do
not match
– Our techniques to retrieve lags behind the growth of
information
• The speed at which you retrieve matters a lot these days
Demamu.M (MSC)

3. Information Need
– The two most important entities in IR are
• Information Items
• Information Needs (In IR we talk also about information
needs)
Demamu.M (MSC)

• 3 (cont’d)
– Some definitions of information need for our purpose
• Is what user want from the IR system
• Is a question that users ask
• Is the desire to know
• Is a desire to fill a gap of knowledge
• Information problem that cause the user to act
Demamu.M (MSC)

Information Retrieval
• The term Information Retrieval was first coined by Calvin Moore (1950)
• Is an Important sub-discipline of Information Science that is concerned with
developing theories and methods of access to information
– Focus is on helping user find information that matches their information
need (User Centered View)
• Is a branch of applied Computer Science that focus on representation, storage,
organization of, and access to information items (System Centered View).
• Is about finding relevant information in large collection of data
Demamu.M (MSC)

Information Retrieval (Definition)
• A good formal definition of information retrieval is given in Baeze-
Yates & Riberio-Neto (1990)
“Information retrieval deals with representation, storage, organization
of, and access to information items. The organization and access of
information items should provide the user with easy access to the
information in which he is interested”
• The definition incorporates all important features of a good information
retrieval system
– Representation
– Storage
– Organization
– Access
– evaluation Demamu.M (MSC)

Challenges in IR
• Representation of information items and information needs
(first problem)
– Document representation is one area of IR
– query representation is another area of IR
• Matching (second problem)
– How to match need Vs. information items
• Modification of representation as a result of judgment (query
expansion or reformulation)
Demamu.M (MSC)

Information Retrieval Systems
• Are systems build to retrieve documents highly likely relevant
to the user
• Are systems built to reduce user’s workload in searching
through the store of documents to find relevant one’s
• Are systems that give information about the presence or
absence of documents in accordance with the query
– Automated abstracts or summaries of documents were developed to
further simplify access to search results
• Are computer based systems (we are talking about automation )
Demamu.M (MSC)

• Are systems that attempt to find relevant documents to respond to
user’s request
• Are devices interposed between a potential user of information
and the information collection itself.
– For a given information problem, the purpose of the system is
to capture wanted items and to filter out unwanted items
Demamu.M (MSC)

What an IR system should do
• Store/archive information
• Provide access to that information
• Answer queries with relevant information
• Stay current
• WISH list
– Understand the user’s queries
– Understand the user’s need
– Acts as an assistant
Demamu.M (MSC)

A typical IR System
Collection of Documents
(Information items
Processor
Internal
Representation
of Documents
User Query/Request
Query Results
Retrieved objects
This is not a detailed schematic illustration of IRs
Demamu.M(MSC)

• Major functions
– Analyze contents of information items
– Represent the contents of the analyzed sources in a way
suitable for matching with users’ queries
– Analyze users information need and represent them in a
form, that will be suitable for matching with the database
– Match the search statement worth with the stored database
– Retrieve or generate information that are relevant in a
ranking which reflects relevance
– Make necessary adjustments in the system based on feedback
from users
Demamu.M (MSC)

Database Retrieval Vs. Information Retrieval
• Information items
– DBMS
• highly structured data (are of known nature), often
homogeneous records, often semantically unambiguous (well
defined semantics)
– IR systems
• Unstructured or unformatted data (as opposed to relational
database). When you go to a specific document it is not
structured as in DB
Demamu.M (MSC)

• Answers
– DBMS:
• Records, tuples, No ranking
• Well defined results
• Perfect precision and recall, each item is relevant
– IR systems
• Vs. Documents, ranked list of documents. The issue
ranking is very important (page through the top k
documents)
• Vs. Imperfect precision and recall, each item has specific
relevance
Demamu.M(MSC)

• Matching
– DBMS:
• Analoguous to db quering: Which docs contain a set of
keywords?
• Exact match; We talk of items that match exactly; Every record
either matches or fails to match a query; No notion of relevnce
• A single erroneous object implies failure!
– IR systems
• Information about a subject or topic
• Partial or best best match; We talk of possibly relevant items not
exact matched items
Demamu.M(MSC)

• Items wanted
– DBMS
• Matching
– IR systems
• relevant
Demamu.M(MSC)

• Query language
• DBMS
• Artificial language
– IR system
• Natural language
Demamu.M (MSC)

Structure of an IR System
• An Information Retrieval System serves as a bridge between the world
of authors and the world of readers/users,
• That is, writers present a set of ideas in a document using a set of
concepts,
Black box
User Documents
• The black box is the processing part of the information
retrieval system,
• It includes mainly indexing and searching
•
Demamu.M(MSC)

• Translation from user need to query
– Usually, manually ( by user himself)
– Tools available to assist the process
• Translation from item to representation
– Often, automatically (by the system)
– Representation can be at different level:
• Full text, abstract only, index terms only, etc.
Structure of an IR System
Demamu.M(MSC)

Quize 5%
• What is IR
• What is black box
• What is IRS
• Compare and contrast DBMS VS IRS
• What do you understand from chapter 1
Mebiratu B(MSC)

End of Chapter 1
Demamu.M(MSC)

Important Concepts and Terms
• Document
– Is (in theory, at least) taken as more-or-less synonymous
with the text in linguistics - that describes any pieces of
linguistic (in the widest sense) material that can reasonably
be considered as unit
Demamu.M(MSC)

• Information items (documents) - Usually text, but possibly also
image, audio, video, etc.
• Textual items/documents- may be of different scope (books,
scientific articles, paragraphs, newspapers, reports, email
message etc.)
• Graphical and multimedia items/documents- images, line
drawings, PPT presentations, web pages, moving pictures/video
• Spoken documents- sound recordings (voice messages, Radio
news, telephone conversation)
• Focus: we will consider textual documents
Demamu.M(MSC)

• Searching
– The way the file is examined and the items in it are taken as
related to a search query
• term
– A term is a semantic unit, a word, phrase, or potentially root
of a word
• Inverted file
– A stored list of index terms with each index term having
links to the documents containing that term.
– The inverted indexes can be extended to include
• Term location information
• Word numbers with in sentences
• Term weights Demamu.M(MSC)

• Query
– Is a request for documents pertaining to some topic
– Has usually been taken to mean the statement by the requester
describing his/her information need
• Database
– Is a collection of documents
Demamu.M(MSC)

Summary
• We have seen that an IR system deals with the sources of
information on the one hand and the users requirements on the
other hand.
• Generally, there are two major tasks (or functions) in an IR
system
- To analyze the contents of the sources of information
as well as well as well as the users’ queries and then
- To match both to retrieve those items which are
relevant.
• These functions can further be elaborated as follows:-
Demamu.M(MSC)

1. Identify the sources of information relevant to the areas of
interest of the target users’ community;
2. Analyze the contents of the sources (documents);
3. Represent the contents of the analyzed sources in a way
that will be suitable for matching with the users’ queries;
4. Analyze users’ queries and to represent them in a form
that will be suitable for matching with the database;
5. Match the search statement with the stored database;
6. Retrieve information that are relevant and
7. Make necessary adjustments in the system based on the
feedback from the user.
Summary
End Of Slide
Demamu.M(MSC)

INTRODUCTION TO INFORMATION RETRIEVALChapter 1-IR.ppt

More Related Content

Similar to INTRODUCTION TO INFORMATION RETRIEVALChapter 1-IR.ppt (20)

More from amarehope21 (14)

Recently uploaded (20)

INTRODUCTION TO INFORMATION RETRIEVALChapter 1-IR.ppt