![EVALUATION OF UNRANKED RETRIEVAL SYSTEM
Unranked retrieval system
Confusion Matrix
Recall [Relevant documents retrieved: Total relevant documents]
Precision [Relevant documents retrieved: Total documents]
Inverse Recall [documents irrelevant and not retrieved: Total irrelevant documents
F- measure [Recall + Precision]
Prevalence [irrelevant: total documents]
Accuracy [relevant record retrieved: total documents]
Error Rate [irrelevant record retrieved : total documents]
Fallout [False positive cases]
Miss Rate [False negative]
7](https://image.slidesharecdn.com/thecomparativestudyofinformationretrievalmodelsusedinsearchengines-181229165937/85/The-comparative-study-of-information-retrieval-models-used-in-search-engines-7-320.jpg)
![EVALUATION CRITRIA FOR RANKDED RETRIVAL
SYSTEM
Ranked retrieval system
K- Precision and R-Precision [K-precision refers to finding results on web search in
ranked documents on first few pages ]
Average precision [Precession is calculated at every point when a ranking list is
matching a relevant document ]
Mean average precession [user wants large number of documents at a same time
for example research papers ]
8](https://image.slidesharecdn.com/thecomparativestudyofinformationretrievalmodelsusedinsearchengines-181229165937/85/The-comparative-study-of-information-retrieval-models-used-in-search-engines-8-320.jpg)
The comparative study of information retrieval models used in search engines
- 1. The comparative study of Information Retrieval Models used in Search Engines By Muhammad Fawad (01-243181-011) Bilal Hussain (01-243181-09) MS(CS)- 2A Bahria University Islamabad, Campus 1
- 2. Outlines Introduction to information retrieval Information retrieval models Evaluation criteria of unranked documents evaluation criteria of ranked documents Introduction to search engines Concludes the findings 2
- 3. Information Retrieval Find out what user need and do it efficiently Collection: set of documents Why: Due to different format of data Goal: to retrieve information relevant to user needs Applications: Text search, image / video search, email search etc. Issues: relevance [useful, topic related, new, interesting, authentic, up-to- date. Understanding query syntax Understanding search engines 3
- 4. Information retrieval models Process of storing and retrieving structure and unstructured data Determine the predictions of what is relevant and what is not A particular way of “looking at things” Documents representation, Query representation, retrieval functions Boolean Model: Exact match , use Boolean logic to combine terms AND, OR , NOT, interaction and union Simple and easy to implement Exact match may lead to retrieve too few or too many documents Retrieve documents are not ranked Uses: Commercial document database systems Example: User Need: I am interesting in learning about vitamins other than vitamin e that are anti-oxidants User’s Boolean Query: antioxidant AND vitamin AND NOT vitamin e 4
- 5. Information retrieval models Vector Space Model: Documents and queries are treated as vectors Each term is associated with weights Weights determine degree of similarity between documents stored in the system and the user’s query Retrieve partial matched and ranked documents Similarity between vectors is determined by inner or dot product Complex in longer documents Index terms are assumed to be mutually independent Probabilistic Model Retrieve extract match or partially match Documents are ranked in decreasing order of their probability Doesn't maintain the frequency with which an index term occurs inside a document 5
- 6. Sr. No Parameters Boolean Model Vector Space Model Probabilistic Model 1 Concept It evaluates queries as evaluating Boolean expressions It uses concept of indexed weights and partial matching to match a document to a query It evaluates the queries by using the ideal set probabilistic index terms 2 Representation Weights are binary. The document is either relevant or irrelevant Index terms are weighted. So, there is ranking created based on these weights Weights are binary. Initially the document either belongs to the ideal set or is considered irrelevant 3 Type of Information Doesn't considers any semantic information Conceders semantic information's Conceders semantic information's 4 Advantage It is simple to evaluate based on query and the document Simple Not restricted to words only since it replaces ‘keywords’ by ‘concepts’ 5 Disadvantage Does not ranked the documents and performance is not that good It is more complex than binary as the index term weighting needs to be This is the most complex model since neither the weights nor the ideal set is initially defined 6 Word Occurrence Does not tell about the number of occurrence Tells about the number of occurrence Tells about the number of occurrence by terms document matrix 7 Output Exact match Best match Best match Comparison of Classical models 6
- 7. EVALUATION OF UNRANKED RETRIEVAL SYSTEM Unranked retrieval system Confusion Matrix Recall [Relevant documents retrieved: Total relevant documents] Precision [Relevant documents retrieved: Total documents] Inverse Recall [documents irrelevant and not retrieved: Total irrelevant documents F- measure [Recall + Precision] Prevalence [irrelevant: total documents] Accuracy [relevant record retrieved: total documents] Error Rate [irrelevant record retrieved : total documents] Fallout [False positive cases] Miss Rate [False negative] 7
- 8. EVALUATION CRITRIA FOR RANKDED RETRIVAL SYSTEM Ranked retrieval system K- Precision and R-Precision [K-precision refers to finding results on web search in ranked documents on first few pages ] Average precision [Precession is calculated at every point when a ranking list is matching a relevant document ] Mean average precession [user wants large number of documents at a same time for example research papers ] 8
- 9. SEARCH ENGINES User interface Gathers data Presents information to end user after applying suitable sorting algorithms Traditional Search Engine: Based on grammar rules, easy to understand Low recall and low precision Semantic Search Engine: Intelligent 9
- 10. Conclusion Search engine performance should maximized by using a hybrid search engine model 10