The smart retrieval experiment ppt
- 1. THE SMART RETRIEVAL EXPERIMENT I.LOGIDHASAN DLIS 2ND YEAR PONDICHERRY UNIVERSITY PONDICHERRY
- 2. INTRODUCTION The SMART system was designed in 1964, largely as an experimental tool for the evaluation of the effectiveness of many different types of analysis and search procedures. Salton characterizes the system through the following steps of its function. It is used to: Take documents and search queries posed in English Perform a fully automatic content analysis of texts Match analyzed search statements and contents of documents Retrieve the stored items which are most similar to the queries.
- 3. SMART system overview Thesaurus look-up procedures Phrase generation methods Statistical term associations Hierarchical term expansion, and so on.
- 4. Evaluation procedures Through the original SMART experiments were conducted in a laboratory environment, the basic aim was to develop a prototype for a fully automated information retrieval system. The evaluation procedures incorporated into the system lent themselves into a pair-wise comparison of the effectiveness of two or more processing methods.
- 5. Cont.. The following evaluation measures were generated by the SMART system: A recall-precision graph reflecting the average precision value at ten discrete recall points-from a recall of 0.1 to a recall of 1.0 in intervals of 0.1 To global measure, known as normalized recall and normalized precision, which together reflect the overall performance level of the system Two simplified global measures, known as rank recall and log precision, respectively.
- 6. Methodology A collection of 1268 abstracts in the field of library science and documentation, comprising about 131,500 English text words, was used for this experiment. The collection contained articles mainly published in American documentation in 1963 and 1964, and also in other journals in the given subject area.
- 7. Cont.. Thus, for each of the 48 queries, a set of four different document sets became available, each consisting of the items termed relevant by a different set of people as follows: A set – relevance assessed by the query author B set – relevance assessed by outside subject expert C set- relevance assessed by either A or B assessor D set- relevance asserted by both A and B assessor.
- 8. cont… The relevance judgment groupings were as follows: Group judges Function A Original group of query authors. Each person in the A group made relevance for his or her six queries. B Non-author judges. Each person in the B group made relevance judgments for six queries corresponding to six different authors from the A group. C The document is relevant to a given query if either the A judge or B judge termed it relevant. D The document is relevant to a given query if both A and B judges termed it relevant.
- 9. Results It was found that, under normal circumstances, an evaluation of performance for a variety of processing methods required an examination of the ranking of the corresponding recall-precision curves, rather than a detailed comparison of the actual recall and precision values. From a ranking of the recall-precision graphs obtained from the several processing methods it was noted that
- 10. THANK YOU