introintrointrointrointrointrointrointro

Course Overview:
An Introduction to Information
Retrieval and Applications
J. H. Wang
Feb. 17, 2014

IR, Spring 2014 www.Vidyarthiplus.com 2
Instructor & TA
• Instructor
– J. H. Wang ( 王正豪 )
– Associate Professor, CSIE, NTUT
– Office: R1534, Technology Building
– E-mail: jhwang@csie.ntut.edu.tw
– Tel: ext. 4238
– Office Hour: 9:00-12:00 am, every Tuesday and
Thursday
• TA
– TBD (R1424, Technology Building)

Course Description
• Course Web Page: for the latest announcements and
updates of schedule, slides, and homeworks
– http://www.ntut.edu.tw/~jhwang/IR/
• Time: 9:10-12:00am, Fri.
• Classroom: R334, Technology Building
• Textbook:
– Christopher D. Manning, Prabhakar Raghavan and Hinrich
Schuetze, Introduction to Information Retrieval, Cambridge
University Press, 2008.
• Available online
• International Student Edition, imported by Kai-Fa ( 開發 )
Publishing
• Prerequisites:
– Basic knowledge of data structures and algorithms, linear
algebra, and probability theory
– Programming experience is *required* for homeworks &
projects

Target Audience
• Seniors
• Master students
• IMEECS (International Master’s Program
in Electrical Engineering and Computer
Science)

Additional References
• References:
– Ricardo Baeza-Yates and Berthier Ribeiro-Neto,
Modern Information Retrieval: The Concepts and Tec
hnology behind Search
, Addison-Wesley, 2011.
• This is the second edition of their book
Modern Information Retrieval in 1999. ( 華通 )
– Bruce Croft, Donald Metzler, and Trevor Strohman,
Search Engines: Information Retrieval in Practice,
Addison-Wesley, 2010. ( 全華 )
– Stefan Buettcher, Charles L.A. Clarke, and Gordon V.
Cormack,
Information Retrieval: Implementing and Evaluating
Search Engines

More Books on IR
• Gerald Salton, Automatic information organization and
retrieval, McGraw-Hill, 1968.
• Gerald Salton and M.J. McGill, Introduction to modern
information retrieval, McGraw-Hill, 1983.
– Two classics, but out-of-print.
• C. J. van Rijsbergen, Information Retrieval, Butterworths,
1979.
– The classic. More than 40 years old, but still worth reading.
• K. Sparck Jones, P. Willett,
Readings in Information Retrieval, Morgan Kaufmann,
1997.
– A collection of classical IR papers. (out of print)
• I.H. Witten, A. Moffat, T.C. Bell. Morgan Kaufmann,
Managing Gigabytes, 2nd edition, 1999.
– The authority on index construction and compression.

Grading Policy
• Homework assignments and
programming exercises: ~40%
• Mid-term exam: ~25%
• Term project: ~35%
– Including proposal, presentation, and final
report

Programming Exercises and Term
Project
• About 3 programming exercises
– Team-based (at most 2 persons per team)
– You can either write your own code or reuse existing
open source code
• The term project
– Either team-based system development (the same as
programming exercises)
– Or academic paper presentation
• Only one person per team allowed
– A proposal is *required* before midterm (Apr. 11,
2014)

About the Term Project
• The score you get depends on the functions,
difficulty and quality of your project
– For system development:
• System functions and correctness
– For academic paper presentation
• Quality and your presentation of the paper
• Major methods/experimental results *must* be presented
• Papers from top conferences are strongly suggested
– E.g. SIGIR, WWW, CIKM, WSDM, JCDL, ICMR, …
• Proposals are *required* for each team, and will be counted
in the score

Online Submission
• Submission instructions
– Programs, project proposals, and project
reports in electronic files must be submitted to
the TA online at:
• Submissions website: (TBD)
– Before submission:
• User name: Your student ID
• Please change your default password at your first
login

What this Course is NOT about
• This course will NOT tell you
– The tips and tricks of using search engines,
although power users might have better ideas on how
to improve them
• There’re plenty of books and websites on that…
– How to find books in libraries,
although it’s somewhat related to the basic IR
concepts
– How to make money on the Web,
although the currently largest search engine did it

What’s Information Retrieval?
• Things that you have been doing all day!
– Searching for something interesting: Web, news,
e-mail, image, video, …
– Asking for advices
– …
• User interests are changing all the time…
– 2011: New Zealand Earthquake
– 2012: Jeremy Lin
– 2013: Meteor Russia
– 2014: ? (next slide)

What’s Information Retrieval

In Google News

In Wikipedia

In Google Images

In Google Video Search

In Google Translate…

Or More Related Keywords
• Blast
• Explosion
• Chelyabinsk
• Asteroid 2012 DA14
• …

What if We Search in Chinese

And More…
• 流星
• 彗星
• 隕石
• 俄羅斯
• 地球
• …
• And other languages…
• And other search engines…
• And social websites…

In Google Trends

And More…

And Other Keywords…

And Social Search…

How do I Know What People
Care about?

What are People Searching in
Taiwan?

What Is Information Retrieval?
• “Information retrieval is a field concerned
with the structure, analysis, organization,
storage, searching, and retrieval of
information.” (Salton, 1968)

Goal
• Information retrieval (IR): a research field
that targets at effectively and efficiently
searching information in text and
multimedia documents
• In this course, we will introduce the basic
text and query models in IR, retrieval
evaluation, indexing and searching, and
applications for IR

A Big Picture

Inverted
Index
User
Interface
Text Operations
Query
Expansion
Indexing
Retrieval
Ranking
Text
query
user need
user feedback
ranked docs
retrieved docs
Doc representation
logical view
inverted file
Document
Collection

Topics
• Text IR
– Indexing and searching
– Query languages and operations
• Retrieval evaluation
• Modeling
– Boolean model
– Vector space model
– Probabilistic model
• Applications for IR
– Multimedia IR
– Web search
– Digital libraries

Organization of the Textbook
• Basics in IR (focus)
– Inverted indexes for boolean queries (Ch.1-5)
– Term weighting and vector space model (Ch. 6-7)
– Evaluation in IR (Ch. 8)
• Advanced Topics
– Relevance feedback (Ch. 9)
– XML retrieval (Ch. 10)
– Probabilistic IR (Ch. 11)
– Language models (Ch. 12)
• Machine learning in IR (useful)
– Text classification (Ch. 13-15)
– Document clustering (Ch. 16-18)
• Web Search
– Web crawling and indexes (Ch. 19-20)
– Link analysis (Ch. 21)

Some Overlap with Other Fields
• Text mining
• Machine Learning
• Natural Language Processing
• Social Network Analysis
• …

Pointers to Other Topics
• Cross-language IR
• Image, video, and multimedia IR
• Speech retrieval
• Music retrieval
• User interfaces
• Parallel, distributed, and P2P IR
• Digital libraries
• Information science perspective
• Logic-based approaches to IR
• Natural language processing techniques
• …

Tentative Schedule
• Before midterm
– Boolean retrieval (1 wk)
– Indexing (2 wks)
– Vector space model and evaluation (2 wk)
– Relevance feedback (1 wk)
– Probabilistic IR (2 wk)
• After midterm
– Text classification (1-2 wk)
– Document clustering (1-2 wk)
– Web search (2 wks)
– Advanced topics: CLIR, IE, … (2 wks)
– Term Project Presentation (3 wks)

Generic Resources
• Wikipedia page on Information Retrieval:
http://en.wikipedia.org/wiki/Informatio
n_retrieval
• Information Retrieval Resources:
http://www-csli.stanford.edu/~hinrich/i
nformation-retrieval.html
•

Academic Resources
• Journals
– ACM TOIS: Transactions on Information Systems
– JASIST: Journal of the American Society of Information Sciences
– IP&M: Information Processing and Management
– IEEE TKDE: Transactions on Knowledge and Data Engineering
• Conferences
– ACM SIGIR: International Conference on Information Retrieval
– WWW: World Wide Web Conference
– ACM CIKM: Conference on Information Knowledge and
Management
– JCDL: ACM/IEEE Joint Conference on Digital Libraries
– ACM WSDM: International Conference on Web Search and Data
Mining
– TREC: Text Retrieval Conference

Teaching in English…
• Slides and lectures will be offered mainly
in English
• For better understanding for domestic
students, important concepts will be
briefly summarized in Chinese

Thanks for Your Attention!
• Any question or comment?
Please feel free to send e-mails to
jhwang@csie.ntut.edu.tw
or discuss with me at my office

introintrointrointrointrointrointrointro

More Related Content

Similar to introintrointrointrointrointrointrointro (20)

Recently uploaded (20)

introintrointrointrointrointrointrointro