Scalability andefficiencypres
- 1. Scalability and Efficiency Challenges in Large-Scale Web Search Engines Ricardo Baeza-Yates B. Barla Cambazoglu Yahoo Labs Barcelona, Spain Tutorial in SIGIR’13, WWW’14, SIGIR’14, MUMIA’14 (based on the book chapter in The Information Retrieval Series, 2011)
- 2. Disclaimer • This talk presents the opinions of the authors. It does not necessarily reflect the views of Yahoo Inc. or any other entity. • Algorithms, techniques, features, etc. mentioned here might or might not be in use by Yahoo or any other company. • Parts of the content (e.g., images, plots) used in this presentation were taken from public web pages.
- 3. Yahoo Labs Barcelona • Groups – data mining – semantic search – social media – web retrieval • Web retrieval group – distributed web retrieval – scalability and efficiency – opinion/sentiment analysis – personalization
- 4. Outline of the Tutorial • Background • Main sections – web crawling – indexing – query processing – caching • Concluding remarks • Open discussion
- 5. Structure of the Main Sections • Definitions • Metrics • Issues and techniques – single computer – cluster of computers – multiple search sites • Research problems • Q&A
- 6. Background
- 7. Brief History of Search Engines • Past - Before browsers - Gopher - Before the bubble - Altavista - Lycos - Infoseek - Excite - HotBot - After the bubble - Yahoo - Google - Microsoft • Current • Global • Google, Bing • Regional • Yandex, Baidu • Future • Facebook ? • …
- 8. Anatomy of a Search Engine Result Page • Main web search results • Direct displays (vertical search results) – image – video – local – shopping – related entities • Query suggestions • Advertisement Focus of this tutorial
- 9. Anatomy of a Search Engine Result Page Related entity suggestions Movie direct display Video direct display AdsAlgorithmic search results Knowledge graph User query
- 10. Actors in Web Search • User’s perspective: accessing information – relevance – speed • Search engine’s perspective: monetization – attract more users – increase the ad revenue – reduce the operational costs • Advertiser’s perspective: publicity – attract more users – pay little
- 12. • Size What Makes Web Search Difficult? • Diversity • Dynamicity • All of these three features can be observed in – the Web – web users
- 13. What Makes Web Search Difficult? • The Web – more than 180 million web servers and 950 million host names – almost 1 billion computers directly connect to the Internet – the largest data repository (estimated as 100 billion pages) – constantly changing – diverse in terms of content and data formats • Users – too many! (over 2.5 billion at the end of 2012) – diverse in terms of their culture, education, and demographics – very short queries (hard to understand the intent) – changing information needs – little patience (few queries posed & few answers seen)
- 14. • Crawl and index a large fraction of the Web • Maintain most recent copies of the content in the Web • Scale to serve hundreds of millions of queries every day • Evaluate a typical query under several hundred milliseconds • Serve most relevant results for a user query Expectations from a Search Engine
- 15. Search Data Centers • Quality and performance requirements imply large amounts of compute resources, i.e., very large data centers • High variation in data center sizes - hundreds of thousands of computers - a few computers
- 16. Cost of Data Centers • Data center facilities are heavy consumers of energy, accounting for between 1.1% and 1.5% of the world’s total energy use in 2010.
- 17. Financial Costs • Costs – depreciation: old hardware needs to be replaced – maintenance: failures need to be handled – operational: energy spending needs to be reduced
- 18. Impact of Resources on Revenue
- 19. Major Components in a Web Search Engine • Indexing • Query processing• Web crawling crawler Web indexer query processor document collection index query results user
- 20. Q&A
- 21. Web Crawling
- 22. Web Crawling • Web crawling is the process of locating, fetching, and storing the pages available in the Web • Computer programs that perform this task are referred to as - crawlers - spider - harvesters
- 23. Web Graph • Web crawlers exploit the hyperlink structure of the Web
- 24. Web Crawling Process • Initialize a URL download queue with some seed URLs • Repeat the following steps – fetch the content of a URL selected from the download queue – store the fetched content in a repository – extract the hyperlinks within the fetched content – add the extracted links into the download queue
- 26. Web Crawling • Crawling divides the Web into three sets – downloaded – discovered – undiscovered seed page discovered (frontier) undiscovered downloaded
- 27. URL Prioritization • A state-of-the-art web crawler maintains two separate queues for prioritizing the download of URLs – discovery queue – downloads pages pointed by already discovered links – tries to increase the coverage of the crawler – refreshing queue – re-downloads already downloaded pages – tries to increase the freshness of the repository
- 28. URL Prioritization (Discovery) • Random (A, B, C, D) • Breadth-first (A) • In-degree (C) • PageRank (B) A B C D seed page discovered (frontier) undiscovered downloaded (more intense red color indicates higher PageRank)
- 29. URL Prioritization (Refreshing) • Random (A, B, C, D) • PageRank (B) • Age (C) • User feedback (D) • Longevity (A) download time order (by the crawler) A BC D last update time order (by the webmaster) A DCB (more intense red color indicates higher PageRank) estimated average update frequency never every minute every day every year (more intense blue color indicates larger user interest)
- 30. Metrics • Quality metrics – coverage: percentage of the Web discovered or downloaded by the crawler – freshness: measure of staleness of the local copy of a page relative to the page’s original copy on the Web – page importance: percentage of important (e.g., popular) pages in the repository • Performance metrics – throughput: content download rate in bytes per unit of time
- 31. Concepts Related to Web Crawling
- 32. Crawling Architectures • Single computer – CPU, RAM, and disk becomes a bottleneck – not scalable • Parallel – multiple computers, single data center – scalable • Geographically distributed – multiple computers, multiple data centers – scalable – reduces the network latency
- 33. An Architectural Classification of ConceptsCrawling Architectures
- 34. Issues in Web Crawling • Dynamics of the Web – web growth – content change • Malicious intent – hostile sites (e.g., spider traps, infinite domain name generators) – spam sites (e.g., link farms)
- 35. Issues in Web Crawling • URL normalization (a.k.a. canonicalization) – case-folding – removing leading “www strings (e.g., www.cnn.com cnn.com) – adding trailing slashes (e.g., cnn.com/a cnn.com/a/) – relative paths (e.g., ../index.html) • Web site properties – sites with restricted content (e.g., robot exclusion), – unstable sites (e.g., variable host performance, unreliable networks) – politeness requirements
- 36. DNS Caching • Before a web page is crawled, the host name needs to be resolved to an IP address • Since the same host name appears many times, DNS entries are locally cached by the crawler Web server Crawler DNS server web page repository DNS cache DNS caching TCP connection HTTP connection
- 37. Multi-threaded Crawling • Multi-threaded crawling – crawling is a network-bound task – crawlers employ multiple threads to crawl different web pages simultaneously, increasing their throughput significantly – in practice, a single node can run around up to a hundred crawling threads – multi-threading becomes infeasible when the number of threads is very large due to the overhead of context switching
- 38. Politeness • Multi-threading leads to politeness issues • If not well-coordinated, the crawler may issue too many download requests at the same time, overloading – a web server – an entire sub-network • A polite crawler – puts a delay between two consecutive downloads from the same server (a commonly used delay is 20 seconds) – closes the established TCP-IP connection after the web page is downloaded from the server
- 39. Robot Exclusion Protocol • A standard from the early days of the Web • A file (called robots.txt) in a web site advising web crawlers about which parts of the site are accessible • Crawlers often cache robots.txt files for efficiency purposes User-agent: googlebot # all services Disallow: /private/ # disallow this directory User-agent: googlebot-news # only the news service Disallow: / # on everything User-agent: * # all robots Disallow: /something/ # on this directory User-agent: * # all robots Crawl-delay: 10 # wait at least 10 seconds Disallow: /directory1/ # disallow this directory Allow: /directory1/myfile.html # allow a subdirectory Host: www.example.com # use this mirror
- 40. Mirror Sites • A mirror site is a replica of an existing site, used to reduce the network traffic or improve the availability of the original site • Mirror sites lead to redundant crawling and, in turn, reduced discovery rate and coverage for the crawler • Mirror sites can be detected by analyzing – URL similarity – link structure – content similarity
- 41. Data Structures • Good implementation of data structures is crucial for the efficiency of a web crawler • The most critical data structure is the “seen URL” table – stores all URLs discovered so far and continuously grows as new URLs are discovered – consulted before each URL is added to the discovery queue – has high space requirements (mostly stored on the disk) – URLs are stored as MD5 hashes – frequent/recent URLs are cached in memory
- 43. Web Partitioning and Fault Tolerance • Web partitioning – typically based on the MD5 hashes of URLs or host names – site-based partitioning is preferable because URL-based partitioning may lead to politeness issues if the crawling decisions given by individual nodes are not coordinated • Fault tolerance – when a crawling node dies, its URLs are partitioned over the remaining nodes
- 44. Parallelization Alternatives • Firewall mode – lower coverage • Crossover mode – duplicate pages • Exchange mode – communication overhead D D X D D D D X Duplicate crawling in crossover mode Not discovered in firewall mode Link communicated in exchange mode
- 45. Geographically Distributed Web Crawling
- 46. Geographically Distributed Web Crawling • Benefits - higher crawling throughput - geographical proximity - lower crawling latency - improved network politeness - less overhead on routers because of fewer hops - resilience to network partitions - better coverage - increased availability - continuity of business - better coupling with distributed indexing/search - reduced data migration
- 47. Geographically Distributed Web Crawling • Four crawling countries - US - Brazil - Spain - Turkey • Eight target countries - US, Canada - Brazil, Chile - Spain, Portugal - Turkey, Greece
- 48. Focused Web Crawling • The goal is to locate and download a large portion of web pages that match a given target theme as early as possible. • Example themes - topic (nuclear energy) - media type (forums) - demographics (kids) • Strategies - URL patterns - referring page content - local graph structure starting page
- 49. Sentiment Focused Web Crawling • Goal: to locate and download web pages that contain positive or negative sentiments (opinionated content) as early as possible starting page
- 50. Hidden Web Crawling • Hidden Web: web pages that a crawler cannot access by simply following the link structure • Examples - unlinked pages - private sites - contextual pages - scripted content - dynamic content hidden web crawler repositoryvocabulary root page result page
- 51. Research Problem – Passive Discovery • URL discovery by external agents – toolbar logs – email messages – tweets • Benefits – improved coverage – early discovery head fetcher parser tail repository frontier crawler the Web toolbar data processor toolbar log URL filter http://... toolbar http://... toolbar
- 52. Research Problem – URL Scheduling • Web master – create – modify – delete • Crawler – discover – download – refresh • How to optimally allocate available crawling resources between the tasks of page discovery and refreshing Inexistant Modified Deleted F Located Undiscovered Fresh Deleted R Undiscovered Web Repository modification deletion f download & purge re-download & purge deletion m re-download m creation Inexistence Frontier deletion l discovery deletion u download re-download f passive transitions active transitions discovery
- 53. Challenges in Distributed Web CrawlingResearch Problem – Web Partitioning • Web partitioning/repartitioning: the problem of finding a Web partition that minimizes the costs in distributed Web crawling – minimization objectives – page download times – link exchange overhead – repartitioning overhead – constraints – coupling with distributed search – load balancing
- 54. Challenges in Distributed Web CrawlingResearch Problem – Crawler Placement • Crawler placement problem: the problem of finding the optimum geographical placement for a given number of data centers – geographical locations are now objectives, not constraints • Problem variant: assuming some data centers were already built, find an optimum location to build a new data center for crawling
- 55. Challenges in Distributed Web CrawlingResearch Problem – Coupling with Search • Coupling with geographically distributed indexing/search – crawled data may be moved to – a single data center – replicated on multiple data centers – partitioned among a number of data centers – decisions must be given on – what data to move (e.g., pages or index) – how to move (i.e., compression) – how often to move (i.e., synchronization)
- 56. Research Problem – Green Web Crawling • Goal: reduce the carbon footprint generated by the web servers while handling the requests of web crawlers • Idea - crawl web sites when they are consuming green energy (e.g., during the day when solar energy is more available) - crawl web sites consuming green energy more often as an incentive to promote the use of green energy
- 57. Published Web Crawler Architectures • Bingbot: Microsoft's Bing web crawler • FAST Crawler: Used by Fast Search & Transfer • Googlebot: Web crawler of Google • PolyBot: A distributed web crawler • RBSE: The first published web crawler • WebFountain: A distributed web crawler • WebRACE: A crawling and caching module • Yahoo Slurp: Web crawler used by Yahoo Search
- 58. Open Source Web Crawlers • DataparkSearch: GNU General Public License. • GRUB: open source distributed crawler of Wikia Search • Heritrix: Internet Archive's crawler • ICDL Crawler: cross-platform web crawler • Norconex HTTP Collector: licensed under GPL • Nutch: Apache License • Open Search Server: GPL license • PHP-Crawler: BSD license • Scrapy: BSD license • Seeks: Affero general public license
- 59. Key Papers • Cho, Garcia-Molina, and Page, "Efficient crawling through URL ordering", WWW, 1998. • Heydon and Najork, "Mercator: a scalable, extensible web crawler", World Wide Web, 1999. • Chakrabarti, van den Berg, and Dom, "Focused crawling: a new approach to topic-specific web resource discovery", Computer Networks, 1999. • Najork and Wiener, "Breadth-first crawling yields high-quality pages", WWW, 2001. • Cho and Garcia-Molina, "Parallel crawlers", WWW, 2002. • Cho and Garcia-Molina, "Effective page refresh policies for web crawlers”, ACM Transactions on Database Systems, 2003. • Lee, Leonard, Wang, and Loguinov, "IRLbot: Scaling to 6 billion pages and beyond", ACM TWEB, 2009.
- 60. Q&A
- 61. Indexing
- 62. Indexing • Indexing is the process of converting crawled web documents into an efficiently searchable form. • An index is a representation for the document collection over which user queries will be evaluated.
- 63. Indexing • Abandoned indexing techniques – suffix arrays – signature files • Currently used indexing technique – inverted index
- 64. Signature File • For a given piece of text, a signature is created by encoding the words in it • For each word, a bit signature is computed – contains F bits – m out of F bits is set to 1 (decided by hash functions) • In case of long documents – a signature is created for each logical text block – block signatures are concatenated to form the signature for the entire document
- 65. Signature File • When searching, the signature for the query keyword is OR’ed with the document signature • Example signature with F = 6 and m = 2 document terms: query terms: apple 10 00 10 apple 10 00 10 (match) orange 00 01 10 banana 01 00 01 (no match) peach 10 01 00 (false match) signature 10 01 10
- 66. Inverted Index • An inverted index has two parts – inverted lists – posting entries – document id – term score – a vocabulary index (dictionary)
- 67. Sample Document Collection Doc id Text 1 pease porridge hot 2 pease porridge cold 3 pease porridge in the pot 4 pease porridge hot, pease porridge not cold 5 pease porridge cold, pease porridge not hot 6 pease porridge hot in the pot
- 68. Inverted Index (2, 1) (4, 1) (5, 1) (1, 1) (4, 1) (5, 1) (6, 1) (3, 1) (6, 1) cold hot in (4, 1) (5, 1) (1, 1) (2, 1) (3, 1) (4, 2) (5, 2) (6, 1) (1, 1) (2, 1) (3, 1) (4, 2) (5, 2) (6, 1) not pease porridge (3, 1) (6, 1) (3, 1) (6, 1) pot the Dictionary Inverted lists
- 69. Inverted Index • Additional data structures - position lists: list of all positions a term occurs in a document - document array: document length, PageRank, spam score, … • Sections - title, body, header, anchor text (inbound, outbound links) (1, 1) (2, 1) (3, 1) (4, 2) (5, 2) (6, 1) 1 1 1 1 4 pease 11 4 7 240 1.7 ... 6 doc id: 4 term count length spam score doc freq
- 70. Metrics • Quality metrics - spam rate: fraction of spam pages in the index - duplicate rate: fraction of exact or near duplicate web pages present in the index • Performance metrics - compactness: size of the index in bytes - deployment cost: time and effort it takes to create and deploy a new inverted index from scratch - update cost: time and space overhead of updating a document entry in the index
- 71. Indexing Documents • Index terms are extracted from documents after some processing, which may involve - tokenization - stopword removal - case conversion - stemming original text: Living in America applying all: liv america in practice: living in america
- 72. Duplicate Detection • Detecting documents with duplicate content - exact duplicates (solution: computing/comparing hash values) - near duplicates (solution: shingles instead of hash values)
- 73. Features • Offline computed features - content: spam score, domain quality score - web graph: PageRank, HostRank - usage: click count, CTR, dwell time • Online computed features - query-document similarity: tf-idf, BM25 - term proximity features
- 74. PageRank • A link analysis algorithm that assigns a weight to each web page indicating its importance • Iterative process that converges to a unique solution • Weight of a page is proportional to - number of inlinks of the page - weight of linking pages • Other algorithms - HITS - SimRank - TrustRank
- 75. Spam Filtering • Content spam - web pages with potentially many popular search terms and with little or no useful information - goal is to match many search queries to the page content and increase the traffic coming from search engines
- 76. Spam Filtering • Link spam - a group of web sites that all link to every other site in the group - goal is to boost the PageRank score of pages and make them more frequently visible in web search results
- 77. Spam Filtering Top 10 PageRank sites • PageRank is subject to manipulation by link farms
- 78. Inverted List Compression • Benefits - reduced space consumption - reduced transfer costs - increased posting list cache hit rate • Gap encoding - original: 17 18 28 40 44 47 56 58 - gap encoded: 17 1 10 12 4 3 9 2 gaps
- 79. Inverted List Compression • Techniques - Unary encoding - Gamma encoding - Delta encoding - Variable byte encoding - Golomb encoding - Rice encoding - PforDelta encoding - Interpolative encoding gap: 1000 unary: 11111111…11111110 (999 ones) gamma: 1111111110:111101000 delta: 1110:010:111101000 vbe: 00000111 11101000
- 80. Document Identifier Reordering • Goal: reassign document identifiers so that we obtain many small d-gaps, facilitating compression • Example old lists: L1: 1 3 6 8 9 L2: 2 4 5 6 9 L3: 3 6 7 9 mapping: 11 29 32 47 58 63 75 86 94 new lists: L1: 1 2 3 4 6 L2: 3 4 7 8 9 L3: 2 3 4 5 old d-gaps: 2 3 2 1 2 1 1 3 3 1 2 new d-gaps: 1 1 1 2 1 3 1 1 1 1 1
- 81. Document Identifier Reordering • Techniques - sorting URLs alphabetically and assigning ids in that order - idea: pages from the same site have high textual overlap - simple yet effective - only applicable to web page collections - clustering similar documents - assigns nearby ids to documents in the same cluster - traversal of document similarity graph - formulated as the traveling salesman problem
- 82. Index Construction • Equivalent to computing the transpose of a matrix • In-memory techniques do not work well with web-scale data • Techniques - two-phase - first phase: read the collection and allocate a skeleton for the index - second phase: fill the posting lists - one-phase - keep reading documents and building an in-memory index - each time the memory is full, flush the index to the disk - merge all on-disk indexes into a single index in a final step
- 83. Index Maintenance • Grow a new (delta) index in the memory; each time the memory is full, flush the in-memory index to disk - no merge - flushed index is written to disk as a separate index - increases fragmentation and query processing time - eventually requires merging all on-disk indexes or rebuilding - incremental indexing - each inverted list contains additional empty space at the end - new documents are appended to the empty space in the list - merging delta index - immediate merge: maintains only one copy of the index on disk - selective merge: maintains multiple generations of the index on disk
- 84. Inverted Index Partitioning/Replication • In practice, the inverted index is – partitioned on thousands of computers in a large search cluster – reduces query response times – allows scaling with increasing collection size – replicated on tens of search clusters – increases query processing throughput – allows scaling with increasing query volume – provides fault tolerance
- 85. Inverted Index Partitioning • Two alternatives for partitioning an inverted index – term-based partitioning – T inverted lists are distributed across P processors – each processor is responsible for processing the postings of a mutually disjoint subset of inverted lists assigned to itself – single disk access per query term – document-based partitioning – N documents are distributed across P processors – each processor is responsible for processing the postings of a mutually disjoint subset of documents assigned to itself – multiple (parallel) disk accesses per query term
- 86. Term-Based Index Partitioning (2, 1) (4, 1) (5, 1) (1, 1) (4, 1) (5, 1) (6, 1) (3, 1) (6, 1) cold hot in (4, 1) (5, 1) (1, 1) (2, 1) (3, 1) (4, 2) (5, 2) (6, 1) (1, 1) (2, 1) (3, 1) (4, 2) (5, 2) (6, 1) not pease porridge (3, 1) (6, 1) (3, 1) (6, 1) pot the P1 P2 P3
- 87. Document-Based Index Partitioning (2, 1) (4, 1) (5, 1) (1, 1) (4, 1) (5, 1) (6, 1) (3, 1) (6, 1) cold hot in (4, 1) (5, 1) (1, 1) (2, 1) (3, 1) (4, 2) (5, 2) (6, 1) (1, 1) (2, 1) (3, 1) (4, 2) (5, 2) (6, 1) not pease porridge (3, 1) (6, 1) (3, 1) (6, 1) pot the P1 P2 P3
- 88. Comparison of Index Partitioning Approaches Document-based Term-based Space consumption Higher Lower Number of disk accesses Higher Lower Concurrency Lower Higher Computational load imbalance Lower Higher Max. posting list I/O time Lower Higher Cost of index building Lower Higher Maintenance cost Lower Higher
- 89. Inverted Index Partitioning • In practice, document-based partitioning is used – simpler to build and update – low inter-query-processing concurrency, but good load balance – low throughput, but high response time – high throughput is achieved by replication – easier to maintain – more fault tolerant • Hybrid techniques are possible (e.g., term partitioning inside a document sub-collection)
- 90. Parallel Index Creation • Possible alternatives for creating an inverted index in parallel - message passing paradigm - doc-based: each node builds a local index using its documents - term-based: posting lists are communicated between the nodes - MapReduce framework Mapper Reducer D1: A B C D2: E B D D3: B C Mapper Mapper Reducer A: D1 B: D1 C: D1 E: D2 B: D2 D: D2 B: D3 C: D3 A: D1 B: D1 D2 D3 C: D1 D3 D: D2 E: D2
- 91. Static Index Pruning • Idea: to create a small version of the search index that can accurately answer most search queries • Techniques - term-based pruning - doc-based pruning • Result quality - guaranteed - not guaranteed • In practice, caching does the same better main index pruned index 1 2
- 92. Tiered Index • A sequence of sub-indexes - former sub-indexes are small and keep more important documents - later sub-indexes are larger and keep less important documents - a query is processed selectively only on the first n tiers • Two decisions need to be made - tiering (offline): how to place documents in different tiers - fall-through (online): at which tier to stop processing the query 3rd tier index 1st tier index 1 2 2nd tier index 3
- 93. 3rd tier index 1st tier index 1 2 2nd tier index 3 Tiered Index • The tiering strategy is based on some document importance metric - PageRank - click count - spam score • Fall-through strategy - query the next index until there are enough results - query the next index until search result quality is good - predict the next tier’s result quality by machine learning
- 94. cluster 1 cluster 3 1 2 cluster 2 2 federator Document Clustering • Documents are clustered - similarity between documents - co-click likelihood • A separate index is built for each document cluster
- 95. cluster 1 cluster 3 1 2 cluster 2 2 federator Document Clustering • A query is processed on the indexes associated with the most similar n clusters • Reduces the workload • Suffers from the load imbalance problem - query topic distribution may be skewed - certain indexes have to be queried much more often
- 96. 100 Scaling Up 2 clusters Caching Adapted from Moffat and Zobel, 2004. Scaling Up 4 clusters 1 cluster 2 clusters
- 97. Multi-site Architectures • Replicated • Partitioned• Centralized
- 98. Research Problem – Indexing Architectures • Replicated • Pruned • Clustered • Tiered • Missing research - a thorough performance comparison of these architectures that includes all possible optimizations - scalability analysis with web-scale document collections and large numbers of nodes
- 99. Research Problem – Off-network Popularity • Traditional features - statistical analysis - link analysis - proximity - spam - clicks - session analysis • Off-network popularity - social signals - network ranker content popularity database search engine result page the Web routers users
- 100. Key Papers • Brin and Page, "The anatomy of a large-scale hypertextual Web search engine", Computer Networks and ISDN Systems, 1998. • Zobel, Moffat, and Ramamohanarao, "Inverted files versus signature files for text indexing". ACM Transactions on Database Systems, 1998. • Page, Brin, Motwani, and Winograd, "The PageRank citation ranking: bringing order to the Web", Technical report, 1998. • Kleinberg, "Authoritative sources in a hyperlinked environment", Journal of the ACM, 1999. • Ribeiro-Neto, Moura, Neubert, and Ziviani, "Efficient distributed algorithms to build inverted files", SIGIR, 1999. • Carmel, Cohen, Fagin, Farchi, Herscovici, Maarek, and Soffer, "Static index pruning for information retrieval systems, SIGIR, 2001. • Scholer, Williams, Yiannis, and Zobel, "Compression of inverted indexes for fast query evaluation", SIGIR, 2002.
- 101. Q&A
- 102. Query Processing
- 103. Query Processing • Query processing is the problem of generating the best- matching answers (typically, top 10 documents) to a given user query, spending the least amount of time • Our focus: creating 10 blue links as an answer to a user query
- 104. Web Search • Web search is a sorting problem! daylight saving example query giant squid barla cambazoglu test sun energy miracle good Indian restaurant Honda CRX Yahoo research my horoscope SIGIR conference deadline test drive download mp3 honey facebook user queries the Web good Indian restaurant f (good Indian restaurant)
- 105. Metrics • Quality metrics - relevance: the degree to which returned answers meet user’s information need. • Performance metrics - latency: the response time delay experienced by the user - peak throughput: number of queries that can be processed per unit of time without any degradation on other metrics
- 106. 1. 2. 3. 4. R Ranking 1 1. 2. 3. 4. R Ranking 2 Optimal 1. 2. 3. 4. R R R R R R Recall: Precision: DCG: NDCG: 1/3 1/3 1 1/4 1/4 3/4 1 0.63 1+0.63+0.5=2.13 1/2.13 0.63/2.13 Measuring Relevance • It is not always possible to know the user’s intent and his information need • Commonly used relevance metrics in practice - recall - precision - DCG - NDCG
- 107. Estimating Relevance • How to estimate the relevance between a given document and a user query? • Alternative models for score computation - vector-space model - statistical models - language models • They all pretty much boil down to the same thing
- 108. Example Scoring Function • Notation - : a user query - : a document in the collection - : a term in the query - : number of documents in the collection - : number of occurrences of the term in the document - : number of documents containing the term - : number of unique terms in the document • Commonly used scoring function: tf-idf s(q,d) = w(t,d) tÎq å ´log N df (t) w(t,d) = tf (t,d) | d | | d | df (t) tf (t,d) N t d q
- 109. Score Computation (using an accumulator array) 2 3 5 7 8 10 5 6 8 11 L1: L2: 1 2 3 4 5 6 7 8 9 10 11 12 0.2 0.5 0.1 0.4 0.3 0.1 0.6 0.1 0.3 0.2 0 0.2 0.5 0 0.7 0.1 0.4 0.6 0 0.1 0.2 0 5 8 3 7 11 2 6 10 1 4 9 12 0.7 0.6 0.5 0.4 0.2 0.2 0.1 0.1 0 0 0 0
- 110. Efficient Score Computation (using a min heap) 5 8 3 0.7 0.6 0.5 2 3 5 7 8 10 5 6 8 11 L1: L2:0.2 0.5 0.1 0.4 0.3 0.1 0.6 0.1 0.3 0.2 0.2 : 2 0.2 : 2 0.5 : 3 0.2 : 2 0.5 : 3 0.7 : 5 3 5 7 2 0.4 : 7 0.5 : 3 0.7 : 5 0.5 : 3 0.6 : 8 0.7 : 5 0.5 : 3 0.6 : 8 0.7 : 5 8 10, 11
- 111. Design Alternatives in Ranking • Document matching - conjunctive (AND) mode - the document must contain all query terms - higher precision, lower recall - disjunctive (OR) mode - document must contain at least one of the query terms - lower precision, higher recall
- 112. Design Alternatives in Ranking • Inverted list organizations - increasing order of document ids - decreasing order of weights in postings - sorted by term frequency - sorted by score contribution (impact) - within the same impact block, sorted in increasing order of document ids 2 3 5 7 8 10 doc id ordered 0.2 0.5 0.1 0.4 0.3 0.1 3 7 8 2 5 10 weight ordered 0.5 0.4 0.3 0.2 0.1 0.1
- 113. Design Alternatives in Ranking • Traversal order - term-at-a-time (TAAT) - document-at-a-time (DAAT) 2 3 5 7 8 10 0.2 0.5 0.1 0.4 0.3 0.1 5 6 8 11 0.6 0.1 0.3 0.2 L1: L2: 4 7 9 10 0.5 0.3 0.1 0.4L3: 2 3 5 7 8 10 0.2 0.5 0.1 0.4 0.3 0.1 5 6 8 11 0.6 0.1 0.3 0.2 L1: L2: 4 7 9 10 0.5 0.3 0.1 0.4L3: Term-at-a-time Document-at-a-time
- 114. Design Alternatives in Ranking • Weights stored in postings - term frequency - suitable for compression - normalized term frequency - no need to store the document length array - not suitable for compression - precomputed score - no need to store the idf value in the vocabulary - no need to store the document length array - not suitable for compression
- 115. Design Alternatives in Ranking • In practice - AND mode: faster and leads to better results in web search - doc-id sorted lists: enables compression - document-at-a-time list traversal: enables better optimizations - term frequencies: enables compression
- 116. Scoring Optimizations • Skipping - list is split into blocks linked with pointers called skips - store the maximum document id in each block - skip a block if it is guaranteed that sought document is not within the block - gains in decompression time - overhead of skip pointers 2 3 5 7 8 10 0.2 0.5 0.1 0.4 0.1 0.15 10 24 ...
- 117. Scoring Optimizations • Dynamic index pruning - store the maximum possible score contribution of each list - compute the maximum possible score for the current document - compare with the lowest score in the heap - gains in scoring and decompression time 2 3 5 7 8 10 0.2 0.5 0.1 0.4 0.1 0.1 3 7 8 11 0.3 0.1 0.2 0.1 L1: L2: 3 7 9 10 0.1 0.2 0.1 0.2L3: 0.5 0.3 0.2 Current top 2: 0.9 0.7 3 7
- 118. Scoring Optimizations • Early termination - stop scoring documents when it is guaranteed that neither document can make it into the top k list - gains in scoring and decompression time 3 7 2 5 8 10 0.5 0.4 0.2 0.1 0.1 0.1 3 8 7 11 0.3 0.2 0.1 0.1 L1: L2: 3 7 9 10 0.2 0.2 0.1 0.1L3: Current heap: 1.0 0.6 3 7 0.2 8
- 119. Snippet Generation • Search result snippets (a.k.a., summary or abstract) - important for users to correctly judge the relevance of a web page to their information need before clicking on its link • Snippet generation - a forward index is built providing a mapping between pages and the terms they contain - snippets are computed using this forward index and only for the top 10 result pages - efficiency of this step is important - entire page as well as snippets can be cached
- 120. • Query processing can be parallelized at different granularities - parallelization within a search node (intra-query parallelism) - multi-threading within a search node (inter-query parallelism) - parallelization within a search cluster (intra-query parallelism) - replication across search clusters (inter-query parallelism) - distributed across multiple data centers Parallel Query Processing
- 121. • Single computer - not scalable in terms of response time • Search cluster - large search clusters (low response time) - replicas of clusters (high query throughput) • Multiple search data centers - reduces user-to-center latencies Query Processing Architectures
- 122. Query Processing in a Data Center Broker rewriters cache blenderFrontend Search cluster Master Child Child Child Child Child user query results Search cluster Search cluster Search cluster • Multiple node types - frontend, broker, master, child
- 123. Parallel Query Processing (central broker)
- 124. Parallel Query Processing (pipelined)
- 125. Query Processing on a Search Node • Two-phase ranking - simple ranking - linear combination of query-dependent and query-independent scores potentially with score boosting - main objective: efficiency - complex ranking - machine learned - main objective: quality
- 126. Machine Learned Ranking • Many features – term statistics (e.g., BM25) – term proximity – link analysis (e.g., PageRank) – spam detection – click data – search session analysis • Popular learners used by commercial search engines – neural networks – boosted decision trees
- 127. Machine Learned Ranking • Example: gradient boosted decision trees – chain of weak learners – each learner contributes a partial score to the final document score • Assuming – 1000 trees – an average tree depth of 10 – 100 documents scored per query – 1000 search nodes • Expensive – 1000*10*100 = 1 million operations per query and per node – around 1 billion comparison for the entire search cluster
- 128. Machine Learned Ranking • Document-ordered traversal (DOT) – scores are computed one document at a time over all scorers – an iteration of the outer loop produces the complete score information for a partial set of documents • Disadvantages – poor branch prediction because a different scorer is used in each inner loop iteration – poor cache hit rates in accessing the data about scorers (for the same reason)
- 129. Machine Learned Ranking • Scorer-ordered traversal (SOT) – scores are computed one score at a time over all documents – an iteration of the outer loop produces the partial score information for the complete set of documents • Disadvantages – memory requirement (feature vectors of all documents need to be kept in memory) – poor cache hit rates in accessing features as a different document is used in each inner loop iteration
- 130. Machine Learned Ranking • Early termination – idea: place predictive functions between scorers – predict during scoring whether a document will enter the final top k and – quit scoring, accordingly
- 131. Multi-site Web Search Architecture Key points - multiple, regional data centers (sites) - user-to-center assignment - local web crawling - partitioned web index - partial document replication - query processing with selective forwarding
- 132. Multi-site Distributed Query Processing • Local query response time – 2 user-to-site latency – local processing time • Forwarded query response time – local query response time – 2 site-to-site latency – non-local query processing time
- 133. Query Forwarding • Problem – selecting a subset of non- local sites which the query will be forwarded to • Objectives – reducing the loss in search quality w.r.t. to evaluation over the full index – reducing average query response times and/or query processing workload
- 134. Machine-Learned Query Forwarding • Two-phase forwarding decision – pre-retrieval classification – query features – user features – post-retrieval classification – result features • The post-retrieval classifier is consulted only if the confidence of the pre-retrieval classifier is not high enough • This enables overlapping local query processing with remote processing of the query
- 135. • Constituents of response latency - user-to-center network latency - search engine latency time - query features - caching - current system workload - page rendering time in the browser Web Search Response Latency User Search frontend Search backend tpre tproc tpost tfb tbf tuf tfu trender
- 136. • Problem: Predict the response time of a query before processing it on the backend search system • Useful for making query scheduling decisions • Solution: Build a machine learning model with many features - number of query terms - total number of postings in inverted lists - average term popularity - … Response Latency Prediction
- 137. • Impact of latency on the user varies depending on - context: time, location - user: demographics - query: intent Impact of Response Latency on Users
- 138. • A/B test by Google and Microsoft (Schurman & Brutlag) Impact of Response Latency on Users dailysearchesperuserrelativetocontrol wk3 wk4 wk5 wk6 wk7 wk8 wk9 wk10 wk11 -1%-0.8%-0.6%-0.4%-0.2%0%0.2% delay removed Persistent Impact of Post-header Delay 200 ms delay 400 ms delay actual trend
- 139. Research Problem – Energy Savings • Observation: electricity prices vary across data centers and depending the time of the day • Idea: forward queries to cheaper search data centers to reduce the electricity bill under certain constraints
- 140. Research Problem – Green Search Engines • Goal: reduce the carbon footprint of the search engine • Query processing - shift workload from data centers that consume brown energy to those green energy - constraints: - response latency - data center capacity
- 141. Open Source Search Engines • DataparkSearch: GNU general public license • Lemur Toolkit & Indri Search Engine: BSD license • Lucene: Apache software license • mnoGoSearch: GNU general public license • Nutch: based on Lucene • Seeks: Affero general public license • Sphinx: free software/open source • Terrier Search Engine: open source • Zettair: open source
- 142. Key Papers • Turtle and Flood, "Query evaluation: strategies and optimizations", Information Processing and Management, 1995. • Barroso, Dean, and Holzle, "Web search for a planet: the Google cluster architecture", IEEE Micro, 2003. • Broder, Carmel, Herscovici, Soffer, and Zien, "Efficient query evaluation using a two-level retrieval process", CIKM, 2003. • Chowdhury and Pass, “Operational requirements for scalable search systems”, CIKM, 2003. • Moffat, Webber, Zobel, and Baeza-Yates, "A pipelined architecture for distributed text query evaluation", Information Retrieval, 2007.
- 143. Key Papers • Turpin, Tsegay, Hawking, and Williams, "Fast generation of result snippets in web search. SIGIR, 2007. • Baeza-Yates, Gionis, Junqueira, Plachouras, and Telloli, "On the feasibility of multi-site web search engines", CIKM, 2009. • Cambazoglu, Zaragoza, Chapelle, Chen, Liao, Zheng, and Degenhardt, "Early exit optimizations for additive machine learned ranking systems", WSDM, 2010. • Wang, Lin, and Metzler, "Learning to efficiently rank", SIGIR, 2010. • Macdonald, Tonellotto, Ounis, "Learning to predict response times for online query scheduling", SIGIR, 2012.
- 144. Q&A
- 145. Caching
- 146. Caching • Cache: quick-access storage system – may store data to eliminate the need to fetch the data from a slower storage system – may store precomputed results to eliminate redundant computation in the backend Main backend Cache speed slower faster workload higher lower capacity larger smaller cost cheaper more expensive freshness more fresh more stale
- 147. Caching • Often appears in a hierarchical form – OS: registers, L1 cache, L2 cache, memory, disk – network: browser cache, web proxy, server cache, backend database • Benefits – reduces the workload – reduces the response time – reduces the financial cost 1 2 3 (fastest) (slowest)
- 148. Metrics • Quality metrics - freshness: average staleness of the data served by the cache • Performance metrics - hit rate: fraction of total requests that are answered by the cache - cost: total processing cost incurred to the backend system
- 149. Caching • Items preferred for caching – more popular over time – more recently requested
- 150. Query Frequency • Skewed distribution in query frequency – Few queries are issued many times (head queries) – Many queries are issued rarely (tail queries)
- 151. Inter-Arrival Time • Skewed distribution in query inter-arrival time – low inter-arrival time is for many queries – high inter-arrival time for few queries
- 152. Posting list cache Intersection cache Posting lists Index server Document cache Documents Document server Result cache Score cache Broker Caches Available in a Web Search Engine • Main caches – result – score – intersection – posting list – document
- 153. Caching Techniques • Static caching – built in an offline manner – prefers items that are accessed often in the past – periodically re-deployed • Dynamic caching – maintained in an online manner – prefers items that are recently accessed – requires removing items from the cache (eviction) • Static/dynamic caching – shares the cache space between a static and a dynamic cache
- 154. Static/Dynamic Caching A C A D A D C A B C C E A static cache was built now static cache (capacity: 2) D F G dynamic cache (capacity: 3) F A B D A G F most frequent most recent A C D B E F G A D B
- 155. Techniques Used in Caching • Admission: items are prevented from being cached • Prefetching: items are cached before they are requested • Eviction: items are removed from the cache when it is full
- 156. Admission • Idea: certain items may be prevented from being cached forever or until confidence is gained about their popularity • Example admission criteria – query length – query frequency XYZ Q ABC Result cache Minimum frequency threshold for admission: 2 Maximum query length for admission: 4 Query stream: ABC IJKLMN ABC ABC XYZ Q XYZ Query cache
- 157. Prefetching • Idea: certain items can be cached before they are actually requested if there is enough confidence that they will be requested in the near future. • Example use case: result page prefetching Reqested: page 1 Prefetch: page 2 and page 3
- 158. Eviction • Goal: remove items that are not likely to lead to a cache hit from the cache to make space for items that are more useful. • Ideal policy: evict the item that will be requested in the most distant future. • Policies: FIFO, LFU, LRU, SLRU • Most common: LRU A D D A A A D C C A D D D C A B B D C E Aevicted: A
- 159. Result Cache Freshness • In web search engines – index is continuously updated or re-built – result caches are almost infinite capacity – staleness problem
- 160. Flushing • Naïve solution: flushing the cache at regular time intervals
- 161. Time-to-Live (TTL) • Common solution: setting a time-to-live value for each item • TTL strategies – fixed: all items are associated with the same TTL value – adaptive: a separate TTL value is set for each item 0 2 4 6 8 10 12 14 16 18 20 22 Hour of the day 0 10 20 30 40 50 60 Backendquerytrafficrate(query/sec) TTL = 0 (no cache) TTL = 1 hour TTL = infinite User query traffic Cache hits (expired entries) Cache misses Cache misses (compulsary) hitting the backend Lower-bound on the backend traffic
- 163. Advanced Solutions • Cache invalidation: the indexing system provides feedback to the caching system upon an index update so that the caching system can identify the stale results – hard to implement – incurs communication and computation overheads – highly accurate
- 164. Advanced Solutions • Cache refreshing: stale results are predicted and scheduled for re-computation in idle cycles of the backend search system – easy to implement – little computational overhead – not very accurate
- 165. Result Cache Refreshing Architecture
- 166. Impact on a Production System
- 167. Result Caching on Multi-site Search Engines • Strategies - Local: each search site caches results of queries issued to itself - redundant processing of the same query when issued to different sites - Global: all sites cache results of all queries - overhead of transferring cached query results to all remote sites - Partial: query results are cached on relevant sites only - trade-off between the local and global approaches - Pointer: remote sites maintain only pointers to the site that cached the query results - reduces the redundant query processing overhead
- 168. Research Problem - Financial Perspective • Past work optimizes - hit rate - backend workload • Optimizing financial cost - result degradation - staleness - current query traffic - peak sustainable traffic - current electricity price 0 2 4 6 8 10 12 14 16 18 20 22 Hour of the day 0 5 10 15 20 25 30 35 40 Backendquerytraffic(query/sec) TTL = 1 hour TTL = 4 hours TTL = 16 hours TTL = infinite Opportunity for prefetching Risk of overflow Potential for load reduction Peak query throughput sustainable by the backend
- 169. Key Papers • Markatos, “On caching search engine query results”, Computer Communications, 2001. • Long and Suel, “Three-level caching for efficient query processing in large web search engines”, WWW, 2005. • Fagni, Perego, Silvestri, and Orlando, “Boosting the performance of web search engines: caching and prefetching query results by exploiting historical usage data”, ACM Transactions on Information Systems, 2006. • Altingovde, Ozcan, and Ulusoy, “A cost-aware strategy for query result caching in web search engines”, ECIR, 2009. • Cambazoglu, Junqueira, Plachouras, Banachowski, Cui, Lim, and Bridge, “A refreshing perspective of search engine caching”, WWW, 2010.
- 170. Q&A
- 171. Concluding Remarks
- 172. Summary • Presented a high-level overview of important scalability and efficiency issues in large-scale web search • Provided a summary of commonly used metrics • Discussed a number of potential research problems in the field • Provided references to available software and the key research work in literature
- 173. Observations • Unlike past research, the current research on scalability is mainly driven by the needs of commercial search engine companies • Scalability of web search engines is likely to be a research challenge for some more time (at least, in the near future) • Lack of hardware resources and large datasets render scalability research quite difficult, especially for researchers in academia
- 174. Suggestions to Newcomers • Follow the trends in the Web, user bases, and hardware parameters to identify the real performance bottlenecks • Watch out newly emerging techniques whose primary target is to improve the search quality and think about their impact on search performance • Re-use or adapt existing solutions in more mature research fields, such as databases, computer networks, and distributed computing • Know the key people in the field (the community is small) and follow their work
- 175. Important Surveys/Books • Web search engine scalability and efficiency - B. B. Cambazoglu and Ricardo Baeza-Yates, “Scalability Challenges in Web Search Engines”, The Information Retrieval Series, 2011. • Web crawling - C. Olston and M. Najork: “Web Crawling”, Foundations and Trends in Information Retrieval, 2010. • Indexing - J. Zobel and A. Moffat, “Inverted files for text search engines”, ACM Computing Surveys, 2006. • Query processing - R. Baeza-Yates and B. Ribeiro-Neto, Modern Information Retrieval (2nd edition), 2011.
- 176. Q&A