how does google works?
Download as PPT, PDF1 like552 views
The document describes the processes of crawling and indexing the web, highlighting the role of web crawlers like Googlebot in discovering and collecting data from webpages. It explains how Google organizes this vast amount of information into an index, enabling efficient search results based on algorithms that evaluate over 200 unique signals. Additionally, the document covers policies regarding spam detection, user privacy, and the importance of preventing harmful content while maintaining a commitment to free expression.
how does google works?
- 2. Crawling & IndexingCrawling & Indexing The journey of a query starts before you ever type a search, with crawling and indexing the web of trillions of documents.
- 3. How Search Works These processes lay the foundation — they're how we gather and organize information on the web so we can return the most useful results to you. Our index is well over 100,000,000 gigabytes, and we’ve spent over one million computing hours to build it. Learn more about the basics in this short video.
- 4. Finding information byFinding information by crawlingcrawling We use software known as “web crawlers” to discover publicly available WebPages. The most well-known crawler is called “Googlebot.” Crawlers look at WebPages and follow links on those pages, much like you would if you were browsing content on the web. They go from link to link and bring data about those WebPages back to Google’s servers. The crawl process begins with a list of web addresses from past crawls and sitemaps provided by website owners. As our crawlers visit these websites, they look for links for other pages to visit.
- 5. The software pays special attention to new sites, changes to existing sites and dead links. Computer programs determine which sites to crawl, how often, and how many pages to fetch from each site. Google doesn't accept payment to crawl a site more frequently for our web search results. We care more about having the best possible results because in the long run that’s what’s best for users and, therefore, our business.
- 6. Organizing information by indexingOrganizing information by indexing The web is like an ever-growing public library with billions of books and no central filing system. Google essentially gathers the pages during the crawl process and then creates an index, so we know exactly how to look things up. Much like the index in the back of a book, the Google index includes information about words and their locations. When you search, at the most basic level, our algorithms look up your search terms in the index to find the appropriate pages.
- 7. The search process gets much more complex from there. When you search for “dogs” you don’t want a page with the word “dogs” on it hundreds of times. You probably want pictures, videos or a list of breeds. Google’s indexing systems note many different aspects of pages, such as when they were published, whether they contain pictures and videos, and much more. With the Knowledge Graph, we’re continuing to go beyond keyword matching to better understand the people, places and things you care about.
- 8. Choice for website owners Most websites don’t need to set up restrictions for crawling, indexing or serving, so their pages are eligible to appear in search results without having to do any extra work. That said, site owners have many choices about how Google crawls and indexes their sites through Webmaster Tools and a file called “robots.txt”. With the robots.txt file, site owners can choose not to be crawled by Googlebot, or they can provide more specific instructions about how to process pages on their sites.
- 9. Site owners have granular choices and can choose how content is indexed on a page-by-page basis. For example, they can opt to have their pages appear without a snippet (the summary of the page shown below the title in search results) or a cached version (an alternate version stored on Google’s servers in case the live page is unavailable). Webmasters can also choose to integrate search into their own pages with Custom Search.
- 10. AlgorithmsAlgorithms You want the answer, not trillions of webpages. Algorithms are computer programs that look for clues to give you back exactly what you want.
- 11. For a typical query, there are thousands, if not millions, of WebPages with helpful information. Algorithms are the computer processes and formulas that take your questions and turn them into answers. Today Google’s algorithms rely on more than 200 unique signals or “clues” that make it possible to guess what you might really be looking for. These signals include things like the terms on websites, the freshness of content, your region and Page Rank.
- 12. Search Projects There are many components to the search process and the results page, and we’re constantly updating our technologies and systems to deliver better results. Many of these changes involve exciting new innovations, such as the Knowledge Graph or Google Instant. There are other important systems that we constantly tune and refine. This list of projects provides a glimpse into the many different aspects of search.
- 13. The Evolution of SearchThe Evolution of Search Our goal is to get you to the answer you're looking for faster, creating a nearly seamless connection between you and the knowledge you seek. If you’re looking to deepen your understanding of how search has evolved, this video highlights some important features like universal results and quick answers.
- 14. Experiments: From Idea to LaunchExperiments: From Idea to Launch A typical algorithmic change begins as an idea from one of our engineers about how to improve search. We take a data-driven approach and all proposed algorithm changes undergo extensive quality evaluation before release. Engineers typically start by running a series of experiments, tweaking small variables and getting feedback from colleagues until they are satisfied and ready to release the experiment to a larger audience.
- 15. Fighting SpamFighting Spam Every day, millions of useless spam pages are created. We fight spam through a combination of computer algorithms and manual review.
- 16. Spam sites attempt to game their way to the top of search results through techniques like repeating keywords over and over, buying links that pass Page Rank or putting invisible text on the screen. This is bad for search because relevant websites get buried, and it’s bad for legitimate website owners because their sites become harder to find. The good news is that Google's algorithms can detect the vast majority of spam and demote it automatically. For the rest, we have teams who manually review sites.
- 17. Identifying SpamIdentifying Spam Spam sites come in all shapes and sizes. Some sites are automatically- generated gibberish that no human could make sense of. Of course, we also see sites using subtler spam techniques. Check out these examples of “pure spam,” which are sites using the most aggressive spam techniques. This is a stream of live spam screenshots that we’ve manually identified and recently removed from appearing in search results.
- 18. Taking ActionTaking Action While our algorithms address the vast majority of spam, we address other spam manually to prevent it from affecting the quality of your results. This graph shows the number of domains that have been affected by a manual action over time and is broken down by the different spam types. The numbers may look large out of context, but the web is a really big place. A recent snapshot of our index showed that about 0.22% of domains had been manually marked for removal.
- 19. Manual Action by MonthManual Action by Month
- 20. Notifying Website OwnersNotifying Website Owners When we take manual action on a website, we try to alert the site's owner to help him or her address issues. We want website owners to have the information they need to get their sites in shape. That’s why, over time, we’ve invested substantial resources in webmaster communication and outreach. The following graph shows the number of spam notifications sent to site owners through Webmaster Tools.
- 21. Messages by MonthMessages by Month
- 22. Listening for FeedbackListening for Feedback Manual actions don’t last forever. Once a website owner cleans up her site to remove spammy content, she can ask us to review the site again by filing a reconsideration request. We process all of the reconsideration requests we receive and communicate along the way to let site owners know how it's going. Historically, most sites that have submitted reconsideration requests are not actually affected by any manual spam action. Often these sites are simply experiencing the natural ebb and flow of online traffic, an algorithmic change, or perhaps a technical problem preventing Google from accessing site content. This chart shows the weekly volume of reconsideration requests since 2006.
- 23. Reconsideration Requests by WeekReconsideration Requests by Week
- 24. PoliciesPolicies We care deeply about the information you find on Google. We strive for a consistent approach that puts users first. We want to organize the world’s information. But what about malware? What about credit card numbers? There are many tricky issues we think about on a daily basis. Here you’ll find a list of policies organized around particular topic areas. We’re starting with policies related primarily to content removals, but this is a living document and we plan to update over time. We’d love to get your feedback and suggestions.
- 25. Access to Information Comes FirstAccess to Information Comes First We believe in free expression and the free flow of Information. We try hard to make information available except for narrowly defined cases like spam, malware, legal requirements and preventing identity theft. Algorithms Over Manual ActionAlgorithms Over Manual Action The relevance and comprehensiveness of our search result is central to helping you find what you’re looking for. We prefer machine solutions to manually organizing information. Algorithms are scalable, so when we make an improvement, it makes things better not just for one search results page, but for thousands or millions. However, there are certain cases where we utilize manual controls when machine solutions aren’t enough. Learn more about Algorithms.
- 26. Exceptions ListsExceptions Lists Like most search engines, in some cases, our algorithms falsely identify sites and we make limited exceptions to improve our search quality. For example, our Safe Search algorithms are designed to protect children from adult content online. When one of these algorithms misidentifies websites (for example essex.edu) we sometimes make manual exceptions to prevent these sites from being classified as pornography. Fighting Spam and MalwareFighting Spam and Malware We hate spam as much as you do. It hurts users by cluttering search results with irrelevant links. We have teams that work to detect spammy websites and remove them from our results. The same goes for phishing websites and malware. Learn more about Fighting Spam.
- 27. Preventing Identity TheftPreventing Identity Theft Upon request, we’ll remove personal information from search results if we believe it could make you susceptible to specific harm, such as identity theft or financial fraud. We generally don’t process removals of national ID numbers from official government websites because in those cases we consider the information to be public. We sometimes refuse requests if we believe someone is attempting to abuse these policies to remove other information from our results. Fighting Child ExploitationFighting Child Exploitation We block search results that lead to child sexual abuse imagery. This is a legal requirement and the right thing to do.
- 28. Shocking ContentShocking Content We want to make sure information is available when you’re looking for it, but we also want to be careful not to show potentially upsetting content when you haven’t asked for it. Accordingly, we may not trigger certain search features for queries where the results could be offensive in several narrowly defined categories. Safe SearchSafe Search When it comes to information on the web, we leave it up to you to decide what’s worth finding. That’s why we have a Safe Search filter, which gives you more control over your search experience by helping you avoid adult content if you’d rather not
- 29. MADE BY: Hritik AgarwalMADE BY: Hritik Agarwal CLASS: 10CLASS: 10thth GUIDED BY: Sir Arvindar SinghGUIDED BY: Sir Arvindar Singh SUBMTTED TO: Sir Arvindar SinghSUBMTTED TO: Sir Arvindar Singh