How to control googlebot
1 like654 views
This document discusses how to control Googlebot's crawling of a website. It notes that Googlebot often does not crawl all pages of large websites due to limited crawl budgets. It recommends analyzing website logs and other metrics like pages crawled, indexed, ranked, impressions, clicks and conversions to understand Googlebot's behavior. The key factors that influence Googlebot are described as the "magic triangle" of links, content, and technical aspects. All three need attention to help Googlebot fully crawl and index a website.
How to control googlebot
- 1. Take Control on Googlebot Slideshare.Net/sergebezborodov @sergebezborodov Serge Bezborodov JetOctopus
- 2. JetOctopus Serge Bezborodov Why do you need to control Googlebot?
- 3. JetOctopus Serge Bezborodov The crawl budget problem
- 4. JetOctopus Serge Bezborodov ~40-60% of pages are uncrawled by Googlebot
- 5. JetOctopus Serge Bezborodov The indexation problem
- 6. JetOctopus Serge Bezborodov Bigger website - less % of crawled pages
- 7. JetOctopus Serge Bezborodov Can Googlebot crawl the whole website?
- 8. JetOctopus Serge Bezborodov “it depends”
- 10. JetOctopus Serge Bezborodov But you need to strive it
- 11. JetOctopus Serge Bezborodov How to control/influence on Googlebot’s behavior?
- 12. JetOctopus Serge Bezborodov Do we need some gray/black hat SEO techniques?
- 14. JetOctopus Serge Bezborodov Magic triangle Links Content Technical
- 15. JetOctopus Serge Bezborodov Which side of the triangle is the most important? Good content + bad interlinking? The best interlinking + scraped content? The best interlinking + best content + poor performance?
- 16. JetOctopus Serge Bezborodov Which side of the triangle is the most important? All of them.
- 17. JetOctopus Serge Bezborodov technical + content + interlinking
- 18. JetOctopus Serge Bezborodov Your magic triangle is unique. It's yours. Only. links content technical links content technical
- 19. JetOctopus Serge Bezborodov What to analyse? (not How?)
- 21. JetOctopus Serge Bezborodov Logs - “I need website’s access logs” - is not techie - it is secure - no user's data needed, only access logs - all possible ways of integration - it is visualised
- 23. JetOctopus Serge Bezborodov Introducing Deep Impact
- 24. JetOctopus Serge Bezborodov Introducing Deep Impact
- 25. JetOctopus Serge Bezborodov Logs is not enough
- 28. JetOctopus Serge Bezborodov pages on website crawled by Googlebot indexed ranked get impressions get clicks conversions
- 29. JetOctopus Serge Bezborodov Thank you @sergebezborodov