The Ultimate Google Indexing Session
2 likes1,941 views
The document discusses the indexing challenges faced by websites in the DACH region, highlighting the varying index coverage percentages among major domains. It compares Google and Bing's indexing strategies, indicating that Bing is generally more willing to index content than Google, who is more selective. The document emphasizes the importance of understanding and diagnosing indexing issues to improve website visibility and traffic.
The Ultimate Google Indexing Session
- 1. The Ultimate Google Indexing Session Bartosz Góralewicz @bart_goralewicz www.onely.com
- 2. DACH – The State of Indexing
- 3. Real indexing issues - examples Shop-apotheke.ch – 31% unindexed pages
- 4. Real indexing issues - examples Billa.at– 95% indexed, but... Kartenshop.billa.at – 60% not indexed v
- 5. Real indexing issues - examples Home24.de 12% of product pages aren't indexed
- 6. www.onely.com Index coverage of top DACH domains 0% 50% 100%
- 7. ikea.com 0.98853 shop-apotheke.at 0.903356 medpex.at 0.803773 xxxlutz.de 0.865271 medikamente-per-klick.de 0.843102 zurrose-shop.ch 0.58929 bonprix.ch 0.570237 alternate.de 0.85022 shop-apotheke.com 0.873758 notino.at 0.921927 medpex.de 0.851589 medpex.de 0.851589 mediamarkt.ch 0.888155 bauundhobby.ch 0.846009 fust.ch 0.840243 peek-cloppenburg.at 0.704799 nike.com 0.945401 bauhaus.at 0.828357 microspot.ch 0.70636 ottoversand.at 0.763981 globus.ch 0.823372 universal.at 0.711728 migros.ch 0.866375 www.onely.com Google Index Coverage of top DACH domains (by revenue)
- 8. How does DACH compare to the world? VS
- 9. https://www.worldwidewebsize.com/ www.onely.com How does DACH compare to the world? 0% 50% 100% DACH: 78% World: 74%
- 11. Is there something bigger at play?
- 12. 206,956,723 Total numer of websites in 2010 www.onely.com How the web has grown 1,935,170,822 Total numer of websites in 2022
- 13. www.onely.com www.onely.com JavaScript is getting heavier, too Source: https://httparchive.org/reports/s tate-of-javascript
- 14. www.onely.com 161 requests 1.8 MB transferred 5.9 MB resources Total 74/165 requests 1.5 / 1.8 MB transferred 5.0 / 5.9 MB resources JS Source: https://www.mediamarkt.de/de/ product/_apple-airpods-3rd- generation-in-ear-kopfhorer- bluetooth-weiss-2771285.html
- 15. www.onely.com 1 episode is around 20 min 20 minutes is 88mb (roughly) 6 mb per page of MediaMarkt.de 88mb(episode) divided by 6 mb = ~15 pages from mediamarkt.de* *Cache disabled MediaMarkt’s page weight measured in "Friends" episodes
- 16. www.onely.com The web is overwhelmed Content flood New technologies 1. 2.
- 17. This creates a bottleneck that cannot be solved with current technology
- 19. www.onely.com The challenge – "What should we index?" How do Bing and Google approach solving this problem?
- 21. Bing Google www.onely.com SEO SEO Indexing … Indexing …
- 22. Quiz Time!
- 23. Who is more likely to index your content? Bing Google
- 24. t www.onely.com stripe.com walmart.com trulia.com etsy.com bbc.com uber.com wayfair.com theguardian.com 96,43% 97,05% 93,15% 80,12% 99,27% 98,85% 96,56% 98,59% 47,10% 65,14% 61,74% 54,78% 89,58% 92,78% 93,05% 98,27% 49,33 31,91 31,40 25,34 9,69 6,06 3,51 0,32 Domain % of pages indexed by Bing % of pages indexed by Google Difference between pages indexed by Bing and Google (pp)
- 25. www.onely.com Two search engines, two different strategies attachment image author post category product 0% 20% 40% 60% 80% 100% Index coverage 0-100% Sitemap category Google Bing Source: one.ly/google-vs-bing
- 26. www.onely.com Google’s index might be shrinking* * Needs further research Source: www.worldwidewebsize.com
- 27. www.onely.com Bing vs Google – summary Bing is more willing to index pages than Google 1 Google crawls more than Bing and is pickier about indexing 2 Both need to pick what to index as the Web grows too heavy 3
- 28. Bing – open and transparent – “We struggle with indexing just like everyone else, we need ~4 years to solve this problem” www.onely.com Google "From a technical point of view, we can’t index everything on the web, so we have to draw the line somewhere. It’s completely normal for websites to have parts of their content indexed and parts of their content not indexed.„ John Muller, SEO OfficeHours, January 28th, 2022 Bing vs Google PR approach
- 29. When search engines are overwhelmed, “Create great content” is not enough. The door to the web is getting smaller.
- 30. Where is my data from?
- 31. How to check if you fit through the door?
- 32. The metrics we are using are not fit for purpose anymore.
- 33. www.onely.com To win, you need to make Google’s job easy. You Google Google You
- 35. www.onely.com The SEO industry still works as if indexing wasn’t our key problem. Backlinks Visibility PageRank Clusters .GOV links CTR Correlation studies Indexing vs. indexation Getting indexed
- 36. Do you remember those 3 domains we started with?
- 37. Shop-apotheke.ch www.onely.com URLs checked ~14k of indexable URLs – not indexed 40% Pages that aren't indexed have less content than other pages on that domain Examples: • https://www.shop-apotheke.ch/de/anti-blasen/ • https://www.shop- apotheke.ch/de/artikel/10813767/speick- thermal-sensitiv-tagescreme.htm
- 38. Billa.at www.onely.com URLs checked 1500 are indexed, but... 95% Kartenshop.billa.at - 56% not indexed Examples: • https://kartenshop.billa.at/unterhaltung/spot ify.html • https://kartenshop.billa.at/unterhaltung/flixb us.html Why? Responsetime almost2s slower than restof domain
- 39. Home24.de www.onely.com URLs checked 9k not indexed 12% Pages that aren't indexed typically have less schema types (less content sections) than other pages on that domain Examples: https://www.home24.de/produkt/eckhalter ung-fuer-aussenleuchten-appleton- polyethylen-grau (8 schema types) VS https://www.home24.de/produkt/aktensch rank-vandol-mini (24 schema types)
- 40. Key sources of indexing problems www.onely.com Quality Website’s architecture Technology
- 41. How do you know WHAT is stopping your website from getting indexed?
- 42. Did you get through the door?
- 43. Key indexing challenge - how to diagnose the issue? www.onely.com • GSC data is not helpful • Crawls don't show indexing
- 44. www.onely.com How to check your indexing in 3 minutes? pages indexed 1000 (from GSC/Bing Webmaster Tools) Indexable pages published 1500 (from e.g. Ryte crawl) = 66% ÷
- 46. Once you have the data all we need is to look after a pattern.
- 47. www.onely.com Recap TL;DR PopularDACH domainsare often poorly indexed Search engines won’t fix it anytime soon Fixing indexing issues is easy, once you know your “stick” Check your indexing % Compare indexed vs. not indexed Find your “stick” Enjoy the traffic and $
- 49. HOMEWORK Bing vs. Google: Which Search Engine Indexes More Content? An SEO Guide to Google Search Console’sIndex Coverage Report Indexing SEO 101: How To Create an Indexing Strategy for Your Website The Ultimate Guide to Indexing SEO www.onely.com