SEO by Hypothesis
4 likes1,680 views
The document discusses an SEO expert's approach to making recommendations by forming hypotheses and testing them. It notes that many SEO recommendations have little impact and presents a framework for developing more effective recommendations through hypothesis-driven testing. This involves specifically hypothesizing how a change will impact metrics, gathering relevant data, measuring the results of split tests, and iterating based on learnings. Examples are provided of hypotheses tested around structured data, date annotations, and information architecture that led to measurable improvements. The document advocates for this approach to yield more impactful results than recommendations made without testing.
![PEOPLE SEE REAL RESULTS FROM BIG CHANGES TO IA
See Alex’s fascinating Mozcon talk [PDF]](https://image.slidesharecdn.com/hypothesis-based-seo-171019083437/85/SEO-by-Hypothesis-52-320.jpg)
SEO by Hypothesis
- 1. SEO BY HYPOTHESIS Recommend → Hypothesise & Test @TomAnthonySEO
- 2. Cooking nuts in mud Replicated it, but devil in details Google black box - can’t always replicate #dmsso@TomAnthonySEOwww.madeinindia.tv
- 3. recommendation! WE GET IDEAS FROM ALL OVER… #dmsso@TomAnthonySEO blog post worked before colleague google guidelines new idea tried for another client
- 4. FROM THERE, WE MAKE RECOMMENDATIONS “Add ‘Free Shipping’ to title tags.” #dmsso@TomAnthonySEO
- 5. DO WE UNDERSTAND THE IMPACT OF THESE CHANGES ?
- 6. WE SURVEYED 2000+ SEOS… future of tv advertising Which URL ranks higher for the search:
- 7. MORE EXPERIENCED SEOS DO BETTER… 30% 40% 50% 60% 70% Laypeople <= 3 yrs SEO > 3 yrs SEO
- 8. … BUT WE DON’T DO GREAT 30% 40% 50% 60% 70% Laypeople <= 3 yrs SEO > 3 yrs SEO Coin flip
- 9. WE CANNOT PREDICT WHAT RANKS WITH ANY ACCURACY !
- 10. WE’VE BEEN A/B TESTING… NBED - we have started testing #dmsso@TomAnthonySEO
- 11. RECOMMENDATION → TEST “Add ‘Free Shipping’ to title tags.” #dmsso@TomAnthonySEO “Let’s test ‘Free Shipping’ in title tags.”
- 16. Before After query search volume which tv should i buy 250 query search volume which tv to buy 1,300
- 17. THAT MADE THINGS WORSE #dmsso@TomAnthonySEO
- 19. <img src="product1.jpg" alt="Home 6 Piece Towel Bale - Geo Grey / Zest">
- 20. NADA… NOTHING… NO IMPACT… #dmsso@TomAnthonySEO
- 22. “Stand back guys, I got this.”
- 23. Before After
- 25. COULD BE OUR RESULTS AREN’T ACCURATE ?
- 26. NOPE, THE PLATFORM IS SOLID. WE SUCK. #dmsso@TomAnthonySEO
- 27. HARDER THAN ANTICIPATED TO PICK WINNERS #dmsso@TomAnthonySEO
- 28. CAN WE MAKE MORE EFFECTIVE RECOMMENDATIONS ?
- 29. LAST YEAR: RECOMMENDED TESTING
- 30. FOLLOW THE CRO EXPERTS - dis.tl/cro-hypothesis
- 31. MEASURE THE RESULT GATHER SOME DATA BUILD A HYPOTHESIS
- 33. IT REQUIRES BEING SPECIFIC Changing <abc> will have an impact on <xyz> & thus improve organic search performance. Based on <evidence>, we hypothesise… #dmsso@TomAnthonySEO
- 34. FOR EXAMPLE.. Changing <title formats, by…> will improve <ctr> & thus improve organic search performance. Based on <survey data>, we hypothesise… #dmsso@TomAnthonySEO
- 35. HYPOTHESIS BENEFITS Made to be more consistent. It ensures there is clarity. #dmsso@TomAnthonySEO To support it we need data.
- 36. GATHER SOME DATA
- 37. Control Influence We have the same data as Google. Google has data that we don’t.
- 38. Control Influence Keyword targeting Internal links We have the same data as Google. Google has data that we don’t.
- 39. Control Influence Website ‘quality’ We have the same data as Google. Google has data that we don’t.
- 40. Control Influence Usage Data We have the same data as Google. Google has data that we don’t.
- 41. Control Influence External Links We have the same data as Google. Google has data that we don’t.
- 42. Control Influence External Links Usage Data Keyword targeting Internal links Website ‘quality’ We have the same data as Google. Google has data that we don’t.
- 43. Control Influence External Links Usage Data Keyword targeting Internal links Website ‘quality’ Test & Model We have the same data as Google. Google has data that we don’t.
- 44. Control Influence External Links Usage Data Keyword targeting Internal links Website ‘quality’ Survey & Study We have the same data as Google. Google has data that we don’t.
- 45. Control Influence External Links Usage Data Keyword targeting Internal links Website ‘quality’ Analyse Better We have the same data as Google. Google has data that we don’t.
- 48. INFORMATION ARCHITECTURE #dmsso@TomAnthonySEO • Few sites are this simple. • Can’t measure impact of changes.
- 49. CURRENT BEST APPROACH #dmsso@TomAnthonySEO Calculate internal PageRank Use interactive visualisations
- 50. VISUALISATIONS - USEFUL, BUT RARELY Credit: Paul Shapiro
- 51. INTERNAL PAGERANK - POWERFUL, BUT QUIRKY Credit: Paul Shapiro
- 52. PEOPLE SEE REAL RESULTS FROM BIG CHANGES TO IA See Alex’s fascinating Mozcon talk [PDF]
- 53. BUT REAL-WORLD CHANGES ARE: HARD TO MAKE & HARD TO UNDO !
- 54. OUR INTUITION IS REALLY BAD. !
- 55. WE ASKED PEOPLE, GIVEN A SITE…
- 56. ADD A NEW PAGE - LINKED FROM HOMEPAGE
- 57. IT LINKS TO SAME PAGES AS HOMEPAGE
- 58. WHAT HAPPENS TO PAGERANK? PageRank Change?
- 59. WILL ASKED ON TWITTER… #dmsso@TomAnthonySEO
- 60. …3/5 PEOPLE GOT IT WRONG NOPE #dmsso@TomAnthonySEO
- 61. GIVEN THAT, HOW CAN WE HYPOTHESISE THE IMPACT? ?
- 63. CRAWL THE SITE
- 64. FETCH & NORMALISE MOZRANK Raw MozRank Normalised MozRank 3.67E-13 1.0% 3.35E-11 94.2% 1.71E-13 0.5% 1.64E-13 0.5% 1.59E-13 0.4% 3.28E-13 0.9% 6.88E-14 0.2% 2.45E-13 0.7% 7.12E-14 0.2% 3.12E-13 0.9% 1.67E-13 0.5% #dmsso@TomAnthonySEO
- 65. OVERLAY MOZRANK ON TO INTERNAL PR ALGO 8.2 5.3 4.3 4.8
- 66. WHAT HAPPENS TO PAGERANK? PageRank Change?
- 67. MODEL EXACTLY WHAT WOULD HAPPEN 8.05 5.3 4.3 4.8 4.05
- 68. FIND THE WHOLE PROCESS HERE http://dis.tl/ia-pagerank #dmsso@TomAnthonySEO
- 69. Control Influence External Links Usage Data Keyword targeting Internal links Website ‘quality’ Survey & Study We have the same data as Google. Google has data that we don’t.
- 71. USAGE DATA: WE CAN’T GUESS #dmsso@TomAnthonySEO It is impossible to guess what people prefer.
- 73. SERP TURKEY
- 74. SERP TURKEY
- 75. PANDA & QUALITY
- 76. • How do you communicate it to clients / bosses? • How do you benchmark it against competitors? • How do you figure out if proposed changes improve it? QUALITY?
- 77. Gather human rater information Google employs thousands of human quality raters to answer questionnaires about many kinds of website
- 78. Train ML models Google uses the human questionnaires as training data for ML models of “quality” Gather human rater information Google employs thousands of human quality raters to answer questionnaires about many kinds of website
- 79. 2011 Panda released The Panda quality algorithm starts being used as a batch process modifying the regular core algorithm Train ML models Google uses the human questionnaires as training data for ML models of “quality” Gather human rater information Google employs thousands of human quality raters to answer questionnaires about many kinds of website
- 80. 2011 Panda released The Panda quality algorithm starts being used as a batch process modifying the regular core algorithm Train ML models Google uses the human questionnaires as training data for ML models of “quality” Gather human rater information Google employs thousands of human quality raters to answer questionnaires about many kinds of website 2016 Panda goes real-time “Quality” becomes a first- class ranking factor in the core algorithm
- 81. QUALITY - RANKING FACTOR IS A PROXY #dmsso@TomAnthonySEO Google use Machine Learning to predict what we can just ask people. Actually measure: ML PROXY FOR QUALITY Want to measure: QUALITY
- 82. 2011: WILL SUGGESTS RUNNING PANDA SURVEYS #dmsso@TomAnthonySEO
- 83. PANDA SURVEY Client site 1 Client site 2 Would you trust information from this website? 72% 64%
- 84. PANDA SURVEY Client site 1 Client site 2 Key competitor Would you trust information from this website? 72% 64% 81%
- 85. PANDA SURVEY Client site 1 Client site 2 Key competitor Would you trust information from this website? 72% 64% 81% Is this website written by experts? 50% 46% 65%
- 86. PANDA SURVEY Client site 1 Client site 2 Key competitor Would you trust information from this website? 72% 64% 81% Is this website written by experts? 50% 46% 65% Would you give this site your credit card details? 29% 21% 43% Are there any noticeable errors on this page? 6% 4% 1% Does this page provide original content or info? 76% 72% 85% Would you recognize this site as an authority? 44% 33% 58% Does this website contain insightful analysis? 72% 62% 81% Would you consider bookmarking pages on this site? 44% 38% 56% Are there excessive adverts on this website? 2% 2% 8% Could pages from this site appear in print? 54% 54% 59%
- 88. DECK - HOW YOU CAN DO SPLIT-TESTING A GUIDE TO SPLIT-TESTING THE NEXT ERA OF SEO dis.tl/diy-ab-test
- 90. HYPOTHESIS CATEGORY PAGE HTML IS VERBOSE, AND CONTAINS A LOT OF CRUFT. ADDING A LIST OF PRODUCTS AS STRUCTURED MARKUP WILL MAKE THE INTENT CLEARER.
- 93. REPEATED IN ANOTHER REGION - 11% AGAIN #dmsso@TomAnthonySEO
- 99. THE HYPOTHESIS DID NOT FIT THESE OTHER SITES. !
- 100. FINAL EXAMPLE - FRESHNESS #dmsso@TomAnthonySEO
- 101. COMPETITORS - MORE FREQUENT DATE ANNOTATIONS #dmsso@TomAnthonySEO
- 102. GOOGLE IDENTIFIES FRESHNESS AS IMPORTANT IN THESE RESULTS REGULARLY MAKING SMALL UPDATES FOR THIS CLIENT’S CONTENT WILL HELP GOOGLE RECOGNISE FRESHNESS. HYPOTHESIS
- 103. ADDED & UPDATED DATE MODIFIED #dmsso@TomAnthonySEO <time itemprop="dateModified" datetime="{{date}}"> Updated on: {{date}} </time>
- 105. RECAP & TAKEAWAYS
- 106. THE MAJORITY OF SEO RECOMMENDATIONS HAVE LITTLE IMPACT OBSERVATION
- 107. NULL/NEGATIVE CHANGES SLOW YOU DOWN #dmsso@TomAnthonySEO
- 108. EVEN EXPERTS STRUGGLE TO PREDICT THE IMPACT OF CHANGES OBSERVATION ME INCLUDED!
- 109. PAGERANK IS ITERATIVE & NOT INTUITIVE NOPE
- 110. IN ISOLATION RANKINGS ARE HARD TO PREDICT 30% 40% 50% 60% 70% Laypeople <= 3 yrs SEO > 3 yrs SEO Coin flip
- 111. HYPOTHESIS FRAMEWORK
- 112. MORE IMPACT & QUICKER RESULTS #dmsso@TomAnthonySEO RUN TESTSBUILD A HYPOTHESISGATHER DATA
- 113. BEGIN TESTING, START HERE… distilled.net/diy-splittester
- 114. BEGIN HYPOTHESISING, START HERE… #dmsso@TomAnthonySEO http://dis.tl/pandaQ http://dis.tl/serpturkey http://dis.tl/ia-pagerank
- 116. THANK YOU! @TomAnthonySEO @WillCritchlow @Dom_Woodman
- 117. distilledodn.com