How a year of SEO split testing changed how I thought SEO worked
- 1. HOW A YEAR OF SEO SPLIT TESTING CHANGED HOW I THOUGHT SEO WORKED DOMINIC WOODMAN
- 4. Hmm that did nothing Crushing failure Ooh traffic went up
- 6. All those links I’m building.
- 8. Hmm that did nothing Crushing failure Ooh traffic went up
- 14. How does SEO testing work & what do you need?
- 15. Getting started & learning how to think about testing
- 16. More than you wanted to know about titles and metas.
- 18. Structured data & dangerous assumptions
- 21. Conclusions
- 22. How does SEO testing work & what do you need?
- 23. SEO split testing is template basedHere’s our example website Home Animals Cats Dogs Unicorns Badgers
- 24. CatsCats We want to test two templates for our animals pages
- 25. Cats Dogs Unicorns Badgers CRO test - Different users, see different templates on the same pages 1 2 Cats BadgersDogs Unicorns
- 26. Cats Dogs Unicorns Badgers SEO test - different users see the same templates on different pages 1 2 Cats Dogs Unicorns Badgers Cats Dogs Unicorns Badgers
- 27. Then we measure the change in traffic, e.g. 3% daily increase
- 29. LIABILITY
- 38. The fans show margin of error
- 39. The test is significant when the shaded area crosses 0
- 40. In this case it happened here:
- 47. Control Variant
- 48. Black is organic traffic for the variant
- 49. Start date for the test
- 50. Take all the data to left and use that to make the model
- 51. The model of the variant is the blue line
- 52. We overlay them
- 53. Compare the model to the variant after the start date
- 54. In this case the black line is higher, it’s a positive test
- 55. A good model, fits with the variant before the start date
- 56. However this was my model. They don’t fit well.
- 57. How much traffic do you need for SEO split testing? Rule of thumb: you need roughly 1000 organic sessions a day to the group of pages you’re testing. #INBOUND2018 @dom_woodman
- 58. That model you just saw, only had ~60 sessions a day.
- 59. Getting started & learning how to think about testing
- 60. 1
- 64. Some fluctuation but null.
- 66. 2
- 68. After
- 69. 10.8% decrease
- 71. 3
- 73. <img src="product1.jpg" alt="Home 6 Piece Towel Bale - Geo Grey / Zest">
- 74. A model so perfectly null I thought I had forgotten to launch it
- 76. 4&5
- 77. Thought leader. PHD in AI. Unbearably smug.
- 79. Before:
- 80. After:
- 81. A stunning 27% drop.
- 83. A 20% drop.
- 84. Can’t write titles to save his life.
- 85. Get a testing framework You need to be able to iterate and learn from tests. Having a shared framework, will help with this. #INBOUND2018 @dom_woodman
- 86. You don’t need the why Knowing the why is great (find it if you can), but you don’t need it to take action. #INBOUND2018 @dom_woodman
- 87. More than you wanted to know about titles and metas.
- 88. 6
- 89. 16
- 90. 1. Reviews 2. Safety Information
- 91. Before
- 92. After
- 93. 10% drop
- 94. 7
- 96. Before
- 97. After
- 98. 14.7% drop
- 99. Fast forward… more tests happen.
- 100. Title tests usually change traffic between ~5 - 15% This stayed equivalent across site size and industry. #INBOUND2018 @dom_woodman
- 101. 56% of all title tag changes were negative Around 37% were null and only 6% were actually positive. It’s really hard to write a good title. #INBOUND2018 @dom_woodman
- 102. Writing titles is just really really hard.
- 103. You can’t stop testing titles. Titles exist in the context of the rest of a SERP, when everyone copies your good title format (and they will) you’ll have to mix it up again. #INBOUND2018 @dom_woodman
- 104. Title/meta effects typically appear in 2-4 days. It’s also easy to validate the changes have been picked up because you can scrape it from the SERPS. #INBOUND2018 @dom_woodman
- 106. 8
- 107. We can render Javascript!
- 110. 6.2% increase.
- 111. 5% increase
- 112. Visible content on the initial page load matters Although there is a lot subtlety to how Google renders JS, we haven’t covered here. #INBOUND2018 @dom_woodman
- 113. 9
- 116. Increase of 3.1%
- 117. SEO content sometimes helps… :( Damn it. #INBOUND2018 @dom_woodman
- 118. The same changes have different effects on different sites. This is the big one. There really isn’t best practice. #INBOUND2018 @dom_woodman
- 119. (Assuming it was topically similar). We’ve saved clients a lot of money, by showing they could reduce content without an effect. Reducing content on non-article pages was often null. #INBOUND2018 @dom_woodman
- 120. Structured data & dangerous assumptions
- 121. 10
- 124. Adding structured data to category pages
- 125. Increase of 11%
- 126. 11
- 127. Increase of 11%
- 128. $$$$$$ - 150K additional sessions a month. How did your last year go?
- 130. 12, 13, 14
- 132. Null.
- 133. Null.
- 134. Null.
- 135. The same changes have different effects on different sites. This is the big one. There really isn’t best practice. #INBOUND2018 @dom_woodman
- 136. The same changes have different effects on different sites. This is the big one. There really isn’t best practice.
- 137. Structured data has an effect outside of rich snippets Occasionally we got big 10-15% wins on important templates, it varied wildly and was mostly null (never negative). #INBOUND2018 @dom_woodman
- 138. Periodically re-challenge your beliefs I was lucky to test a successful site first. In a different order, I might’ve given up on my hypothesis, before finding the right site. #INBOUND2018 @dom_woodman
- 139. 15
- 141. Increase of 16%
- 142. 16
- 144. 5 star much value very marketer
- 146. 0.0% change. Again impressively null.
- 147. If you don’t have intent, bells and whistles fail. 5 star rich snippets increased traffic by 16% on the right site with the right intent. When the intent wasn’t there, they appeared, but did nothing. #INBOUND2018 @dom_woodman
- 148. Risky business...
- 150. 17
- 153. <time itemprop="dateModified" datetime="{{date}}"> Updated on: {{date}} </time>
- 154. 8% increase
- 155. Freshness does matter. But I wouldn’t recommend faking last modified dates... #INBOUND2018 @dom_woodman
- 156. Agile testing can help you take larger risks. If you can measure and quickly roll out/roll back a test, you can try things you might not normally feel comfortable doing. #INBOUND2018 @dom_woodman
- 158. 18
- 163. Increase of 12%
- 164. Increase of 12%
- 165. Null. Go figure.
- 166. You don’t need to argue, when you can test. When you have solid testing framework and can build things quickly and easily, testing is easier than arguing and removes arguments from a relationship. #INBOUND2018 @dom_woodman
- 169. Also null.
- 171. Happiness isn’t just up and to the right Instead the focus turns to other metrics like how many tests are we running, how can we improve the testing process. #INBOUND2018 @dom_woodman
- 172. A negative test you rolled back is a bullet dodged Negative tests feel like wasted time. Once you realise without testing you probably would’ve rolled out, it feels a lot better. #INBOUND2018 @dom_woodman
- 173. You’re about to be forced out of silos. You’re going to need to coordinate and work tightly with product and QA teams if you’re not already. #INBOUND2018 @dom_woodman
- 174. Conclusions
- 175. Hmm that did nothing Crushing failure Ooh traffic went up
- 176. The same changes have different effects on different sites. Really can’t emphasize this one enough. #INBOUND2018 @dom_woodman
- 177. Get a testing framework. Most tests fail or are null. Having a framework will help you move faster and find those wins. #INBOUND2018 @dom_woodman
- 178. Testing will improve your relationships. You’ll have to spend less time arguing and it creates a culture of curiosity. #INBOUND2018 @dom_woodman
- 179. Periodically re-challenge your beliefs You probably have some beliefs about what works or doesn’t work which are wrong from blind chance. Re-test them. #INBOUND2018 @dom_woodman
- 180. Making changes to sections of pages on templates ● Making SEO changes with tag manager ● Cloud flare edge workers ● DistilledODN General useful posts on testing frameworks & velocity ● Hypothesis framework ● Running a weekly growth meeting Do it yourself How does split testing work? ● How does SEO split testing work? Examples ● Pinterest - Demystifying SEO with experiments ● Etsy - SEO title tag testing Measuring SEO split tests ● Google’s original causal impact paper ● A DIY tool for measuring SEO split tests ● A walkthrough of the R Causal Impact library
- 181. bit.ly/inbound-odn