Split Testing for SEO - 9 Months of Learning

Editor's Notes

• #3: I’m sure you read it on the programme
• #5: We made our jobs harder and made ourselves easier to fire. So how did we talk ourselves out of it?
• #6: So how did we talk ourselves out of it?
• #7: So how did we talk ourselves out of it?
• #9: This isn’t the same as when you’re kid. When your a kid it’s the imagination that gets you
• #12: You know what's scary:
• #14: That doesnt happen as a kid, When a teacher asks, what’s going to happen as they pour vinegar in a volcano and you have no idea. But it’s amazing and you learn One of my favourite days in primary school -- we’re in primary school -- we’re making dioramas for our project on hawaii (st lucia?), someones down the houses, the beach, the trees. -- our teacher she comes out with this big paper mache volcano, -- holds up a bottle of vinegar and asks us whats about to happen, -- we’ve not seen it we have no idea. David asks if she’ll drink it. -- pours half a bottle of vinegar into the top and it explodes with lava -- it was so cool -- all the unknown - we had no idea how it worked, it was like magic -- what made that happen? None of us had any idea but we wanted to know
• #15: As an adult standing in a board room when someone hands you a new set of numbers asks you a question you don’t know, it’s terrifying
• #16: -- turn into triple slide -- sitting looking at report - someone is asking, what effect will this seo change have? -- now not knowing doesn’t have that same sense of mystery or joy -- i have no idea what change it will have -- we’re not been able to measure this -- but we’re paid to come in and do this. I have to know.i can’t admit it will be incredibly difficult to measure these changes, so i make a estimation with ctr and rank, of course i’m totally wrong
• #18: Boy am i jealous of PPC everytime there is a quarterly meeting and they say,Last quarter we said we’d add buy more stuff into all our titles.  you gave us £16,000 we gave you 800 leads last quater. This quarter we gave you 900 leads.
• #25: Sadly that’s how for most of us, things change as we get older. It gets harder and harder to ask questions and admit we don’t know things. Particularly when we’re all meant to be experts on it. And this is where we struggle in business. Here’s the problem -- seo is hard to measure -- but i can’t measure my changes against that, i have to control for seasonality and google updates, so we talk about ranks -- even then i’m hand wavy. What if I rank for new keywords? Russ talks about search console data. But businesses speak the language of traffic and revenue, they need numbers -- we sit as consultants and seo managers as the experts -- people ask us about this black box -- they want those numbers and we can’t get them accurately. But we can’t admit that, we’re not kids we’re experts and so begins bad numbers and hand waving. SEO has always had these problems, but some things have changed: -- it’s got harder, the system has got harder to game and more complicated. It’s no longer the case The solution -- leave our egos at the door and accept what has been accepted in CRO & growth hacking that, the role is not pick and choose and never measure -- we have to make an evaluation of the maths of case studies -- we have to learn from rigourous tests -- we have to test things ourselves
• #29: What have we learnt from tests -- structured data is a good thing -- we’ve seen uplifts consistently from adding structured data to schema -- render experiment from icanvas -- block changes to title tags have been unsuccessful -- removing seo text from category pages has helped and hindered -- removing ads has not had any consistent effects -- title change to exact match keyword was negative -- schema for upwork -- entities for next web -- combining pages for realtruck
• #30: What have we learnt from tests -- structured data is a good thing -- we’ve seen uplifts consistently from adding structured data to schema -- render experiment from icanvas -- block changes to title tags have been unsuccessful -- removing seo text from category pages has helped and hindered -- removing ads has not had any consistent effects -- title change to exact match keyword was negative -- schema for upwork -- entities for next web -- combining pages for realtruck
• #31: What have we learnt from tests -- structured data is a good thing -- we’ve seen uplifts consistently from adding structured data to schema -- render experiment from icanvas -- block changes to title tags have been unsuccessful -- removing seo text from category pages has helped and hindered -- removing ads has not had any consistent effects -- title change to exact match keyword was negative -- schema for upwork -- entities for next web -- combining pages for realtruck
• #34: Boy am i jealous of PPC everytime there is a quarterly meeting and they say,Last quarter we said we’d add buy more stuff into all our titles.  you gave us £16,000 we gave you 800 leads last quater. This quarter we gave you 900 leads.
• #37: We can’t just cut and paste other people’s ideas & results anymore
• #38: but people want to know what our changes will do
• #39: so we have to estimate the effect of our changes beforehand
• #40: and measure it after the fact.
• #41: But we’re meant to be experts people expect us to know
• #42: and we want that money
• #44: Measuring SEO is hard
• #45: Which means we use all these data sources, we use ranks and we calculate CTR estimates to estimate traffic.
• #46: But this is all smoke, mirrors and hand waving,
• #47: but it does mean we get our money in the short term
• #48: But it also means our estamates are more likely to be wrong and make no sense
• #49: Nobody cares when everything is going well
• #50: But as soon as things run into trouble everyone begins to question your estimates and measuring and point out they’re awful
• #51: Once you’ve been burned by this a couple times
• #52: You might give it a try with just ranks & clicks
• #53: But then you’re unconvincing
• #54: And you don’t get the money, because the other departments make better justifications so nothing happens
• #57: So how did we talk ourselves out of it?
• #63: Split your pages into 2 buckets, make changes to one of them and measure the difference in traffic from google
• #64: Split your pages into 2 buckets, make changes to one of them and measure the difference in traffic from google
• #74: -- turn into triple slide -- sitting looking at report - someone is asking, what effect will this seo change have? -- now not knowing doesn’t have that same sense of mystery or joy -- i have no idea what change it will have -- we’re not been able to measure this -- but we’re paid to come in and do this. I have to know.i can’t admit it will be incredibly difficult to measure these changes, so i make a estimation with ctr and rank, of course i’m totally wrong
• #75: -- turn into triple slide -- sitting looking at report - someone is asking, what effect will this seo change have? -- now not knowing doesn’t have that same sense of mystery or joy -- i have no idea what change it will have -- we’re not been able to measure this -- but we’re paid to come in and do this. I have to know.i can’t admit it will be incredibly difficult to measure these changes, so i make a estimation with ctr and rank, of course i’m totally wrong
• #78: -- turn into triple slide -- sitting looking at report - someone is asking, what effect will this seo change have? -- now not knowing doesn’t have that same sense of mystery or joy -- i have no idea what change it will have -- we’re not been able to measure this -- but we’re paid to come in and do this. I have to know.i can’t admit it will be incredibly difficult to measure these changes, so i make a estimation with ctr and rank, of course i’m totally wrong
• #82: -- turn into triple slide -- sitting looking at report - someone is asking, what effect will this seo change have? -- now not knowing doesn’t have that same sense of mystery or joy -- i have no idea what change it will have -- we’re not been able to measure this -- but we’re paid to come in and do this. I have to know.i can’t admit it will be incredibly difficult to measure these changes, so i make a estimation with ctr and rank, of course i’m totally wrong
• #86: -- turn into triple slide -- sitting looking at report - someone is asking, what effect will this seo change have? -- now not knowing doesn’t have that same sense of mystery or joy -- i have no idea what change it will have -- we’re not been able to measure this -- but we’re paid to come in and do this. I have to know.i can’t admit it will be incredibly difficult to measure these changes, so i make a estimation with ctr and rank, of course i’m totally wrong
• #89: -- turn into triple slide -- sitting looking at report - someone is asking, what effect will this seo change have? -- now not knowing doesn’t have that same sense of mystery or joy -- i have no idea what change it will have -- we’re not been able to measure this -- but we’re paid to come in and do this. I have to know.i can’t admit it will be incredibly difficult to measure these changes, so i make a estimation with ctr and rank, of course i’m totally wrong
• #94: -- turn into triple slide -- sitting looking at report - someone is asking, what effect will this seo change have? -- now not knowing doesn’t have that same sense of mystery or joy -- i have no idea what change it will have -- we’re not been able to measure this -- but we’re paid to come in and do this. I have to know.i can’t admit it will be incredibly difficult to measure these changes, so i make a estimation with ctr and rank, of course i’m totally wrong
• #95: -- turn into triple slide -- sitting looking at report - someone is asking, what effect will this seo change have? -- now not knowing doesn’t have that same sense of mystery or joy -- i have no idea what change it will have -- we’re not been able to measure this -- but we’re paid to come in and do this. I have to know.i can’t admit it will be incredibly difficult to measure these changes, so i make a estimation with ctr and rank, of course i’m totally wrong
• #96: -- turn into triple slide -- sitting looking at report - someone is asking, what effect will this seo change have? -- now not knowing doesn’t have that same sense of mystery or joy -- i have no idea what change it will have -- we’re not been able to measure this -- but we’re paid to come in and do this. I have to know.i can’t admit it will be incredibly difficult to measure these changes, so i make a estimation with ctr and rank, of course i’m totally wrong
• #97: -- turn into triple slide -- sitting looking at report - someone is asking, what effect will this seo change have? -- now not knowing doesn’t have that same sense of mystery or joy -- i have no idea what change it will have -- we’re not been able to measure this -- but we’re paid to come in and do this. I have to know.i can’t admit it will be incredibly difficult to measure these changes, so i make a estimation with ctr and rank, of course i’m totally wrong
• #98: -- turn into triple slide -- sitting looking at report - someone is asking, what effect will this seo change have? -- now not knowing doesn’t have that same sense of mystery or joy -- i have no idea what change it will have -- we’re not been able to measure this -- but we’re paid to come in and do this. I have to know.i can’t admit it will be incredibly difficult to measure these changes, so i make a estimation with ctr and rank, of course i’m totally wrong
• #99: -- turn into triple slide -- sitting looking at report - someone is asking, what effect will this seo change have? -- now not knowing doesn’t have that same sense of mystery or joy -- i have no idea what change it will have -- we’re not been able to measure this -- but we’re paid to come in and do this. I have to know.i can’t admit it will be incredibly difficult to measure these changes, so i make a estimation with ctr and rank, of course i’m totally wrong
• #100: -- turn into triple slide -- sitting looking at report - someone is asking, what effect will this seo change have? -- now not knowing doesn’t have that same sense of mystery or joy -- i have no idea what change it will have -- we’re not been able to measure this -- but we’re paid to come in and do this. I have to know.i can’t admit it will be incredibly difficult to measure these changes, so i make a estimation with ctr and rank, of course i’m totally wrong
• #101: -- turn into triple slide -- sitting looking at report - someone is asking, what effect will this seo change have? -- now not knowing doesn’t have that same sense of mystery or joy -- i have no idea what change it will have -- we’re not been able to measure this -- but we’re paid to come in and do this. I have to know.i can’t admit it will be incredibly difficult to measure these changes, so i make a estimation with ctr and rank, of course i’m totally wrong