Building a Testing Playbook by Andrew Richardson
Download as PPTX, PDF2 likes819 views
The document emphasizes the importance of testing for improving conversion rates and provides guidance on tools and processes necessary for effective experimentation. It highlights common challenges organizations face, such as resistance to testing and the need for better alignment among teams. Practical tips and strategies for testing, including user experience considerations and prioritization of hypotheses, are also discussed to enhance advertising outcomes.
Building a Testing Playbook by Andrew Richardson
- 2. I CAN’T EAT LUNCH YET THIS GUY KEEPS TALKING…
- 3. This is your BRAIN An attention span of 8 seconds *https://advertise.bingads.microsoft.com/en-us/insights @analyticandrew #eMetrics
- 5. IS MY CONVERSION RATE GOOD OR BAD? WHY DO I NEED TO TEST? BUT NO ONE WANTS TO DO TESTING AT MY ORGANIZATION. PLATFORM, PEOPLE, PROCESS WHAT DO I TEST AND HOW DO I TRACK IT? TIPS AND SUGGESTIONS FOR GETTING STARTED TAKEAWAYS @analyticandrew #eMetrics
- 6. IS MY CONVERSION RATE GOOD OR BAD? WHY DO I NEED TO TEST? BUT NO ONE WANTS TO DO TESTING AT MY ORGANIZATION. PLATFORM, PEOPLE, PROCESS WHAT DO I TEST AND HOW DO I TRACK IT? TIPS AND SUGGESTIONS FOR GETTING STARTED TAKEAWAYS @analyticandrew #eMetrics
- 8. 2.6% 2.8% 2.8% 3.2% 3.3% 3.4% 5.0% 5.0% 5.5% 6.0% Higher Education Real Estate Health Legal Home Improvement Business Services Travel Business Consulting Credit & Lending Vocational Studies & Job Training *Unbounce Conversion Benchmark Report – March 2017 HOW DO YOU COMPARE?
- 9. Never stop testing, and your advertising will never stop improving. - David Ogilvy -
- 10. IS MY CONVERSION RATE GOOD OR BAD? WHY DO I NEED TO TEST? BUT NO ONE WANTS TO DO TESTING AT MY ORGANIZATION. PLATFORM, PEOPLE, PROCESS WHAT DO I TEST AND HOW DO I TRACK IT? TIPS AND SUGGESTIONS FOR GETTING STARTED TAKEAWAYS @analyticandrew #eMetrics
- 12. No more HIPPO
- 13. 5xHigher ROI than PPC Up to
- 14. IS MY CONVERSION RATE GOOD OR BAD? WHY DO I NEED TO TEST? BUT NO ONE WANTS TO DO TESTING AT MY ORGANIZATION. PLATFORM, PEOPLE, PROCESS WHAT DO I TEST AND HOW DO I TRACK IT? TIPS AND SUGGESTIONS FOR GETTING STARTED TAKEAWAYS @analyticandrew #eMetrics
- 17. We have NO IDEAwhat to test
- 21. IS MY CONVERSION RATE GOOD OR BAD? WHY DO I NEED TO TEST? BUT NO ONE WANTS TO DO TESTING AT MY ORGANIZATION. PLATFORM, PEOPLE, PROCESS WHAT DO I TEST AND HOW DO I TRACK IT? TIPS AND SUGGESTIONS FOR GETTING STARTED TAKEAWAYS @analyticandrew #eMetrics
- 22. Nothing should be OFF LIMITS
- 25. Testing is often about CHANGINGpolicy & perception
- 26. IS MY CONVERSION RATE GOOD OR BAD? WHY DO I NEED TO TEST? BUT NO ONE WANTS TO DO TESTING AT MY ORGANIZATION. PLATFORM, PEOPLE, PROCESS WHAT DO I TEST AND HOW DO I TRACK IT? TIPS AND SUGGESTIONS FOR GETTING STARTED TAKEAWAYS @analyticandrew #eMetrics
- 27. Ok, I am ALL INNow what?
- 28. Tool Selection: BUILD or BUY
- 33. THE PEOPLE The most important thing
- 34. THE PEOPLE The most important thing BUT!
- 35. THE PEOPLE The most important thing one: focus efforts where they matter most two: avoid conflict-put opinions to the test three: empower people to optimize & be data-drive four: align teams around shared goals five: collaborate around hypothesis generation, test ideas & execution
- 38. RESEARCH 1. “If A/B tests are used in lieu of research, the variations are essentially guesses. We can improve outcomes of A/B tests by incorporating UX research to: •improve cause identification •develop more realistic hypotheses •identify more opportunities for experimentation” Nielsen Norman Group: www.nngroup.com
- 39. RESEARCH 1.
- 40. RESEARCH 1.
- 41. RESEARCH 1.
- 42. RESEARCH 1.
- 43. RESEARCH 1.
- 44. A theory can be proved by experiment; but no path leads from experiment to the birth of a theory.
- 45. RESEARCH 1. Issue • What is the conversion problem? Cause Theory • What do you think is causing the problem? Variation Hypotheses • What do you propose changing? Goals • What outcomes are expected?
- 46. RESEARCH 1. Issue • What is the conversion problem? Cause Theory • What do you think is causing the problem? Variation Hypotheses • What do you propose changing? Goals • What outcomes are expected?
- 47. RESEARCH 1.
- 48. RESEARCH 1. User-Experience to Improve Optimization Testing • Defining user intent and objections • Why did you visit? Were you successful? If not, why? • Exposing interface flaws • Changing colors vs. changing form fields • Measuring findability • Tree testing, site search, heatmaps • Cleaning up design variations • Reduce friction, maximize testing
- 49. RESEARCH 1. Bug tracking Brainstorming Site Search Web Analytics Site Surveys Chat
- 50. PRIORITIZE 2. • TEST • Where you put stuff for testing • INSTRUMENT • Fixing, adding, improving tagging, events, etc. • HYPOTHESIZE • Found something, but no clear solution • JUST DO IT (JDI) • No brainers • INVESTIGATE • Need to dig further, ask more questions Craig Sullivan: @optimiseordie
- 51. PRIORITIZE 2.
- 52. PRIORITIZE 2. Craig Sullivan Hypothesis Kit one: Because we saw (data/feedback) two: We expect that (change) will cause (impact) three: We’ll measure this using (data metric) https://medium.com/@optimiseordie/hypothesis-kit-2-eff0446e09fc
- 53. PRIORITIZE 2. Craig Sullivan Advanced Kit one: Because we saw (qual and quant data) two: We expect that (change) for (population) will cause (impact(s)) three: We except to see (data metric(s) change) over a period of (x business cycles) https://medium.com/@optimiseordie/hypothesis-kit-2-eff0446e09fc
- 54. EXPERIMENT 3.
- 55. EXPERIMENT 3.
- 56. EXPERIMENT 3.
- 57. EXPERIMENT 3.
- 58. ANALYZE, LEARN, REPEAT 4. Insert random cycle chart here
- 59. IS MY CONVERSION RATE GOOD OR BAD? WHY DO I NEED TO TEST? BUT NO ONE WANTS TO DO TESTING AT MY ORGANIZATION. PLATFORM, PEOPLE, PROCESS WHAT DO I TEST AND HOW DO I TRACK IT? TIPS AND SUGGESTIONS FOR GETTING STARTED TAKEAWAYS @analyticandrew #eMetrics
- 60. • Headlines • Sub headlines • Paragraph Text • Testimonials • Call to Action text • Call to Action Button • Links • Images • Content near the fold • Social proof • Media mentions • Awards and badges
- 63. IS MY CONVERSION RATE GOOD OR BAD? WHY DO I NEED TO TEST? BUT NO ONE WANTS TO DO TESTING AT MY ORGANIZATION. PLATFORM, PEOPLE, PROCESS WHAT DO I TEST AND HOW DO I TRACK IT? TIPS AND SUGGESTIONS FOR GETTING STARTED TAKEAWAYS @analyticandrew #eMetrics
- 65. Never stop testing, and your advertising will never stop improving. - David Ogilvy -
Editor's Notes
- #47: Common way to solve problems