Testing Is Not CRO But It’s Still Darn Important
Download as PPTX, PDF0 likes3,794 views
The document discusses the importance of A/B testing for improving user experience and increasing conversion rates, outlining various testing types such as synchronous, asynchronous, and multivariate testing. It emphasizes the need for sufficient data and proper conditions to run effective tests while highlighting alternative optimization methods for situations where testing is not feasible. Additionally, it offers insights into optimizing offers and user interactions through qualitative research, retargeting, and usability studies.
![“Respondents targeting more than 20% of their
visitors with personalized content achieved a 5%
conversion rate, which was 2x the average
conversion rate of 2.5% in their study” [Adobe
2014 Survey]](https://image.slidesharecdn.com/3a-testingisnotcrobutitsstilldarnimportant-150728202903-lva1-app6892/85/Testing-Is-Not-CRO-But-It-s-Still-Darn-Important-78-320.jpg)
Testing Is Not CRO But It’s Still Darn Important
- 1. TESTING IS NOT CRO BUT IT’S STILL DARN IMPORTANT
- 2. FIRST LET’S COVER SOME A/B TESTING BASICS
- 3. Types Of Tests Synchronous Asynchronous Sequential Testing A/B/…/N Testing Multivariate Testing Test Types
- 4. Synchronous Testing July 1st - July 15th July 16th - July 30th
- 6. A/b/…/n Testing You’ll Get Page Conversion Data (Not Element By Element) Better For Low Traffic Sites
- 7. A/b/…/n Testing Some Rules: Isolate Individual Elements If You Don’t Isolate Elements, DO NOT attribute the lift or loss to any single element
- 8. Multivariate Testing You’ll Get Element By Element Data Don’t Run If Less Than 100k Pageviews Per Month
- 9. NOW LET’S GIVE A/B TESTING SOME LOVE
- 10. I BUILT MY CAREER AROUND SPLIT TESTING
- 12. IT’S THE ONLY WAY TO GET DEFINITIVE ANSWERS IN A CLEAN ENVIRONMENT
- 13. TESTING HELPS GUIDE DECISIONS AND IMPROVE THE USER EXPERIENCE WHILE BOOSTING PROFITS
- 26. EVERYONE WANTS TO TEST EVEN WHEN THEY CAN’T
- 27. SIMPLY PUT: NOT EVERYONE CAN OR SHOULD BE SPLIT TESTING
- 29. NOT BEING ABLE TO TEST DOESN’T MEAN…
- 30. …NOT BEING ABLE TO OPTIMIZE
- 31. THERE ARE PLENTY OF WAYS TO BOOST CONVERSIONS OUTSIDE OF SPLIT TESTING
- 32. TODAY IN A NUTSHELL 1. When You Can Test 2. Should You Actually Test 3. What To Do When You Can’t/Shouldn’t Test
- 33. Part Can I Actually Run A Test
- 34. Testing Eligibility • ~100 Conversion Per Variation • Run for at least a full week • Not run longer than 8 weeks • Test can reach 95% confidence in this time frame
- 35. Testing Eligibility • ~100 Conversion Per Variation • Run for at least a full week • Not run longer than 8 weeks • Test can reach 95% confidence in this time frame
- 36. Conversion Per Day Chart Minimum Number Of Conversions Per Day Test Period (Days) 2 Variants 3 Variants 4 Variants 5 Variants 7 29 43 58 72 14 15 22 29 36 21 10 15 20 24 28 8 11 15 18 35 6 9 12 15 42 5 8 10 12 49 5 7 9 11 56 4 6 8 9
- 38. Problem: Not Enough Conversions Solution: Run Longer Secondarily: If Outside Of 8 Week Testing Window - Don’t Test It
- 39. VARIABLE NUMBER ISN’T THE ONLY THING THAT DICTATES TIME!
- 40. THE PERCENT LIFT IS A HUGE TIMING FACTOR
- 43. In order for this test to get statistical results, it would require a 10% lift
- 44. AND SO IS THE CONFIDENCE RATEBut we’ll talk more about that later
- 45. Does Your Page Get Enough Daily Conversions? YES No Can’t Test - Sorry! Is There A Big Enough Lift Potential?* YES No This Is A Testable Page Can’t Test - Sorry! *With An Appropriate Confidence Rate/Index
- 46. Part Should I Actually Run That Test
- 50. Functional Does the page work and do what it’s supposed to?
- 53. Accessible Can people of all skill levels on any device use the site? What are the barriers on the site?
- 56. Usable Can people understand your site, i.e., is it user friendly?
- 57. Intuitive Does the site have a logical flow? Does the offer match the source?
- 58. Persuasive Is your site convincing people to buy or take an action?
- 62. “For every test you run, there are an infinite number of other tests that you could have run” - Justindamus 120 AD
- 64. Part How To Optimize If I Can’t/Shouldn’t Test
- 65. You Have A Few Options • Evaluate Your Offer • On-Site Retargeting • Changes Based On Qualitative Research
- 67. It’s not a traffic problem, it’s an offer problem
- 68. It’s not a testing problem, it’s an offer problem
- 69. You might have to head back to the drawing board completely. Does your product/service provide real value?
- 70. You also might be going after the wrong traffic source. Identify if your ideal customer is ‘hanging out’ on the traffic medium
- 72. What is On Site Retargeting? Evaluating current interactions and providing a relevant offer based on these interactions.
- 73. Time On Site Number Of Pageviews Type Of Content Consumed Exit Intent Referring Source
- 75. Just Under Half Of All Offer Pageviews Sent By Optimonk
- 76. Typical Campaign Conversion: 69% With Exit pop: 80.7%
- 77. Using this method we got 2,689 more leads in just 14 days!
- 78. “Respondents targeting more than 20% of their visitors with personalized content achieved a 5% conversion rate, which was 2x the average conversion rate of 2.5% in their study” [Adobe 2014 Survey]
- 82. Here are a few things to try • Click/Scroll Tracking • Usability Testing • User Surveys • Session Recordings
- 86. People who actually scroll to the bottom click Not enough people are scrolling that far down ~15% of all clicks on this page are on the nav bar
- 87. Try a shorter variation to get people to scroll to the CTA. Add multiple CTA buttons at areas where scrolling is highest. Hide the nav bar on this page
- 90. USER SURVEYS
- 91. • Was it was easy to find what you were looking for? • Were the words/vocab used to define categories/sub-categories clear to you? • Do you have any suggestions to improve our website navigation? • Does our website look credible to you? • Is our pricing clear? • Is there anything else you’d like to know before signing up with us? • Will you shop with us again? Why/why not? • Do you think the form has any confusing/unnecessary input fields? QUESTION IDEAS
- 94. Shows individual recordings of a user interacting with your site. Unlike heatmaps, this isn’t aggregated data
- 95. Really blow their mind with a real life user recording with audio
- 96. TRICKS TO TEST WHEN YOU DON’T HAVE THE TRAFFIC
- 98. Test Legnth Minimum Conversions Per Variation Per Day 7 Days 29 14 Days 15 21 Days 10 28 Days 8 35 Days 6 42 Days 5 49 Days 5 56 Days 4
- 103. Lift: 20% Confidence Rate: 95% _________________ Required Sample Size: 7,682
- 104. Lift: 20% Confidence Rate: 90% _________________ Required Sample Size: 6,052 21% Decrease
- 105. REMEMBER - SPLIT TESTING IS NOT OPTIMIZATION…
- 107. Split Testing Personalization User Surveys Funnel Analysis Web Analytics Personas Neurotesting Advanced Segments Eye Tracking Scroll Tracking CRO
- 108. SPLIT TESTING CAN ACTUALLY HURT A COMPANY
- 109. SO PRACTICE SAFE TESTING WITH A PAGE THAT MATTERS
- 110. • Email: justin.rondeau@digitalmarketer.com • Twitter: @jtrondeau • Website: digitalmarketer.com Justin Rondeau Optimization Manager, Digital Marketer Let’s Talk More!