The 11 Biggest A/B Testing Mistakes and How To Avoid Them
Download as PPTX, PDF2 likes769 views
The document presents a webinar featuring Peep Laja discussing common A/B testing mistakes and how to avoid them. Key mistakes highlighted include not having a data-driven hypothesis, using too small a sample size, and failing to run tests for adequate durations. The session encourages participants to engage on social media and emphasizes the importance of continual testing and awareness of various testing threats.
![#thinkppc
(function(){
var experimentID = 2809164105;
window.optimizely = window.optimizely || [];
if (typeof window.optimizely.data.experiments[experimentID] !== 'undefined') {
var experimentName = window.optimizely.data.experiments[experimentID].name;
var variationName =
window.optimizely.data.state.variationNamesMap[experimentID];
window.ga = window.ga || function() {(window.ga.q = window.ga.q ||
[]).push(arguments);}; window.ga.l =+ new Date();
window.ga('send', 'event', 'Optimizely', experimentID + ' ' + experimentName,
variationName, {'nonInteraction': 1});
}
})();](https://image.slidesharecdn.com/11biggesta-btestingmistakespeeplajahanapinfinal-150528180841-lva1-app6892/85/The-11-Biggest-A-B-Testing-Mistakes-and-How-To-Avoid-Them-25-320.jpg)
The 11 Biggest A/B Testing Mistakes and How To Avoid Them
- 1. #thinkppc HOSTED BY: The 11 13 Biggest A/B Testing Mistakes and How To Avoid Them Featuring Peep Laja
- 2. #thinkppc Presenters • Peep Laja – Guest Presenter – Founder of Conversion XL – @peeplaja • Roopa Carpenter – Moderator – CRO Manager at Hanapin Marketing – @roopacarpenter
- 3. #thinkppc Join the conversation • Include the hashtag #thinkppc in your Twitter tweets. Or use the webinar question box to send us questions.
- 4. #thinkppc Live Poll Question #1 How long have you been doing CRO? #thinkppc A. Less than 1 year B. 1-3 years C. 3-5 years D. 5+ years
- 5. #thinkppc Live Poll Question #2 How do you manage your account(s)? #thinkppc a) I’m part of an in-house PPC team. b) I do all the marketing myself. c) I work with an agency. d) I’m a consultant.
- 6. #thinkppc Peep Laja, ConversionXL @peeplaja Typical scenario: A company runs 100 tests / year. Lots of wins. But no difference on the bank account.
- 7. #thinkppc Precious time wasted on stupid tests Testing mistake #1:
- 8. #thinkppc You don’t have a data- driven, learning- oriented hypothesis Testing mistake #2:
- 10. #thinkppc You think you know what will work Testing mistake #3:
- 11. #thinkppc Your sample size is way too small Testing mistake #4:
- 12. #thinkppc
- 13. #thinkppc Calculate needed sample size in advance:
- 14. #thinkppc I won’t even look at results until there’s at least 250-350 conversions PER variation. Way more if you want to segment the data
- 15. #thinkppc Your tests don’t run long enough Testing mistake #5:
- 16. #thinkppc
- 17. #thinkppc You don’t test full weeks at a time Testing mistake #6:
- 18. #thinkppc
- 19. #thinkppc You run tests when you have no traffic Testing mistake #7:
- 21. #thinkppc Test data is not sent to third-party analytics Testing mistake #8:
- 23. #thinkppc
- 25. #thinkppc (function(){ var experimentID = 2809164105; window.optimizely = window.optimizely || []; if (typeof window.optimizely.data.experiments[experimentID] !== 'undefined') { var experimentName = window.optimizely.data.experiments[experimentID].name; var variationName = window.optimizely.data.state.variationNamesMap[experimentID]; window.ga = window.ga || function() {(window.ga.q = window.ga.q || []).push(arguments);}; window.ga.l =+ new Date(); window.ga('send', 'event', 'Optimizely', experimentID + ' ' + experimentName, variationName, {'nonInteraction': 1}); } })();
- 26. #thinkppc Peep Laja, ConversionXL @peeplaja
- 27. #thinkppc You give up after your first test for a hypothesis fails. Testing mistake #9:
- 28. #thinkppc
- 29. #thinkppc Full case study: http://conversionxl.com/case-study-how-we-improved-landing-page-conversion/ 79.3% uplift in conversions
- 30. #thinkppc
- 31. #thinkppc
- 32. #thinkppc
- 33. #thinkppc
- 34. #thinkppc
- 35. #thinkppc
- 38. #thinkppc You’re not aware of validity threats Testing mistake #10:
- 40. #thinkppc You’re ignoring small gains Testing mistake #11:
- 41. #thinkppc 5% monthly increase in your conversion rate will result in 80% uplift for the year. That’s how math works.
- 42. #thinkppc You copy competitors / leaders / other people’s tests Testing mistake #12:
- 45. #thinkppc Somebody else’s test results are useless
- 46. #thinkppc “Site shows stats from various A/B tests - Finally I've got evidence to show clients on a load of design decisions!”
- 47. #thinkppc
- 48. #thinkppc The process, the method, the analysis
- 49. #thinkppc You’re not running tests at all times Testing mistake #13:
- 50. #thinkppc PPC Retainers Need some assistance for your PPC? We offer on-going and one-time retainers. Learn More: http://www.hanapinmarketing.com/ppc-retainers/
- 52. #thinkppc Have more questions? Thank you for attending our webinar! #thinkppc • Contact us Directly: » Hanapin Feedback: marketing@hanapinmarketing.com