SlideShare a Scribd company logo
10 A/B Testing Mistakes That Make Your Wallet Cry !
Don’t Just Measure
Conversion Rate!
Sequential Testing
Is Not A/B Testing.
Random Testing.
Prioritization
of tests.
Small website
changes give small
conversion changes.
Calling tests early
Don’t surprise
your regulars.
Pre-plan your
tests sample size.
Website wide
consistency
for tests.
Segmenting requires
a bigger sample size.
A B
www.convert.com

More Related Content

PPTX
Living with acceptance tests: Beyond Write-Once (XP NYC)
PDF
Think Tank Presentation 5-2012
PPTX
Easy Breezy Small Business Bookkeeping
PDF
Selenium Shots by Kyle J. Ginavan & Ryan Felton
PPTX
How to reach a Data Driven culture
PDF
DAMA Webinar - Big and Little Data Quality
PDF
Determine Your Data Strategy
PDF
Impact Hiring: How Data Will Transform Youth Employment
Living with acceptance tests: Beyond Write-Once (XP NYC)
Think Tank Presentation 5-2012
Easy Breezy Small Business Bookkeeping
Selenium Shots by Kyle J. Ginavan & Ryan Felton
How to reach a Data Driven culture
DAMA Webinar - Big and Little Data Quality
Determine Your Data Strategy
Impact Hiring: How Data Will Transform Youth Employment

Viewers also liked (12)

PDF
PROG_UntoldStory ISV eBook_0706c FINAL
PDF
Big Data Revolution: Are You Ready for the Data Overload?
PPTX
Data Culture Keynote and Exec Track Birm Dec 8th
PDF
CDO Webinar: 2017 Trends in Data Strategy
PPTX
Data Gravity, IoT, and Time Series - ThingMonk 2015
Ā 
PDF
This Isn't 'Big Data.' It's Just Bad Data.
PDF
Data Science, Machine Learning and Neural Networks
PPTX
Introduction to Data Science
PDF
A Statistician's View on Big Data and Data Science (Version 1)
PDF
Myths and Mathemagical Superpowers of Data Scientists
PDF
Big Data [sorry] & Data Science: What Does a Data Scientist Do?
PDF
Working With Big Data
PROG_UntoldStory ISV eBook_0706c FINAL
Big Data Revolution: Are You Ready for the Data Overload?
Data Culture Keynote and Exec Track Birm Dec 8th
CDO Webinar: 2017 Trends in Data Strategy
Data Gravity, IoT, and Time Series - ThingMonk 2015
Ā 
This Isn't 'Big Data.' It's Just Bad Data.
Data Science, Machine Learning and Neural Networks
Introduction to Data Science
A Statistician's View on Big Data and Data Science (Version 1)
Myths and Mathemagical Superpowers of Data Scientists
Big Data [sorry] & Data Science: What Does a Data Scientist Do?
Working With Big Data
Ad

Recently uploaded (20)

PPTX
Supervised vs unsupervised machine learning algorithms
PPTX
The THESIS FINAL-DEFENSE-PRESENTATION.pptx
PPTX
Introduction to Firewall Analytics - Interfirewall and Transfirewall.pptx
PPTX
Business Acumen Training GuidePresentation.pptx
PPTX
1_Introduction to advance data techniques.pptx
PPTX
CEE 2 REPORT G7.pptxbdbshjdgsgjgsjfiuhsd
PPT
Miokarditis (Inflamasi pada Otot Jantung)
PDF
.pdf is not working space design for the following data for the following dat...
PPTX
IB Computer Science - Internal Assessment.pptx
PPTX
STUDY DESIGN details- Lt Col Maksud (21).pptx
PPTX
Introduction to Knowledge Engineering Part 1
PPTX
advance b rammar.pptxfdgdfgdfsgdfgsdgfdfgdfgsdfgdfgdfg
PDF
Lecture1 pattern recognition............
PPT
Chapter 2 METAL FORMINGhhhhhhhjjjjmmmmmmmmm
PPTX
05. PRACTICAL GUIDE TO MICROSOFT EXCEL.pptx
PPTX
Introduction-to-Cloud-ComputingFinal.pptx
PPTX
Computer network topology notes for revision
PPTX
Moving the Public Sector (Government) to a Digital Adoption
PPTX
ALIMENTARY AND BILIARY CONDITIONS 3-1.pptx
Supervised vs unsupervised machine learning algorithms
The THESIS FINAL-DEFENSE-PRESENTATION.pptx
Introduction to Firewall Analytics - Interfirewall and Transfirewall.pptx
Business Acumen Training GuidePresentation.pptx
1_Introduction to advance data techniques.pptx
CEE 2 REPORT G7.pptxbdbshjdgsgjgsjfiuhsd
Miokarditis (Inflamasi pada Otot Jantung)
.pdf is not working space design for the following data for the following dat...
IB Computer Science - Internal Assessment.pptx
STUDY DESIGN details- Lt Col Maksud (21).pptx
Introduction to Knowledge Engineering Part 1
advance b rammar.pptxfdgdfgdfsgdfgsdgfdfgdfgsdfgdfgdfg
Lecture1 pattern recognition............
Chapter 2 METAL FORMINGhhhhhhhjjjjmmmmmmmmm
05. PRACTICAL GUIDE TO MICROSOFT EXCEL.pptx
Introduction-to-Cloud-ComputingFinal.pptx
Computer network topology notes for revision
Moving the Public Sector (Government) to a Digital Adoption
ALIMENTARY AND BILIARY CONDITIONS 3-1.pptx
Ad

10 A/B Testing Mistakes that Make Your Wallet Cry

  • 1. 10 A/B Testing Mistakes That Make Your Wallet Cry ! Don’t Just Measure Conversion Rate! Sequential Testing Is Not A/B Testing. Random Testing. Prioritization of tests. Small website changes give small conversion changes. Calling tests early Don’t surprise your regulars. Pre-plan your tests sample size. Website wide consistency for tests. Segmenting requires a bigger sample size. A B www.convert.com