SlideShare a Scribd company logo
Experiment Driven Development
  The “real” engineering in software

            @mikegehard
       BS Chemical Engineering
       MS Software Engineering
How do you know that changes you
make to your web application are the
“right” ones?
What is “right”?
It depends.
What makes or saves my company
money?
Number of purchases
Number of new subscriptions
What is your theory about that
metric?
If you don’t know where you are
going, how do you know when you get
there?
By providing a larger skip link on the
signup page, we will increase the
number of purchases without
sacrificing new subscriptions.
A/B Testing while collecting data
Analyze the data without bias.
Leave the egos at the door.
When compared to the control, the
larger skip link increased purchases by
5% and did not decrease new
subscriptions.
Surprise?!?!?!?!?!
What’s next?
If you want to know why something
happened then propose a new
experiment to collect that data.
Repeat
Propose a theory about business
metric(s)

Experiment and measure those
metrics

Analyze the data and make a decision

Iterate and prosper
Thank you!
@mikegehard

More Related Content

PDF
Martijn Scheijbeler @ All Things DATA 2016
PDF
Conversion Hotel 2016 - Amy Harrison
PDF
Conversion Hotel 2016 - Dela Quist
PPTX
4 Ways Reports Suck (And How To Fix Them) - Alex Cohen - SMX Advanced London ...
PPTX
The true meaning of data
PPTX
The true meaning of data by Maciej Dabrowski
PDF
Validation and hypothesis based product management by Abdallah Al-Khalidi
PDF
Data for Hong Kong startups
Martijn Scheijbeler @ All Things DATA 2016
Conversion Hotel 2016 - Amy Harrison
Conversion Hotel 2016 - Dela Quist
4 Ways Reports Suck (And How To Fix Them) - Alex Cohen - SMX Advanced London ...
The true meaning of data
The true meaning of data by Maciej Dabrowski
Validation and hypothesis based product management by Abdallah Al-Khalidi
Data for Hong Kong startups

What's hot (20)

PDF
"The Startup's Guide to Customer Retention," Retention Science >> Jerry Jao [...
PPTX
Using Enhanced Ecommerce in Google Analytics for Business Growth
PPTX
GOING THE EXTRA MILE: Using Client Data to Improve Client Service, Loyalty, A...
PPTX
Analysis of "A Predictive Analytics Primer" by Thomas H. Davenport
PDF
Opticon 2017 Experimenting with Stats Engine
PPTX
A Predictive Analytics Primer
PDF
Measuring Roi etourism forum
PPTX
Directions for Hadoop Innovation, Yahoo
PDF
Grand Explorers: What we can learn from data innovators
PDF
Unleashing the Hero Data
PPT
10 tips for actionable web analytics
PPTX
Product Management Top 10 metrices
PPTX
Cross Device Measurement - All Things Data Conference
PPTX
How Predictive Analytics Can Help Strengthen Your Re-marketing Strategy
PDF
Analytics for startups
PDF
Unlocking the 4 Gateways of Growth
PPTX
Seeing signal through noise
PPTX
Introduction to cro
PDF
Skill session Google Analytics @WeWork Amsterdam (Mike van Hoenselaar)
PPT
Analysis Paralysis
"The Startup's Guide to Customer Retention," Retention Science >> Jerry Jao [...
Using Enhanced Ecommerce in Google Analytics for Business Growth
GOING THE EXTRA MILE: Using Client Data to Improve Client Service, Loyalty, A...
Analysis of "A Predictive Analytics Primer" by Thomas H. Davenport
Opticon 2017 Experimenting with Stats Engine
A Predictive Analytics Primer
Measuring Roi etourism forum
Directions for Hadoop Innovation, Yahoo
Grand Explorers: What we can learn from data innovators
Unleashing the Hero Data
10 tips for actionable web analytics
Product Management Top 10 metrices
Cross Device Measurement - All Things Data Conference
How Predictive Analytics Can Help Strengthen Your Re-marketing Strategy
Analytics for startups
Unlocking the 4 Gateways of Growth
Seeing signal through noise
Introduction to cro
Skill session Google Analytics @WeWork Amsterdam (Mike van Hoenselaar)
Analysis Paralysis
Ad

Viewers also liked (18)

PPT
Startup Metrics For Scottish Pirates (AARRR!) v1.3
PPTX
Experiment: The Lean startup
KEY
Experiment design primer
PDF
EDD (Experiment Driven Development)
PDF
Experiment Design - strategy and markets determine what you should test
PPT
Startup Metrics for Pirates: AARRR! (Startonomics SF 2008)
DOCX
Final Year project
PDF
The Green Lab - [05 A] Experiment design (basics)
PPT
Introduction to Reverse Engineering
PDF
Reverse engineering
PPT
Reverse engineering
PPTX
Reverse engineering
PDF
9. design of experiment
PPTX
Reverse engineering
PPTX
Experimental design
PPT
Experimental Design
PPT
Reverse Engineering
PPTX
Reverse engineering & its application
Startup Metrics For Scottish Pirates (AARRR!) v1.3
Experiment: The Lean startup
Experiment design primer
EDD (Experiment Driven Development)
Experiment Design - strategy and markets determine what you should test
Startup Metrics for Pirates: AARRR! (Startonomics SF 2008)
Final Year project
The Green Lab - [05 A] Experiment design (basics)
Introduction to Reverse Engineering
Reverse engineering
Reverse engineering
Reverse engineering
9. design of experiment
Reverse engineering
Experimental design
Experimental Design
Reverse Engineering
Reverse engineering & its application
Ad

Similar to Experiment Driven Development (20)

PDF
Milion Dollar Impact Through Metrics, Analytics & A/B Testing
PDF
Alistaire croll lean analytics - montreal lean startup circle - september 2018
PDF
Growth hacking: the growth hacker guide to analytics: how to get to data-dri...
PPTX
User Analytics Testing - SeleniumCamp 2015
PDF
Data-Driven UI/UX Design with A/B Testing
PDF
Optimisator : 8 Ways Analytics Helps Your Business Grow
PDF
Optimizely building your_data_dna_e_booktthh
PDF
[Elite Camp 2016] Martijn Scheijbeler - Optimization for Content Sites
 
PDF
Growing Your Business Through Experimentation
PDF
SEO Tactics, Fanbase Management, and Trends to watch
PPT
Ei Presentation on analytics
PPTX
A/B testing
PDF
5 B2B Content Marketing Hacks to Drive More Leads
PDF
10 things to do and check in Google Analytics for Construction Companies
PDF
A/B Testing Blueprint | Pirate Skills
PPTX
Using Data to Inform Information Architecture and User Experience
PDF
Ecommerce Test Ideation: What to Optimize and Why
PDF
How To Use Analytics To Improve Your Website
PDF
Predictably Improve Your B2B Tech Company's Performance by Leveraging Data
PDF
Measurement and metrics
Milion Dollar Impact Through Metrics, Analytics & A/B Testing
Alistaire croll lean analytics - montreal lean startup circle - september 2018
Growth hacking: the growth hacker guide to analytics: how to get to data-dri...
User Analytics Testing - SeleniumCamp 2015
Data-Driven UI/UX Design with A/B Testing
Optimisator : 8 Ways Analytics Helps Your Business Grow
Optimizely building your_data_dna_e_booktthh
[Elite Camp 2016] Martijn Scheijbeler - Optimization for Content Sites
 
Growing Your Business Through Experimentation
SEO Tactics, Fanbase Management, and Trends to watch
Ei Presentation on analytics
A/B testing
5 B2B Content Marketing Hacks to Drive More Leads
10 things to do and check in Google Analytics for Construction Companies
A/B Testing Blueprint | Pirate Skills
Using Data to Inform Information Architecture and User Experience
Ecommerce Test Ideation: What to Optimize and Why
How To Use Analytics To Improve Your Website
Predictably Improve Your B2B Tech Company's Performance by Leveraging Data
Measurement and metrics

More from Mike Gehard (8)

PDF
Sustainable developmentwithoutninjas
PDF
Agile the Pivotal Way
PDF
Introduction to Pair Programming
PPTX
Better Ruby Through Design Principles
KEY
Developing Developers
KEY
Focus and Meditation
PDF
Agile the pivotal way
KEY
Meditation and Software
Sustainable developmentwithoutninjas
Agile the Pivotal Way
Introduction to Pair Programming
Better Ruby Through Design Principles
Developing Developers
Focus and Meditation
Agile the pivotal way
Meditation and Software

Recently uploaded (20)

PDF
Review of recent advances in non-invasive hemoglobin estimation
PDF
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
PDF
Spectral efficient network and resource selection model in 5G networks
PDF
Building Integrated photovoltaic BIPV_UPV.pdf
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
PDF
The Rise and Fall of 3GPP – Time for a Sabbatical?
PDF
Unlocking AI with Model Context Protocol (MCP)
PPTX
20250228 LYD VKU AI Blended-Learning.pptx
PDF
NewMind AI Monthly Chronicles - July 2025
PDF
NewMind AI Weekly Chronicles - August'25 Week I
PDF
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
PDF
Per capita expenditure prediction using model stacking based on satellite ima...
PDF
Chapter 3 Spatial Domain Image Processing.pdf
PDF
Agricultural_Statistics_at_a_Glance_2022_0.pdf
PDF
cuic standard and advanced reporting.pdf
PDF
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
PPTX
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
PDF
Mobile App Security Testing_ A Comprehensive Guide.pdf
PDF
Advanced methodologies resolving dimensionality complications for autism neur...
PDF
KodekX | Application Modernization Development
Review of recent advances in non-invasive hemoglobin estimation
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
Spectral efficient network and resource selection model in 5G networks
Building Integrated photovoltaic BIPV_UPV.pdf
Dropbox Q2 2025 Financial Results & Investor Presentation
The Rise and Fall of 3GPP – Time for a Sabbatical?
Unlocking AI with Model Context Protocol (MCP)
20250228 LYD VKU AI Blended-Learning.pptx
NewMind AI Monthly Chronicles - July 2025
NewMind AI Weekly Chronicles - August'25 Week I
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
Per capita expenditure prediction using model stacking based on satellite ima...
Chapter 3 Spatial Domain Image Processing.pdf
Agricultural_Statistics_at_a_Glance_2022_0.pdf
cuic standard and advanced reporting.pdf
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
Mobile App Security Testing_ A Comprehensive Guide.pdf
Advanced methodologies resolving dimensionality complications for autism neur...
KodekX | Application Modernization Development

Experiment Driven Development