Practical Price Testing
Pricing Challenge in Summary
Nichols & May “$65 funeral”
About Me
 I have run over 500 marketing experiments
 Many of the most successful tests have been related
to pricing & packaging
 I teach a/b testing courses around topics such as
statistics and design of experiments
Have run experiments for...
Intro
Price testing isn’t just the price. It encompasses:
 Price levels
 Price gaps
 Price anchors
 Discount strategy
 Line up strategy
 Feature composition
 Cannibalization
 Pricing by Marketing Channel
 Pricing by Customer Segment
 Pricing by Vertical
 and more
Broader Impact
Pricing decisions don’t just impact immediate
conversion. It also impacts...
 customer satisfaction and loyalty
 ability to sell other things in the short-term
 refund rates
 future revenue stream possibilities
 brand perception
 customer expectations
Can you? Should you?
There’s some debate around the legitimacy of price testing
against live customers.
 It is done all the time.
Be careful not to discriminate.
 Randomization is not discrimination.
Be careful around consistency of experience
 Test targeting can be tricky in particular for logged-out
experiences
 Test quickly and have a back-pocket offer.
Toolset
 Tougher to do with off-the-shelf testing tools
 Often requires building own testing and reporting
capabilities
Goals
As with any experiment, what are the key success metrics? Just
because it’s a “price test” doesn’t mean everyone is necessarily on
the same page at the start.
KPI options
 Top line revenue
 Price charged at checkout
 Bottom line profit short-term
 Bottom line profit long-term
 New customers
 NPS / WTR / WOM etc.
 etc.
Effort vs Reward
 Price testing is at the harder end of the optimization
spectrum
 The payoff is well worth the investment however
 I have seen > 40% lifts from price testing
 Price testing has the broadest impact: not a page or
segment usually but an overlay on top of the whole
business – huge leverage
Lab vs Live
 Ideal is to start testing in the lab (surveys, focus
groups, conjoints, etc.)
 Once you have pretty good confidence in certain
directions, then move it to live tests
 Does not have to be done in that order however.
Controlling Variables
 Simplest approach is to test in phases:
 Packages (e.g., components of the offer)
 Lineup (e.g., how many packages there are)
 Messaging (e.g., how to sell each package)
 Prices for Core Offering (the actual price levels)
 Price levels for Add-On Offerings
 Timing or Sequence of Sales Cycle
 Discounts
Be sure that you are either testing just one of these factors at a
time, or else establish an appropriate experimental design to tease
out the effects of the different factors.
Continuous vs Discrete Variables
 Price is continuous, can test more points on the
curve without additional sampling overhead
 Number of packages is not quite continuous
 1 means no choice
 Odd numbers are best
 Limited to single digits
When to Price Test
 You have new products
 Your products have changed
 Your competition has changed their pricing
 You’re at a new part of the market adoption curve
 You changed strategic directions
Subscriptions
 SaaS? Have to use cohorts
 Trial period? Have to track free to paid
 Monthly subscription? Get a few points on the curve
and estimate
 Annual subscription? Harder to estimate
Choosing Your Customers
 In some cases, the prices you choose determines the
kinds of customers you’ll get.
 The are downstream implications of acquiring different
types of customers.
 Lower list prices or frequently discounted prices bring in
more more conscious customers. The may be:
 less loyal
 less likely to buy future offerings
 more likely to cancel
 more likely to use customer support
Discounting
 You should study the
 frequency of discounts
 steepness of discounts
 duration of discounted period
 predictability of discounts
 impact on brand perception
 impact on loyalty
 impact on margins
 discounting wars with competitors
Scarcity
Scarcity is closely related to price
 The price will go up soon
 The discount will end soon
 There are a limited number
 This offer only available for customers from... /
who...
Fake Door Testing
 Possible in a modified manner with price tests, but
can be tricky. Can pair with limited quantity
messaging when done authentically.
 More easily done with product
Free
 A free version can bring in lots of potential customers
 Offering a free version can be viewed as an advertising expense
 Free can be important or even necessary from a competitive standpoint
 Free can also impact brand perception and ability to sell very high end products
 Free can also incur support costs
 Free can be a way to get valuable customer feedback to improve the product
 Free can be a way to build a network effect for certain offerings
 Free has a cannibalization impact that can be measured
Free Trial
To test:
 Credit card required vs not
 Trial period
 Auto-charged or not
 Feature limits
 What happens if customer doesn’t end up paying. Do
any features remain? Any data saved?
Refer a Friend as Pricing
 “Refer 3 friends to get an additional 1 GB of space”
is a form of price... instead of paying you money,
they are paying you in-kind with social-based
marketing
 How else can you “charge” without asking for
dollars?
 content uploads
 P2P disk usage
 data or other ways to help build/grow the product
Price Display
 Actual Price
 Total amount
 Per Month, Per Day etc.
 Perceived Price
 Valued at
 Compare at
 Price before of after discounts?
 Component Value
 Price Display
 Strike though
 List
 Discount Display
 % off
 $ off
 You save
 Badges
 Best value
 Most popular
Magic numbers
 Endings .99 .95 etc
 Round numbers
 Trick specificity
Package Hiding
 Companies tend to ever expanding product flavors and
package sets. Over time what started out as a rational
for the business becomes a drag on conversion
 Simplest way to approach price testing is to trim back the
lines offered to a more management set (1 to 3)
 Supplementary packages can be hidden completely or
pushed down to deeper levels or segment-based
marketing – excluded from general pop messaging
 This approach will give some idea of loss due to hazards
of excessive choice
Loyal Customer Pricing
 Upgrader discounts
 1st year premium
 Minimum period
Quantity Pricing
 Buy more save more
 Usage based
Indecision Risk
 Too many price value trade-offs and prospects bail
out
 Must also find right level of product detail to not
overwhelm, but still sufficiently inform
Component Values
 Hard to market all the features or benefits
 Should quantify relative value of each and present
as such
 Each package must have one sufficiently high value
component for important customer segments
Negative Example
Negative Example
 Strike through only on one package
 Huge jump between low and mid package
 Highlight (“best value”) on most expensive package
 Package/Pricing but no CTA based on that info
 State fees buried elsewhere
 Inconsistent messaging about differences between
packages
 Combination of transaction, subscription/trial, and lead
gen not easily parsed
Positive Example
Positive Example
 Odd number of packages
 Center package is recommended
 Pricing displayed at the lower-seeming “per month” amount
 Original strikethrough price, % off highlighted & amount saved
 Dead-simple benefits list; Clutter free design
 Visual hierarchy of CTAs
 Risk-free messaging
 Minimal asterisking
This page was tested extensively:
Game Changing Examples
 Amazon Prime
 Amazon partner / affiliate store
 Apple App store
 Southwest Airlines
 Netflix
 Adobe Creative Cloud
Conclusion
 Price testing encompasses a huge array of testing
possibilities
 Price testing requires a serious commitment and
some finesse to get right
 Price testing is likely the biggest lever you have

More Related Content

PPS
Testing Strategies
PDF
Email Testing Basics to Increase Your Email Conversion Rate
PPT
How to price your services
PPTX
Pay What You Want
PDF
Multi-Touch Attribution Models: Improve Your Advertising ROI
PPTX
The Paradigm: From Sales to Profits Using Optimization Analytics
PPT
“Who's Calling? Customizing the Caller Experience to Feed the Bottom Line”
PDF
4 steps to writing a killer subject line
Testing Strategies
Email Testing Basics to Increase Your Email Conversion Rate
How to price your services
Pay What You Want
Multi-Touch Attribution Models: Improve Your Advertising ROI
The Paradigm: From Sales to Profits Using Optimization Analytics
“Who's Calling? Customizing the Caller Experience to Feed the Bottom Line”
4 steps to writing a killer subject line

Similar to Practical Price Testing (20)

PPTX
SaaStr Annual 2024: How to Leverage Pricing and Packaging to Drive Growth, Re...
PDF
Digitale verlage by Günther Haslbeck / Ovenga Media
KEY
Pricing
PPTX
Building Innovative Subscription-based Businesses: Lecture 3
PPTX
Subscribed 2019 - Customer First Approach to Pricing
PDF
Monika Saha - Key Considerations When Operationalizing Your Pricing Strategy
PPTX
Subscribed NYC 2017: Monetizing Subscription Services
PDF
Pricing - what's good pricing
PDF
Unlocking Scale Through Pricing
PDF
Your Price is Wrong: Pricing Your Products for Maximum Return, Alan Albert, M...
PPTX
Subscribed 2016: Monetizing Subscription Services
PDF
Flat6 Labs Abu Dhabi Ignite Program - Startup Product Pricing Workshop 2021.0...
PPTX
10 Principles of Pricing
PDF
Lean pricing startups
PDF
Product pricing apr_2013
PDF
Pricing 101 for Startups
PDF
Pricing Strategies Guide - How To Define Pricing
PDF
Book Summary : Confessions of the Pricing Man: how price affects everything
PDF
The CMO's Guide to the Subscription Economy: Innovative Pricing and Packaging...
PPTX
Price is Right: Pricing in Today's Digital Association
SaaStr Annual 2024: How to Leverage Pricing and Packaging to Drive Growth, Re...
Digitale verlage by Günther Haslbeck / Ovenga Media
Pricing
Building Innovative Subscription-based Businesses: Lecture 3
Subscribed 2019 - Customer First Approach to Pricing
Monika Saha - Key Considerations When Operationalizing Your Pricing Strategy
Subscribed NYC 2017: Monetizing Subscription Services
Pricing - what's good pricing
Unlocking Scale Through Pricing
Your Price is Wrong: Pricing Your Products for Maximum Return, Alan Albert, M...
Subscribed 2016: Monetizing Subscription Services
Flat6 Labs Abu Dhabi Ignite Program - Startup Product Pricing Workshop 2021.0...
10 Principles of Pricing
Lean pricing startups
Product pricing apr_2013
Pricing 101 for Startups
Pricing Strategies Guide - How To Define Pricing
Book Summary : Confessions of the Pricing Man: how price affects everything
The CMO's Guide to the Subscription Economy: Innovative Pricing and Packaging...
Price is Right: Pricing in Today's Digital Association
Ad

Recently uploaded (20)

PPTX
Strategic Sage Digital-The Professional Digital Marketing Company in Mohali.pptx
PDF
Retaining SEO Rankings During Website Redesign.pdf
PDF
How to Break Into AI Search with Andrew Holland
PPTX
1 percent Clicks, percent Traffic Loss-Your SEO Stack Isn’t Built for AI
PPT
Market Segmentation and Positioning(3).ppt
PDF
Snapshot of Consumer Behaviors of July 2025-EOLiSurvey (EN).pdf
PPT
Market research before Marketing Research .PPT
PDF
Missing skill for SEO in AI Era eSkydecode.pdf
PDF
The Role of Search Intent in Shaping SEO Strategies in 2025
PPTX
APA Examples Reference Examples Style and
PPT
Introduction to consumer behavior(1).PPT
PDF
Biography of Brady Beitlich
PDF
Generation Alpha Report 2025 x DKC Analytics.pdf
PPTX
AI usage and the Social Media Marketing World
PDF
Social Media Marketing in 2025 blog 1 2.pdf
PPTX
Transform Your Business with Top Digital Marketing Services_EGlogics.pptx
PPTX
Best Social Media Marketing Company in Lucknow
PDF
SEO vs. AEO: Optimizing for Google vs AI-Powered Search Assistants
PPTX
Digital-Marketing-Strategy-Trends-and-Best-Practices-for-2025 PPT3.pptx
DOCX
Auctioneer project lead by Ali Hasnain jappa
Strategic Sage Digital-The Professional Digital Marketing Company in Mohali.pptx
Retaining SEO Rankings During Website Redesign.pdf
How to Break Into AI Search with Andrew Holland
1 percent Clicks, percent Traffic Loss-Your SEO Stack Isn’t Built for AI
Market Segmentation and Positioning(3).ppt
Snapshot of Consumer Behaviors of July 2025-EOLiSurvey (EN).pdf
Market research before Marketing Research .PPT
Missing skill for SEO in AI Era eSkydecode.pdf
The Role of Search Intent in Shaping SEO Strategies in 2025
APA Examples Reference Examples Style and
Introduction to consumer behavior(1).PPT
Biography of Brady Beitlich
Generation Alpha Report 2025 x DKC Analytics.pdf
AI usage and the Social Media Marketing World
Social Media Marketing in 2025 blog 1 2.pdf
Transform Your Business with Top Digital Marketing Services_EGlogics.pptx
Best Social Media Marketing Company in Lucknow
SEO vs. AEO: Optimizing for Google vs AI-Powered Search Assistants
Digital-Marketing-Strategy-Trends-and-Best-Practices-for-2025 PPT3.pptx
Auctioneer project lead by Ali Hasnain jappa
Ad

Practical Price Testing

  • 2. Pricing Challenge in Summary Nichols & May “$65 funeral”
  • 3. About Me  I have run over 500 marketing experiments  Many of the most successful tests have been related to pricing & packaging  I teach a/b testing courses around topics such as statistics and design of experiments Have run experiments for...
  • 4. Intro Price testing isn’t just the price. It encompasses:  Price levels  Price gaps  Price anchors  Discount strategy  Line up strategy  Feature composition  Cannibalization  Pricing by Marketing Channel  Pricing by Customer Segment  Pricing by Vertical  and more
  • 5. Broader Impact Pricing decisions don’t just impact immediate conversion. It also impacts...  customer satisfaction and loyalty  ability to sell other things in the short-term  refund rates  future revenue stream possibilities  brand perception  customer expectations
  • 6. Can you? Should you? There’s some debate around the legitimacy of price testing against live customers.  It is done all the time. Be careful not to discriminate.  Randomization is not discrimination. Be careful around consistency of experience  Test targeting can be tricky in particular for logged-out experiences  Test quickly and have a back-pocket offer.
  • 7. Toolset  Tougher to do with off-the-shelf testing tools  Often requires building own testing and reporting capabilities
  • 8. Goals As with any experiment, what are the key success metrics? Just because it’s a “price test” doesn’t mean everyone is necessarily on the same page at the start. KPI options  Top line revenue  Price charged at checkout  Bottom line profit short-term  Bottom line profit long-term  New customers  NPS / WTR / WOM etc.  etc.
  • 9. Effort vs Reward  Price testing is at the harder end of the optimization spectrum  The payoff is well worth the investment however  I have seen > 40% lifts from price testing  Price testing has the broadest impact: not a page or segment usually but an overlay on top of the whole business – huge leverage
  • 10. Lab vs Live  Ideal is to start testing in the lab (surveys, focus groups, conjoints, etc.)  Once you have pretty good confidence in certain directions, then move it to live tests  Does not have to be done in that order however.
  • 11. Controlling Variables  Simplest approach is to test in phases:  Packages (e.g., components of the offer)  Lineup (e.g., how many packages there are)  Messaging (e.g., how to sell each package)  Prices for Core Offering (the actual price levels)  Price levels for Add-On Offerings  Timing or Sequence of Sales Cycle  Discounts Be sure that you are either testing just one of these factors at a time, or else establish an appropriate experimental design to tease out the effects of the different factors.
  • 12. Continuous vs Discrete Variables  Price is continuous, can test more points on the curve without additional sampling overhead  Number of packages is not quite continuous  1 means no choice  Odd numbers are best  Limited to single digits
  • 13. When to Price Test  You have new products  Your products have changed  Your competition has changed their pricing  You’re at a new part of the market adoption curve  You changed strategic directions
  • 14. Subscriptions  SaaS? Have to use cohorts  Trial period? Have to track free to paid  Monthly subscription? Get a few points on the curve and estimate  Annual subscription? Harder to estimate
  • 15. Choosing Your Customers  In some cases, the prices you choose determines the kinds of customers you’ll get.  The are downstream implications of acquiring different types of customers.  Lower list prices or frequently discounted prices bring in more more conscious customers. The may be:  less loyal  less likely to buy future offerings  more likely to cancel  more likely to use customer support
  • 16. Discounting  You should study the  frequency of discounts  steepness of discounts  duration of discounted period  predictability of discounts  impact on brand perception  impact on loyalty  impact on margins  discounting wars with competitors
  • 17. Scarcity Scarcity is closely related to price  The price will go up soon  The discount will end soon  There are a limited number  This offer only available for customers from... / who...
  • 18. Fake Door Testing  Possible in a modified manner with price tests, but can be tricky. Can pair with limited quantity messaging when done authentically.  More easily done with product
  • 19. Free  A free version can bring in lots of potential customers  Offering a free version can be viewed as an advertising expense  Free can be important or even necessary from a competitive standpoint  Free can also impact brand perception and ability to sell very high end products  Free can also incur support costs  Free can be a way to get valuable customer feedback to improve the product  Free can be a way to build a network effect for certain offerings  Free has a cannibalization impact that can be measured
  • 20. Free Trial To test:  Credit card required vs not  Trial period  Auto-charged or not  Feature limits  What happens if customer doesn’t end up paying. Do any features remain? Any data saved?
  • 21. Refer a Friend as Pricing  “Refer 3 friends to get an additional 1 GB of space” is a form of price... instead of paying you money, they are paying you in-kind with social-based marketing  How else can you “charge” without asking for dollars?  content uploads  P2P disk usage  data or other ways to help build/grow the product
  • 22. Price Display  Actual Price  Total amount  Per Month, Per Day etc.  Perceived Price  Valued at  Compare at  Price before of after discounts?  Component Value  Price Display  Strike though  List  Discount Display  % off  $ off  You save  Badges  Best value  Most popular
  • 23. Magic numbers  Endings .99 .95 etc  Round numbers  Trick specificity
  • 24. Package Hiding  Companies tend to ever expanding product flavors and package sets. Over time what started out as a rational for the business becomes a drag on conversion  Simplest way to approach price testing is to trim back the lines offered to a more management set (1 to 3)  Supplementary packages can be hidden completely or pushed down to deeper levels or segment-based marketing – excluded from general pop messaging  This approach will give some idea of loss due to hazards of excessive choice
  • 25. Loyal Customer Pricing  Upgrader discounts  1st year premium  Minimum period
  • 26. Quantity Pricing  Buy more save more  Usage based
  • 27. Indecision Risk  Too many price value trade-offs and prospects bail out  Must also find right level of product detail to not overwhelm, but still sufficiently inform
  • 28. Component Values  Hard to market all the features or benefits  Should quantify relative value of each and present as such  Each package must have one sufficiently high value component for important customer segments
  • 30. Negative Example  Strike through only on one package  Huge jump between low and mid package  Highlight (“best value”) on most expensive package  Package/Pricing but no CTA based on that info  State fees buried elsewhere  Inconsistent messaging about differences between packages  Combination of transaction, subscription/trial, and lead gen not easily parsed
  • 32. Positive Example  Odd number of packages  Center package is recommended  Pricing displayed at the lower-seeming “per month” amount  Original strikethrough price, % off highlighted & amount saved  Dead-simple benefits list; Clutter free design  Visual hierarchy of CTAs  Risk-free messaging  Minimal asterisking This page was tested extensively:
  • 33. Game Changing Examples  Amazon Prime  Amazon partner / affiliate store  Apple App store  Southwest Airlines  Netflix  Adobe Creative Cloud
  • 34. Conclusion  Price testing encompasses a huge array of testing possibilities  Price testing requires a serious commitment and some finesse to get right  Price testing is likely the biggest lever you have