SlideShare a Scribd company logo
9/30/2011




      What to Expect from This
              Session
   Explore the fundamentals and benefits of Lift Modeling
    and Constraint-Based Optimization Analytics

   Learn how CVS Caremark has utilized their POS
    channel to deliver relevant offers and improve ROI

        - Behind-the-scenes analytic engine that drives
          differentiated offers to unique types of customers
        - Program evolution
        - Lessons learned




              Introductions




                                                                      1
9/30/2011




                          Who We Are
     Consulting firm specializing in information-based marketing
      and customer intelligence

     Founded in 1999, with headquarters in Burlington, MA

     Business problem solvers, skilled in analytics

     Broad, practical experience across multiple industries

     Outcomes driven - strive to deliver work that produces
      significant bottom line impact




                          What We Do
                   Customer
                  Intelligence
                 and Reporting
                                                   Support Business Decision
  Multi-media
                                                   Making / Processes
                                   Predictive
  Marketing                       Analytics and
 Effectiveness                      Scoring

                                                   Increase Business Growth
                                                   and Profitability
  Campaign                         Customer
 Planning and                    Acquisition and
 Measurement                      Development
                                                   Optimize Marketing
                    Loyalty
                   Program                         Spending
                   Strategy




                                        CVS



 What does CVS stand for?
           a. Consumer Value Stores
           b. Caroline, Vanessa, Sophie
           c. Convenience, Value, Service
           d. Computer Vision Syndrome




                                                                                      2
9/30/2011




                          CVS



    Nation’s largest retail pharmacy chain
    Recently opened 7,300th store
    CVS fills more than 1 out of every 7 prescriptions
     in America
    ExtraCare is the largest and most successful
     retail loyalty program in the country




     ExtraCare Loyalty Program



                          (brief video)




    Do you have your ExtraCare
              Card?
 68 MM active ExtraCare cardholders

 70% of CVS transactions are on the card

 ExtraCare customers visit the store every month, which
    is twice as often as the typical non-ExtraCare shopper

 Every second, 32 ExtraCare cards are scanned in the US

 The very next day, these transactions can be
    transformed into customer insights




                                                                    3
9/30/2011




              Lift Modeling and
                 Optimization




           Incremental Sales Potential
“Best” customers are not always the same ones who drive campaign profitability

        Best Customers - Top Sales             Promotion Sensitive,
                                              High Incremental Sales




   Limit or Exclude from                                    New Population -
   Discounting Strategy                                     Test Offer Levels

                Response Model                       Lift Model
                                     Best
                                     Offers




                           Lift Modeling
            Focus directly on the consumer behavior that drives ROI



     Predicts incremental response rather than total response

     Model score provides a mechanism to target customer
      with the highest likelihood to deliver incremental sales

     The challenge: incremental sales is an aggregate
      measure, while modeling and targeting must be done at
      the individual level




                                                                                        4
9/30/2011




            Offer Optimization Engine
                                         Objective:
        Determine the optimal offer assignment for each customer given
                        certain goals and constraints


        Automated approach to offer assignment; allows quick evaluation
         of numerous scenarios

        Optimization targets: maximum lift sales, program revenue, units,
         ROI, margin impact; minimum incentive cost

        Optimization Constraints: budget, incentive cost, ROI hurdle,
         total sales

        Flexibility is a key benefit




                  Optimization in Action




                       How do you do it?
Start by asking: are you leveraging everything you know about your customer?

                                              Shopping
                                              Behavior
             Promotion                                              Category Breadth
          History / Channel
             Preference



                                                                          Basket Profiles
    Digital Behavior




                                                                          Tenure
           Payment Type



                               Employees /
                                                         Demographics
                              Non-Employees




                                                                                                   5
9/30/2011




                      POS Marketing
                         at CVS




           POS Coupons - Overview
    Sophisticated targeting engine drives delivery of POS Coupons

                                                        Printed at Bottom of Cash Register Receipt
                                                        Primary Objective: Drive return visits
                                                        Customer data (what they buy, who they are) drives
                                                         offer assignment
                                                        Offers: Product-Specific or “Open-Ended”
                                                        Open-Ended Offers
                                                                 -      Discount on any non-Rx item in the store
                                                                 -      Automated trigger program
                                                                 -      Redeemable on next visit
                                                                 -      Broader offers drive most sales lift and ROI
                                                                 -      Largest CVS-funded incentive program




CVS Approach – Trigger Offers
Customized and Dynamic Targeted Offers
                                                                     Illustrative


  Customer 1:              Customer 2:                      Customer 3:
 Spends $12.50           Spends $22 per                   Spend $15 per 
 per trip, visits       trip and, visits 4x              trip and, visits 5x 
     3x Qtr                   Month                             Qtr

                                                                                                 ExtraCare/coupon
                                                                                                        John Smith
   Offer Configuration and Cadence determined
    quarterly by mining millions of data Points:                                       Save $4 on any $20 purchase.
                                                                                                        (UP TO $4.00 VALUE)

           Category           Past     Profit              Rx
           Shopped
                    Demos
                              Offers   Level
                                                Promos
                                                          Freq                                  EXPIRES 11/4/2011
                                                                                       ExtraCare® Card #*******8047
                                                                                            *EXTRACARE CARD MUST BE PRESENTED TO GET THESE
                             Customer 2:
  Customer 1                                              Customer 3:                  SAVINGS. EXCLUDES TOBACCO, ALCOHOL, AND PRESCRIPTIONS. NO
                            Matched up to                                             CASH BACK. TAX CHARGED ON PRE-COUPON PRICE WHERE REQUIRED.
Matched up to                                            Matched up to                                  LIMIT ONE PER CUSTOMER.
                             $10 off $50                                                                                                3363805547
$3 off $15 offer                                         $4 off $20 offer
                                offer




                                                                                                                                                            6
9/30/2011




            Evolution of POS Program
Ongoing innovation and refinement has ensured lasting program success
                                                                                                                             Step 5
                                                                                                                                         
                                                                                           Step 4                         Optimize
                                                                                                                          Optimization
                                                     Step 3
                                                                                         Innovate                           System
                          Step 2                    Refine                              Lift Modeling
                                                 Response,
   Step 1            Target and                   Modeling,
                                                Differentiation
                      Execute
Build Fact                R, F, M
  Base
Test and Learn                                                We were here


    This could not be achieved without a commitment to test and learn




      The Catalyst: Declining ROI
 Response model targeting was beginning to be unprofitable – need for
            investigating drivers and potential solutions

                                                Campaign ROI
              20%

              15%

              10%

               5%

               0%
                                                Campaign   Incremental     Discount /        Net           Net
                                                  Sales     Sales (Lift)   Coupons       Incremental   Incremental
              -5%                                                                           Sales         Margin



             -10%

             -15%
                     Q1        Q2          Q3          Q4              Q1               Q2             Q3            Q4

            Key Question: Who drives incremental sales?
                                    2009                                                     2010




                    Modeling Approach
 Explore a wide range of variables to capture key customer
  behaviors
       -    Promotion sensitivity and history, brand preferences, loyalty, demos

 Create multiple predictive models for each step in the
  coupon redemption process
                 Print                              Redeem                                                   Spend



            visit, purchase                     visit, purchase                                           basket size

 Estimate a wide range of success metrics
 Optimize offer assignment, unique to each customer, given
  predicted behavior in response to each offer scenario




                                                                                                                                                    7
9/30/2011




                      Program Structure
 The final solution involves multiple models that capture the profit dynamics

                           Econometric                     Business
     Data Inputs                                                                    Optimization
                           Modeling                        Metrics

        Shopping                                                Sales
        Behavior /
                                                                                       No Offer
         Loyalty
         Price &
       Promotional
                                                            Sales Lift                $3 off $10
        Sensitivity
                                Predictive
      Coupon Usage               Models                         Cost                  $4 off $20
                                   (14)
        Category                                                Margin
                                                                                      $5 off $25
        Patterns                                                 Lift
                               Qualifier (1)
                               Redemption (8)
        Customer               Sales (5)
      Demographics                                               ROI                 $10 off $50




   Recommended Target Changes
Significant changes to targeting and offer assignment were recommended




 Old Target                    Drop                 Keep           Add                    New Target


                                                                                 Higher incremental
                                                                                  sales
                                                 Don’t Target
                                                                                 Stronger ROI




                                      The Test
       A test was implemented to compare the new approach to the old


                                        Eligible Customers




                                                                         New Strategy - Lift
           Existing Strategy -
                                                                           Modeling and
         Response Model Based
                                                                         Optimization Based




                Control Group                                               Control Group




                                                                                                              8
9/30/2011




                         Test Results
                                    It didn’t work!

                 Incremental Sales and Profit Per Customer
                       Incremental Sales       Incremental Profit




                   Existing Model                  New Model


                      $3 and $10 coupon worked well
                      $4 coupon drove negative results
                      Identified disconnect in success metrics




 If At First You Don’t Succeed…

 Granted another opportunity to “split” the eligible
  population in a head-to-head test

 Refined and implemented models using more recent data

 Tested alternative approaches
         - Model Based Targeting and Offer Assignment
         - “Strategy-Based” approach – art plus science




               Art + Science Wins!
                   Incremental Sales and Profit Per Customer

                        Incremental Sales     Incremental Profit




             Existing Model    New - Model Based New - Strategy Based



   ROI            4%                     15%                       16%




                                                                                9
9/30/2011




                Lessons Learned

 Lift Modeling is not straight forward – be open to testing
      - Most challenging: predicting incremental sales dollars and ROI
      - There are several alternative modeling approaches

 Focus on the right metrics, and be consistent with
  corporate KPIs


 Optimization is likely to require human intervention – solid
  business understanding and good intuition




                    In Conclusion

 Leverage as much data and customer insight as you can
  to maximize

 The next generation of marketing analytics is moving
  beyond traditional response modeling
      -   Understand your success metrics and use analytic tools that
          focus directly on those metrics
      -   Be aware of the challenges


 The best solution requires a balance of technique and
  business knowledge




                  Getting Started

 Set up randomized testing with appropriate treated /
  control quantities

 Capture historical campaign data
      - Customer characteristics at time of execution
      - Response results

 Ensure operational system infrastructure is in place

 Identify resource to develop solution




                                                                               10
9/30/2011




                           Thank You!

        Marcy Riordan                     Michael Parduhn

     ANALYTIC VISION. BUSINESS IMPACT.




For more information contact: mriordan@iknowtion.com (781) 494-9989




                                                                            11

More Related Content

PDF
The impact and benefits of improved customer targeting
PDF
PDF
Building The Perfect Offer
PDF
HCLT Whitepaper: Road to Precision Retailing
PDF
dianomi adtech London presentation 2011
PDF
Path To Revolutionary Merchandising And Marketing Decisions
PPT
Big Data, Bigger Campaigns: Using IBM’s Unica and Netezza Platforms to Increa...
PDF
Prove Your Advantage: TCO Sales and Marketing Tools
The impact and benefits of improved customer targeting
Building The Perfect Offer
HCLT Whitepaper: Road to Precision Retailing
dianomi adtech London presentation 2011
Path To Revolutionary Merchandising And Marketing Decisions
Big Data, Bigger Campaigns: Using IBM’s Unica and Netezza Platforms to Increa...
Prove Your Advantage: TCO Sales and Marketing Tools

What's hot (20)

PDF
Mastering the Art of Relentless Marketing
PDF
Customer Intelligence & Analytics - Part IV: An Evolutionary Process: Moving ...
PDF
2011 May10 Sma Panel On Sales Productivity
PPT
Frost And Sullivan 2009
PPSX
GRS Market Research
PDF
Customer relationship management the emperor has no clothes
PPSX
The Business Mirror Sample Report
PDF
Chap05 The Communication Process
PDF
Product variety and its cost to your company
PDF
Product variety and Sales enablement
PPTX
Distil Info - Platform solutions Retail
PPTX
Sales Management - Capturing Voice of the Customer
PDF
Emnos Insight Portal - Product Folder
PPTX
The Changing Face of B2B Sales
PDF
iMetrics 2012. Uta Hermanns. Customer Journey Tracking. Elevate Multi-Channel...
PDF
Crm strategy contributes to business value in tough times
PDF
Flevy.com - Retail Gross Profit Recovery
PDF
Shopper centric
PPTX
Faster and Better: A 360 Approach to Customer Profitability
PPTX
Driving a Culture of Profitability into Your Sales Organization
Mastering the Art of Relentless Marketing
Customer Intelligence & Analytics - Part IV: An Evolutionary Process: Moving ...
2011 May10 Sma Panel On Sales Productivity
Frost And Sullivan 2009
GRS Market Research
Customer relationship management the emperor has no clothes
The Business Mirror Sample Report
Chap05 The Communication Process
Product variety and its cost to your company
Product variety and Sales enablement
Distil Info - Platform solutions Retail
Sales Management - Capturing Voice of the Customer
Emnos Insight Portal - Product Folder
The Changing Face of B2B Sales
iMetrics 2012. Uta Hermanns. Customer Journey Tracking. Elevate Multi-Channel...
Crm strategy contributes to business value in tough times
Flevy.com - Retail Gross Profit Recovery
Shopper centric
Faster and Better: A 360 Approach to Customer Profitability
Driving a Culture of Profitability into Your Sales Organization
Ad

Viewers also liked (16)

PDF
Notes Version: Powerhouse Panel Using Data to Improve Offers, Product Lines &...
PDF
Notes Version The Paradigm From Sales To Profits Using Optimization Analytics
PPTX
From Volcano Ash to Customer Satisfaction
PDF
Notes Version: Loyalty Leaders Tell All Using Data to Create Relevance
PDF
Beyond the Click: Understanding the Full Value of Email to Your Business
PDF
Customer Intelligence & Analytics - Part I
PPT
Loyalty Leaders Tell All: Using Data to Create Relevance
PPSX
Content Is Still King: Providing Ongoing Value to Customers
PPS
B2B Marketing Part 4: 50 tips to power up your efforts
PDF
Notes Version: Sustainability Elements of a Responsible Request for Proposal
PPSX
Grow Your Business the Content Marketing Way
PDF
Customer Connections: Using Customer Data to Improve Email Relevance & Revenues
PDF
Notes Version: Paradigm Shift The Changing Face of Loyalty
PDF
Integrated Strategies for Customer Loyalty and Value Optimization
PPTX
What Color is Response? An Inside Look at How Color Improves Your Sales
PDF
35+ Optimization Techniques You Can Make in Your Google AdWords Account
Notes Version: Powerhouse Panel Using Data to Improve Offers, Product Lines &...
Notes Version The Paradigm From Sales To Profits Using Optimization Analytics
From Volcano Ash to Customer Satisfaction
Notes Version: Loyalty Leaders Tell All Using Data to Create Relevance
Beyond the Click: Understanding the Full Value of Email to Your Business
Customer Intelligence & Analytics - Part I
Loyalty Leaders Tell All: Using Data to Create Relevance
Content Is Still King: Providing Ongoing Value to Customers
B2B Marketing Part 4: 50 tips to power up your efforts
Notes Version: Sustainability Elements of a Responsible Request for Proposal
Grow Your Business the Content Marketing Way
Customer Connections: Using Customer Data to Improve Email Relevance & Revenues
Notes Version: Paradigm Shift The Changing Face of Loyalty
Integrated Strategies for Customer Loyalty and Value Optimization
What Color is Response? An Inside Look at How Color Improves Your Sales
35+ Optimization Techniques You Can Make in Your Google AdWords Account
Ad

Similar to Notes Version: The Paradigm From Sales To Profits Using Optimization Analytics (20)

PPTX
Data Journey to Buyerlytics
PDF
Analytics cross-selling-retail-banking
PDF
Marketing ROI Workshop
PDF
eBook: Channel Incentives
PDF
Predicting Customer Behavior - An Introduction to iSky
PDF
Return On Marketing Investment
PPTX
Sales management in it
PDF
Dsa brand activation model how to measure your activation performance
PPTX
Musigmasadlerfeb2012
PPTX
Innerworkings Pitch - Think Small to Get Big 3-4-13
PDF
Data Driven IMC English Ver
PDF
IBM Transforming Customer Relationships Through Predictive Analytics
PDF
Sigma B2 B Analytics Intro 4 10
PDF
Customer segmentation
PDF
Dynamic Relationship Charging for Retail Banking
 
PDF
Loyalty Marketing Masterclass Singapore July 2010
PPT
Understanding user behaviour in the omni channel world- jwt
PDF
ICON_IMJ conference2012 chapter2_マーケティング投資配分最適化
PDF
Retaining & nurturing customers (Igniter breakfast seminar, 2008)
PDF
Chap02 The Role Of Imc In The Marketing Process
Data Journey to Buyerlytics
Analytics cross-selling-retail-banking
Marketing ROI Workshop
eBook: Channel Incentives
Predicting Customer Behavior - An Introduction to iSky
Return On Marketing Investment
Sales management in it
Dsa brand activation model how to measure your activation performance
Musigmasadlerfeb2012
Innerworkings Pitch - Think Small to Get Big 3-4-13
Data Driven IMC English Ver
IBM Transforming Customer Relationships Through Predictive Analytics
Sigma B2 B Analytics Intro 4 10
Customer segmentation
Dynamic Relationship Charging for Retail Banking
 
Loyalty Marketing Masterclass Singapore July 2010
Understanding user behaviour in the omni channel world- jwt
ICON_IMJ conference2012 chapter2_マーケティング投資配分最適化
Retaining & nurturing customers (Igniter breakfast seminar, 2008)
Chap02 The Role Of Imc In The Marketing Process

More from Vivastream (20)

PDF
Exchange Solutions Datasheet_Ecommerce
PDF
Exchange Solutions Datasheet_Customer Engagement Roadmap
TXT
PDF
PPT
PPTX
Jeeva jessf
PDF
Vivastream Poster
PDF
Vivastream Poster
PDF
PDF
Breaking Up is Hard to Do: Small Businesses’ Love Affair with Checks
PDF
EY Smart Commerce Report
PDF
EY Global Consumer Banking Survey 2014
PDF
EY Global Consumer Banking Survey
PDF
Serano
PDF
Accura XV
PDF
Automation for RDC and Mobile
PDF
Healthcare Payments Automation Center
PDF
Next Generation Recognition Solutions
PDF
Automation Services
PDF
Company Overview
Exchange Solutions Datasheet_Ecommerce
Exchange Solutions Datasheet_Customer Engagement Roadmap
Jeeva jessf
Vivastream Poster
Vivastream Poster
Breaking Up is Hard to Do: Small Businesses’ Love Affair with Checks
EY Smart Commerce Report
EY Global Consumer Banking Survey 2014
EY Global Consumer Banking Survey
Serano
Accura XV
Automation for RDC and Mobile
Healthcare Payments Automation Center
Next Generation Recognition Solutions
Automation Services
Company Overview

Notes Version: The Paradigm From Sales To Profits Using Optimization Analytics

  • 1. 9/30/2011 What to Expect from This Session  Explore the fundamentals and benefits of Lift Modeling and Constraint-Based Optimization Analytics  Learn how CVS Caremark has utilized their POS channel to deliver relevant offers and improve ROI - Behind-the-scenes analytic engine that drives differentiated offers to unique types of customers - Program evolution - Lessons learned Introductions 1
  • 2. 9/30/2011 Who We Are  Consulting firm specializing in information-based marketing and customer intelligence  Founded in 1999, with headquarters in Burlington, MA  Business problem solvers, skilled in analytics  Broad, practical experience across multiple industries  Outcomes driven - strive to deliver work that produces significant bottom line impact What We Do Customer Intelligence and Reporting Support Business Decision Multi-media Making / Processes Predictive Marketing Analytics and Effectiveness Scoring Increase Business Growth and Profitability Campaign Customer Planning and Acquisition and Measurement Development Optimize Marketing Loyalty Program Spending Strategy CVS  What does CVS stand for? a. Consumer Value Stores b. Caroline, Vanessa, Sophie c. Convenience, Value, Service d. Computer Vision Syndrome 2
  • 3. 9/30/2011 CVS  Nation’s largest retail pharmacy chain  Recently opened 7,300th store  CVS fills more than 1 out of every 7 prescriptions in America  ExtraCare is the largest and most successful retail loyalty program in the country ExtraCare Loyalty Program (brief video) Do you have your ExtraCare Card?  68 MM active ExtraCare cardholders  70% of CVS transactions are on the card  ExtraCare customers visit the store every month, which is twice as often as the typical non-ExtraCare shopper  Every second, 32 ExtraCare cards are scanned in the US  The very next day, these transactions can be transformed into customer insights 3
  • 4. 9/30/2011 Lift Modeling and Optimization Incremental Sales Potential “Best” customers are not always the same ones who drive campaign profitability Best Customers - Top Sales Promotion Sensitive, High Incremental Sales Limit or Exclude from New Population - Discounting Strategy Test Offer Levels Response Model Lift Model Best Offers Lift Modeling Focus directly on the consumer behavior that drives ROI  Predicts incremental response rather than total response  Model score provides a mechanism to target customer with the highest likelihood to deliver incremental sales  The challenge: incremental sales is an aggregate measure, while modeling and targeting must be done at the individual level 4
  • 5. 9/30/2011 Offer Optimization Engine Objective: Determine the optimal offer assignment for each customer given certain goals and constraints  Automated approach to offer assignment; allows quick evaluation of numerous scenarios  Optimization targets: maximum lift sales, program revenue, units, ROI, margin impact; minimum incentive cost  Optimization Constraints: budget, incentive cost, ROI hurdle, total sales  Flexibility is a key benefit Optimization in Action How do you do it? Start by asking: are you leveraging everything you know about your customer? Shopping Behavior Promotion Category Breadth History / Channel Preference Basket Profiles Digital Behavior Tenure Payment Type Employees / Demographics Non-Employees 5
  • 6. 9/30/2011 POS Marketing at CVS POS Coupons - Overview Sophisticated targeting engine drives delivery of POS Coupons  Printed at Bottom of Cash Register Receipt  Primary Objective: Drive return visits  Customer data (what they buy, who they are) drives offer assignment  Offers: Product-Specific or “Open-Ended”  Open-Ended Offers - Discount on any non-Rx item in the store - Automated trigger program - Redeemable on next visit - Broader offers drive most sales lift and ROI - Largest CVS-funded incentive program CVS Approach – Trigger Offers Customized and Dynamic Targeted Offers Illustrative Customer 1: Customer 2: Customer 3: Spends $12.50  Spends $22 per  Spend $15 per  per trip, visits  trip and, visits 4x  trip and, visits 5x  3x Qtr Month Qtr ExtraCare/coupon John Smith Offer Configuration and Cadence determined quarterly by mining millions of data Points: Save $4 on any $20 purchase. (UP TO $4.00 VALUE) Category Past  Profit Rx Shopped Demos Offers Level Promos Freq EXPIRES 11/4/2011 ExtraCare® Card #*******8047 *EXTRACARE CARD MUST BE PRESENTED TO GET THESE Customer 2: Customer 1 Customer 3: SAVINGS. EXCLUDES TOBACCO, ALCOHOL, AND PRESCRIPTIONS. NO Matched up to  CASH BACK. TAX CHARGED ON PRE-COUPON PRICE WHERE REQUIRED. Matched up to  Matched up to  LIMIT ONE PER CUSTOMER. $10 off $50  3363805547 $3 off $15 offer $4 off $20 offer offer 6
  • 7. 9/30/2011 Evolution of POS Program Ongoing innovation and refinement has ensured lasting program success Step 5  Step 4 Optimize Optimization Step 3 Innovate System Step 2 Refine Lift Modeling Response, Step 1 Target and Modeling, Differentiation Execute Build Fact R, F, M Base Test and Learn We were here This could not be achieved without a commitment to test and learn The Catalyst: Declining ROI Response model targeting was beginning to be unprofitable – need for investigating drivers and potential solutions Campaign ROI 20% 15% 10% 5% 0% Campaign Incremental Discount / Net Net Sales Sales (Lift) Coupons Incremental Incremental -5% Sales Margin -10% -15% Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Key Question: Who drives incremental sales? 2009 2010 Modeling Approach  Explore a wide range of variables to capture key customer behaviors - Promotion sensitivity and history, brand preferences, loyalty, demos  Create multiple predictive models for each step in the coupon redemption process Print Redeem Spend visit, purchase visit, purchase basket size  Estimate a wide range of success metrics  Optimize offer assignment, unique to each customer, given predicted behavior in response to each offer scenario 7
  • 8. 9/30/2011 Program Structure The final solution involves multiple models that capture the profit dynamics Econometric Business Data Inputs Optimization Modeling Metrics Shopping Sales Behavior / No Offer Loyalty Price & Promotional Sales Lift $3 off $10 Sensitivity Predictive Coupon Usage Models Cost $4 off $20 (14) Category Margin $5 off $25 Patterns Lift  Qualifier (1)  Redemption (8) Customer  Sales (5) Demographics ROI $10 off $50 Recommended Target Changes Significant changes to targeting and offer assignment were recommended Old Target Drop Keep Add New Target  Higher incremental sales Don’t Target  Stronger ROI The Test A test was implemented to compare the new approach to the old Eligible Customers New Strategy - Lift Existing Strategy - Modeling and Response Model Based Optimization Based Control Group Control Group 8
  • 9. 9/30/2011 Test Results It didn’t work! Incremental Sales and Profit Per Customer Incremental Sales Incremental Profit Existing Model New Model  $3 and $10 coupon worked well  $4 coupon drove negative results  Identified disconnect in success metrics If At First You Don’t Succeed…  Granted another opportunity to “split” the eligible population in a head-to-head test  Refined and implemented models using more recent data  Tested alternative approaches - Model Based Targeting and Offer Assignment - “Strategy-Based” approach – art plus science Art + Science Wins! Incremental Sales and Profit Per Customer Incremental Sales Incremental Profit Existing Model New - Model Based New - Strategy Based ROI 4% 15% 16% 9
  • 10. 9/30/2011 Lessons Learned  Lift Modeling is not straight forward – be open to testing - Most challenging: predicting incremental sales dollars and ROI - There are several alternative modeling approaches  Focus on the right metrics, and be consistent with corporate KPIs  Optimization is likely to require human intervention – solid business understanding and good intuition In Conclusion  Leverage as much data and customer insight as you can to maximize  The next generation of marketing analytics is moving beyond traditional response modeling - Understand your success metrics and use analytic tools that focus directly on those metrics - Be aware of the challenges  The best solution requires a balance of technique and business knowledge Getting Started  Set up randomized testing with appropriate treated / control quantities  Capture historical campaign data - Customer characteristics at time of execution - Response results  Ensure operational system infrastructure is in place  Identify resource to develop solution 10
  • 11. 9/30/2011 Thank You! Marcy Riordan Michael Parduhn ANALYTIC VISION. BUSINESS IMPACT. For more information contact: mriordan@iknowtion.com (781) 494-9989 11