SlideShare a Scribd company logo
Next-Generation Campaign
  Section Title
 Validation & Optimization:
     A Sprint Case Study



     VictorSilva   Andrea Molette
In 2007, comScore’s First Post-Buy Analysis Across 8 Digital US
    Campaigns Showed Execution Left a Lot to be Desired …

      70%
                          Percent of Ad Impressions for 8 Campaigns
      60%


      50%


      40%


      30%           61%

      20%


      10%                                               19%
                                        8%                              12%
      0%
            In US But Not Target     Hit Target       Hit Target      Outside US
                                   Frequency >=5    Frequency <=4
Branding advertisers on TV are accustomed to audience
       guarantees and expect the same in digital
      Accuracy of cookie-based digital plan delivery is
                        problematic:
       Cookie Deletion                                Cookie Proliferation




           X             Cookies Are Not People




                                       Source: comScore 2011
Some Things We’ve Learned About Digital
         Media Plan Delivery
• The negative impact of cookie deletion
   – Cookie deletion inflates ad frequency and deflates ad reach
     by as much as a factor of 2.5X
• Targeting accuracy using cookies:
   –   70% for 1 demo (e.g. women)
   –   48% for 2 demos (e.g. women age 18-34)
   –   11% for 3 demos (e.g. women age 18-34 with kids)
   –   36% for behavioral targeting (e.g. people visiting travel sites)
Cookies Can’t Accurately Identify Who is Using a Computer at any
           Given Point in Time due to Multiple Users


         Over 64% of home users share a computer with other users




                          3+ users         1 user

                          32% of the36%
                               30%
                          time, someone other
                            than the Facebook
                            logged-in person is
                             actually using the
                                  2 users
                                 computer
                                 32%
Cookie Deletion is a Global Reality
             …and a Global Challenge
           Accurately counting reach with cookies is not possible,
yet is currently the method used in most ad servers and analytics systems

                                          Ad Server Cookies


                                Percent of         Average # of cookies
                Country      computers deleting   per computer for same
                                                        campaign


       Australia                    37%                       5.7
       Brazil                       40%                       6.6
       U.K.                         35%                       5.9
       U.S.                         35%                       5.4
Arguably the Most Important Digital Advertising Initiative To Date:
      Making Measurement Make Sense (3MS) Mission




    Reduce costs of doing business due to complexity of
     digital advertising ecosystem
    ‘Single Tag’ solution to reduce complexity
    Improve reporting of ad exposure
    Bolster confidence that ads delivered are actually
     visible
What is a vGRP?
• validated Gross Rating Points (GRP) based on validated
  impressions delivered to the Total Census Population for selected
  Geographic Market, or 100 * % Pop Reach * Average Frequency
  for the reporting period
• Comparable to GRPs used in television because it used the same
  calculations
• vGRPs must:
   –   Deliver in the target geography
   –   Not be fraudulent deliveries
   –   Be in brand safe content
   –   Be viewable
   –   Hit the target audience
vGRP can provide more accurate analysis of campaign
             effectiveness in Marketing Mix Models


       GRP: Negative                             vGRP: Positive
        Correlation                                Correlation
100                                   80   100                            80

 90                                        90
                                      70                                  70
 80                                        80
                                      60                                  60
 70                                        70                     vGRP
                              GRP     50                                  50
 60                                        60                     Sales
                              Sales
 50                                   40   50                             40

 40                                        40
                                      30                                  30
 30                                        30
                                      20                                  20
 20                                        20
                                      10                                  10
 10                                        10

  0                                   0      0                            0

        Month 1     Month 2                      Month 1   Month 2
CHANGING HOW THE WORLD SEES
    DIGITAL ADVERTISING
vCE US Charter Study:
12 Major Branded Advertisers Came Together to Lead & Learn

  18 campaigns
  2 billion impressions
  400,000 sites




                         Allstate
Charter Study replicated in Europe with similar results

  15 campaigns
  640 million impressions
  213,000 sites
Study Objective

Quantify incidence of sub-optimal ad delivery across key
                ad delivery dimensions…




… to better understand sources of waste and identify
opportunities to extract more value for all players in the online
advertising ecosystem
Importantly, all impressions in the study were
delivered in iframes, including the notoriously difficult-
          to-measure cross-domain iframes

    Directory on site   Same-domain iframe     Cross-domain iframe
                             (friendly)            (un-friendly)

         ad                   ad                         ad
         web                   web                      web
         site                  site                     site




                                          61+% of
                                       iframed ads
                                          use this
                                          method
Across all campaigns, the average in-view rate was
   69%, meaning 3 out of 10 ads weren’t seen

           Percentage of Ads In-View by Campaign
vCE Charter Study:
In-View Rates Need to Be Improved




 US                                EU
 69%                               67%
AVERAGE                           AVERAGE



Campaign In-View ad rates ranged from:
US 55% to 93% EU 64% to 72%
In-view rates by site ranged from 7% to
100%, suggesting the importance of validation across all
                         sites
   Percentage of Ads In-View by Site
Large sites scored better than long-tail sites

              Percentage of Ads Served In-View



    77%         74%                  70%
                                                 61%
    66%         63%                  61%
                                                 55%


                             US     EU
The Classic Leaderboard delivered the strongest in-view rates but
  there was significant variance across all sites with a range of
                    7% to 93% using this size
 Percent of Ads Delivered In-View by Ad Size




                                                      one potential
                                                            cause?
                                               the relationship between ad
                                                      sizes and their typical
                                                 placement on a web page
Digital Ad Economics:
     The Good Guys Aren’t Necessarily Winning

                Low correlation of In-View Rates & CPM


                                                                 R²=0.0373




An equally as weak correlation was also observed between CPM and ability to
                     hit a primary demographic target
On a campaign-by-campaign basis, one performed
flawlessly, while another delivered 15% of its impressions to
                    the wrong geography
                Percent of Ads Delivered In Geography by Campaign




   Sub-optimal geographic delivery is often a result of communication or human
                          error, and it can be remedied
                       with in-flight alerting and blocking
Of those ads delivered outside of their target, a good
portion of them landed in countries where English is
            not even the primary language
% of Ads Delivered to Geographic Market Among All Impressions Delivered Outside of N.A.
Brand safety should be of the utmost importance to advertisers.
Even one poorly delivered ad can leave a very bad impression

Percent of Campaigns with Impressions Delivered
Next to Content Deemed “Not Brand Safe”
The Above-the-Fold Myth?
Above-the-fold in-view rates ranged from 48% to 100%




                                                 source: comScore vCE charter study
Section Title
There is good news for advertisers and publishers

 Analogous to TV audience guarantees
 Eliminating unseen online inventory supply results
  in:
    More effective / efficient campaigns and less waste for
     advertisers
    More accurate metrics for market mix models
    Better proof of digital ad effectiveness
 Increased transparency/accountability means
  increased confidence in digital
Thank You!
Download the whitepaper, continue the conversation

More Related Content

ZIP
Sprint's Next Generation Campaign Validation & Optimization
PPTX
Next-Generation Campaign Section Title Validation & Optimization
PDF
Yahoo!'s An Ad is Not Just an Ad Research
PDF
PERSONAL RECOMMENDATIONS AND CONSUMER OPINIONS POSTED ONLINE ARE THE MOST TRU...
PDF
Global trust-in-advertising-2012
PDF
Dutch Customer Futures Digital 19 6 8
PDF
comScore online video 11.01.11 videoTLV
PPTX
Brother Gemalto
Sprint's Next Generation Campaign Validation & Optimization
Next-Generation Campaign Section Title Validation & Optimization
Yahoo!'s An Ad is Not Just an Ad Research
PERSONAL RECOMMENDATIONS AND CONSUMER OPINIONS POSTED ONLINE ARE THE MOST TRU...
Global trust-in-advertising-2012
Dutch Customer Futures Digital 19 6 8
comScore online video 11.01.11 videoTLV
Brother Gemalto

What's hot (19)

PDF
Doubleclick Display Benchmark Report 2009, US, 07-2010
PDF
The death of the click (с) Osnat Zaretsky
PPT
Viral Advertising IDC - English
PDF
Mobile Marketing
PDF
Brand Advocacy By Tom Doctoroff
PDF
wom marketing by WOMlabo
PDF
Research: TV media buying optimisation with YouTube
KEY
Vespillo Keynote Presentation
PPTX
1015 omma social eli goodman
PDF
Hands.mobi presentation Jan 2012
PDF
Metropolia Marketing Talks 19.11.2009
PDF
Overcoming challenges of implementing mobile audience measurement studies in ...
PDF
WapStart Mobile Ad Network 2013 EN
PDF
Humans Networking
PPTX
Bjorn Kaspring - IAB Germania - On-screen advertising has an implicit effect
PDF
Sport business webinar the monetisation of football content 120404
PPT
Media terminology
PDF
Tagga Udi Breakfast Presentation
PPSX
Changing Channels with Confidence
Doubleclick Display Benchmark Report 2009, US, 07-2010
The death of the click (с) Osnat Zaretsky
Viral Advertising IDC - English
Mobile Marketing
Brand Advocacy By Tom Doctoroff
wom marketing by WOMlabo
Research: TV media buying optimisation with YouTube
Vespillo Keynote Presentation
1015 omma social eli goodman
Hands.mobi presentation Jan 2012
Metropolia Marketing Talks 19.11.2009
Overcoming challenges of implementing mobile audience measurement studies in ...
WapStart Mobile Ad Network 2013 EN
Humans Networking
Bjorn Kaspring - IAB Germania - On-screen advertising has an implicit effect
Sport business webinar the monetisation of football content 120404
Media terminology
Tagga Udi Breakfast Presentation
Changing Channels with Confidence
Ad

Viewers also liked (8)

PDF
Mobile and Retail: How Lowe’s Develops and Leverages a Complete View of the C...
PPTX
How Can 3 Event-Triggered Systems Be Different?
PPSX
Mobile's Changing Face: The Multiscreen Impulse Economy
PDF
SAP's Excellence in Integrated Marketing: Converting Online Traffic to Leads
PDF
Using Customer Data to Improve Email Relevancy
PPSX
311380
PPTX
How Technology Can Enable Product and Process Innovation
PPT
Mobile and Retail: How Lowe’s Develops and Leverages a Complete View of the C...
How Can 3 Event-Triggered Systems Be Different?
Mobile's Changing Face: The Multiscreen Impulse Economy
SAP's Excellence in Integrated Marketing: Converting Online Traffic to Leads
Using Customer Data to Improve Email Relevancy
311380
How Technology Can Enable Product and Process Innovation
Ad

Similar to Sprint's Next Generation Campaign Validation & Optimization (20)

PDF
Putting the future_in_focus
PDF
Com score vce brandingmetrics vn-2012-11-02
PDF
Comscore vce brandingmetrics vn-2012-11-02
PPTX
Presentación de Gian Fulgoni en IAB Conecta 2012
PDF
"Let the evolution begin" @comScore
PDF
comScore Study Finds Professionally-Produced Video Content And User-Generated...
PDF
The efficiency of online advertising (Andrew Felbert - Nielsen)
PDF
Татьяна Хандурова - com_score
PDF
VIDEO AD TECHNOLOGY 101
PDF
Persuasive Potential of Consumer-Produced Content
PDF
Changing How Asia (and the World) Sees Digital Advertising
PDF
Praseed Prasad on DSPs and relevance to marketers at ad:tech Bangalore
PDF
Ipsos ASI's Top 10 Trends for 2013
PPT
Presentación de Nigel Hollis en IAB Conecta 2012
PDF
Measuring & Optimizing Reach and Resonance in Digital Advertising
PPTX
How Kellogg's Brands Work Their Digital Strategies
PPTX
Directi Case Study Contest - Relationships Matter from ISB Hyderabad
PDF
5 Digital Trends for 2013 - Dynamit
PDF
The Changing Nature of Campaign Management
PPTX
adtech SF 2012 Facebook how display works differently by appssavvy and Optimedia
Putting the future_in_focus
Com score vce brandingmetrics vn-2012-11-02
Comscore vce brandingmetrics vn-2012-11-02
Presentación de Gian Fulgoni en IAB Conecta 2012
"Let the evolution begin" @comScore
comScore Study Finds Professionally-Produced Video Content And User-Generated...
The efficiency of online advertising (Andrew Felbert - Nielsen)
Татьяна Хандурова - com_score
VIDEO AD TECHNOLOGY 101
Persuasive Potential of Consumer-Produced Content
Changing How Asia (and the World) Sees Digital Advertising
Praseed Prasad on DSPs and relevance to marketers at ad:tech Bangalore
Ipsos ASI's Top 10 Trends for 2013
Presentación de Nigel Hollis en IAB Conecta 2012
Measuring & Optimizing Reach and Resonance in Digital Advertising
How Kellogg's Brands Work Their Digital Strategies
Directi Case Study Contest - Relationships Matter from ISB Hyderabad
5 Digital Trends for 2013 - Dynamit
The Changing Nature of Campaign Management
adtech SF 2012 Facebook how display works differently by appssavvy and Optimedia

More from Vivastream (20)

PDF
Exchange Solutions Datasheet_Ecommerce
PDF
Exchange Solutions Datasheet_Customer Engagement Roadmap
TXT
PDF
PPT
PPTX
Jeeva jessf
PDF
Vivastream Poster
PDF
Vivastream Poster
PDF
PDF
Breaking Up is Hard to Do: Small Businesses’ Love Affair with Checks
PDF
EY Smart Commerce Report
PDF
EY Global Consumer Banking Survey 2014
PDF
EY Global Consumer Banking Survey
PDF
Serano
PDF
Accura XV
PDF
Automation for RDC and Mobile
PDF
Healthcare Payments Automation Center
PDF
Next Generation Recognition Solutions
PDF
Automation Services
PDF
Company Overview
Exchange Solutions Datasheet_Ecommerce
Exchange Solutions Datasheet_Customer Engagement Roadmap
Jeeva jessf
Vivastream Poster
Vivastream Poster
Breaking Up is Hard to Do: Small Businesses’ Love Affair with Checks
EY Smart Commerce Report
EY Global Consumer Banking Survey 2014
EY Global Consumer Banking Survey
Serano
Accura XV
Automation for RDC and Mobile
Healthcare Payments Automation Center
Next Generation Recognition Solutions
Automation Services
Company Overview

Sprint's Next Generation Campaign Validation & Optimization

  • 1. Next-Generation Campaign Section Title Validation & Optimization: A Sprint Case Study VictorSilva Andrea Molette
  • 2. In 2007, comScore’s First Post-Buy Analysis Across 8 Digital US Campaigns Showed Execution Left a Lot to be Desired … 70% Percent of Ad Impressions for 8 Campaigns 60% 50% 40% 30% 61% 20% 10% 19% 8% 12% 0% In US But Not Target Hit Target Hit Target Outside US Frequency >=5 Frequency <=4
  • 3. Branding advertisers on TV are accustomed to audience guarantees and expect the same in digital Accuracy of cookie-based digital plan delivery is problematic: Cookie Deletion Cookie Proliferation X Cookies Are Not People Source: comScore 2011
  • 4. Some Things We’ve Learned About Digital Media Plan Delivery • The negative impact of cookie deletion – Cookie deletion inflates ad frequency and deflates ad reach by as much as a factor of 2.5X • Targeting accuracy using cookies: – 70% for 1 demo (e.g. women) – 48% for 2 demos (e.g. women age 18-34) – 11% for 3 demos (e.g. women age 18-34 with kids) – 36% for behavioral targeting (e.g. people visiting travel sites)
  • 5. Cookies Can’t Accurately Identify Who is Using a Computer at any Given Point in Time due to Multiple Users Over 64% of home users share a computer with other users 3+ users 1 user 32% of the36% 30% time, someone other than the Facebook logged-in person is actually using the 2 users computer 32%
  • 6. Cookie Deletion is a Global Reality …and a Global Challenge Accurately counting reach with cookies is not possible, yet is currently the method used in most ad servers and analytics systems Ad Server Cookies Percent of Average # of cookies Country computers deleting per computer for same campaign Australia 37% 5.7 Brazil 40% 6.6 U.K. 35% 5.9 U.S. 35% 5.4
  • 7. Arguably the Most Important Digital Advertising Initiative To Date: Making Measurement Make Sense (3MS) Mission  Reduce costs of doing business due to complexity of digital advertising ecosystem  ‘Single Tag’ solution to reduce complexity  Improve reporting of ad exposure  Bolster confidence that ads delivered are actually visible
  • 8. What is a vGRP? • validated Gross Rating Points (GRP) based on validated impressions delivered to the Total Census Population for selected Geographic Market, or 100 * % Pop Reach * Average Frequency for the reporting period • Comparable to GRPs used in television because it used the same calculations • vGRPs must: – Deliver in the target geography – Not be fraudulent deliveries – Be in brand safe content – Be viewable – Hit the target audience
  • 9. vGRP can provide more accurate analysis of campaign effectiveness in Marketing Mix Models GRP: Negative vGRP: Positive Correlation Correlation 100 80 100 80 90 90 70 70 80 80 60 60 70 70 vGRP GRP 50 50 60 60 Sales Sales 50 40 50 40 40 40 30 30 30 30 20 20 20 20 10 10 10 10 0 0 0 0 Month 1 Month 2 Month 1 Month 2
  • 10. CHANGING HOW THE WORLD SEES DIGITAL ADVERTISING
  • 11. vCE US Charter Study: 12 Major Branded Advertisers Came Together to Lead & Learn 18 campaigns 2 billion impressions 400,000 sites Allstate
  • 12. Charter Study replicated in Europe with similar results 15 campaigns 640 million impressions 213,000 sites
  • 13. Study Objective Quantify incidence of sub-optimal ad delivery across key ad delivery dimensions… … to better understand sources of waste and identify opportunities to extract more value for all players in the online advertising ecosystem
  • 14. Importantly, all impressions in the study were delivered in iframes, including the notoriously difficult- to-measure cross-domain iframes Directory on site Same-domain iframe Cross-domain iframe (friendly) (un-friendly) ad ad ad web web web site site site 61+% of iframed ads use this method
  • 15. Across all campaigns, the average in-view rate was 69%, meaning 3 out of 10 ads weren’t seen Percentage of Ads In-View by Campaign
  • 16. vCE Charter Study: In-View Rates Need to Be Improved US EU 69% 67% AVERAGE AVERAGE Campaign In-View ad rates ranged from: US 55% to 93% EU 64% to 72%
  • 17. In-view rates by site ranged from 7% to 100%, suggesting the importance of validation across all sites Percentage of Ads In-View by Site
  • 18. Large sites scored better than long-tail sites Percentage of Ads Served In-View 77% 74% 70% 61% 66% 63% 61% 55% US EU
  • 19. The Classic Leaderboard delivered the strongest in-view rates but there was significant variance across all sites with a range of 7% to 93% using this size Percent of Ads Delivered In-View by Ad Size one potential cause? the relationship between ad sizes and their typical placement on a web page
  • 20. Digital Ad Economics: The Good Guys Aren’t Necessarily Winning Low correlation of In-View Rates & CPM R²=0.0373 An equally as weak correlation was also observed between CPM and ability to hit a primary demographic target
  • 21. On a campaign-by-campaign basis, one performed flawlessly, while another delivered 15% of its impressions to the wrong geography Percent of Ads Delivered In Geography by Campaign Sub-optimal geographic delivery is often a result of communication or human error, and it can be remedied with in-flight alerting and blocking
  • 22. Of those ads delivered outside of their target, a good portion of them landed in countries where English is not even the primary language % of Ads Delivered to Geographic Market Among All Impressions Delivered Outside of N.A.
  • 23. Brand safety should be of the utmost importance to advertisers. Even one poorly delivered ad can leave a very bad impression Percent of Campaigns with Impressions Delivered Next to Content Deemed “Not Brand Safe”
  • 24. The Above-the-Fold Myth? Above-the-fold in-view rates ranged from 48% to 100% source: comScore vCE charter study
  • 26. There is good news for advertisers and publishers  Analogous to TV audience guarantees  Eliminating unseen online inventory supply results in:  More effective / efficient campaigns and less waste for advertisers  More accurate metrics for market mix models  Better proof of digital ad effectiveness  Increased transparency/accountability means increased confidence in digital
  • 27. Thank You! Download the whitepaper, continue the conversation

Editor's Notes

  • #12: To get some market reaction, we introduced a charter program of some really large advertisers.We were delighted by the response to the program, and let me take the opportunity to say to them, Thanks you for participatingNow let us talk about what we learned.
  • #15: Prior to the introduction of vCE, there hasn’t been a proven standard technology in the marketplace that could measure all three types of iframes to get a complete read of visibilityMeasuring the first two – directly on the site iframes and friendly iframes is relatively easy, and there have been solutions for these. However, unfriendly iframes, or cross domain iframes, are hard to measure in terms of location on the screen. Unfortunately, a large percent of ads are delivered this way, meaning that missing this piece of the equation, means not covering the entire story. vCE accounts for ALL THREE providing insight into total visibility.
  • #19: Difference of in-view rates between Top 50 sites and long tail sites in their category was a full 16-percentage points in the USEuropean Results:Top 50                  66%Top 100                63%Top 500                61%501+                    55% The only minor difference with the US is that we’ve taken the full range of sites rather than just a category We had half the publisher volume we saw in the US, and a lot have very small numbers of impressions which, when visible, skew upwards the long tail visibility average.  The overall data set tells the best story and is most consistent with the original US slide.
  • #25: To clarify, being in-view has nothing to do with being above or below the fold. In fact, there are some myths the charter study helped to debunk. For some campaigns less than half of the above the fold placements where actually seen.
  • #26: On the other hand some placements below the fold where in almost 70% of the time.