Applying The Science Of Measurement To The Art Of Advertising 1 March 2010 Slideshare

Editor's Notes

• #8: The real/ultimate death of mass media 500 channels on TV & less influence of TV networks Higher TV rates for fewer viewers Higher Newspaper rates, lower readership Printed newspapers will be extinct by 2014 – (Davos 2007) Radio going satellite (Sirius, XM) Effects of time shifting and on-demand (TIVO, DVR’s, Digital, etc.) Messages delivered or offered to smaller segments Nirvana is segments of one… But they constantly change Understand the media consumption habits of your customers How do they receive, process, transmit and circulate communications? How does it change with demographics, psychographics, geography, timing and as new media and channels are created? Note the 3 C’s of 1960’s advertising: Creative, Cigarettes and Canadian Club Whiskey
• #9: The quid pro quo among marketers, channels & consumers is gone Marketers are resistant to support broad media & channels Consumers unwilling to pay for information from media Multichannel marketing is now the rule Broadcast, print, direct mail, email, web… None stand alone Purchase decisions are collaborative, not linear Be with customers on their journeys, nurturing them along the way Brand marketing is no longer about just changing perceptions, it’s about changing behavior Moved from exposure to involvement and engagement Brand & response are interwoven
• #13: The last couple of years have seen a number of efforts by marketing professionals and academics to summarize marketing’s performance in one metric.  Reicheld’s Net Promoter Score, Peppers and Rogers’ Return On Customer and Doyle’s Increase in Shareholder Value are examples of such efforts.  While some of these metrics can be useful performance indicators, they will never be able to explain the workings of marketing in its entirety. 
• #14: 33
• #17: 1. Business contribution These are effectiveness measures comparing the performance of the online channel with other channels. Examples: • Online revenue contribution ($, %) – direct and indirect (i.e. transacted online and referred online from offline); • Online profit contribution ($, %) – Profit contribution to the company in the period; • Online sales transaction contribution (n, %) – direct and indirect (% sales online may differ considerably from % revenue or profit contribution if there is a different average order value or profitability online); • Online service transaction contribution (n, %, $) – what percentage of different types of customer service occur online. Cost savings can be calculated for these also; • Online reach % – Share of online users attracted to the site in an industry category in a week or month assessed by services such as Hitwise or Netratings. Strictly, reach should be assessed through reaching customers via third party sites; • Online market share – % of online market revenue captured in comparison with offline. This is difficult to establish in some markets, dependent on industry collaboration; • Online customer migration – % of existing customers using online services. 2. Marketing outcomes: • Sales (n, $) (If relevant); • Leads (n) (registrations of other opportunities to sell); • Cost per Acquisition (CPA) – Promotional cost of obtaining a first time sale; • Other costs – Cost of good sold and average margin. Cost of service; • Average order value (Basket size); • Lifetime value ($) for different customer groups; • Average touch frequency – for example, for e-mail marketing. 3. Customer satisfaction: • Customer satisfaction and loyalty indices; • Number of comments from site and e-mail (% favorable and unfavorable); • Brand metrics (brand favorability); • Site performance and availability; • E-mail enquiry response time and accuracy. 4. Customer behavior: • Site engagement rates (Bounce rates overall and for different pages); • Site conversion rates (Visit to Sale, Visit to opportunity and Opportunity to Sale); • E-mail conversion rates (Newsletter and campaign related); • Visits involving a page view in different categories (Product pages, Service pages, Where to Buy, Contact Us). Visits / customers can be scored according to this; • Visits to purchase/Time to purchase – indication of number of visits involved with purchase; • Number of products purchased per customer; • Transaction behavior (Recency, Frequency, Monetary value analysis for different categories and customer types). RF analysis also relevant for site visits, e-mail response and different service types; • Activity or participation levels (Percentage of customer base / registrations who are actively using online service(s)). nActivated, nActive, nDormant, nLapsed, etc.; • Loyalty or churn metrics (% of customers repeat purchasing in given time, e.g. 1 year). 5. Channel promotion: • Referrer mix from different sources (direct, search, affiliates, etc); • Share of search (main terms within market); • Cost Per Click/Cost Per Contact (Visitors) average and CPM average for online/offline ads. 6. Social Media A variety of measures that quantify some very soft measurements. This is state of the art just now.
• #20: Like other forms of direct marketing testing, merchants can use either A/B or Multivariate testing to test headlines, offers body copy, offers, calls to action, images, product pages, order pages, guarantees, prices, bonus offers, and many other variables Merchants often report dramatic increases when testing different copy text, form layouts, landing pages, images and background colors. However, not all elements produce the same increase in conversions, and by looking at the results from different tests, it is possible to identify those elements that consistently produce the greatest increase in conversions for a marketer’s web site. Different customer groups react differently, which is why testing is so important.
• #21: There are two kinds of web testing for E-Commerce sites. A/B testing, also called split testing, allows the testing of two different versions of a design, copy or offer, to see which performs the best. For decades, this has been a classic method in direct mail, where companies often split their mailing lists and send out different versions of a mailing to different recipients. A/B testing is also popular on the Web, where it's easy to make your site show different page versions to different visitors. Use A/B testing when a web site gets fewer than 1,000 page views per week. It is useful when testing big things. For example, if moving complete sections provides an advantage or if changing the overall copy and design works better than the established copy and design. Multivariate testing is the most robust way to test a lot of variables at one time. This advanced statistical methodology can test the effectiveness of limitless combinations. The only limits on the number of combinations and the number of variables in a multivariate test are the amount of time it will take to get a statistically valid sample of visitors and a marketer’s computational power. This form of testing can only be done when a web site receives more than 1,000 page views per week. It can help an E-Commerce merchant optimize multiple content changes in different parts of a multiple web pages simultaneously.
• #22: A/B testing is the simplest and easiest form of web testing. Based on “the Scientific Method” of statistical analysis, it’s main limitation is that only way variable can be tested at one time. In the example, all variables remain the same with the exception of the design. So, design is the variable that was tested. Headlines, offers, images or products could be tested. Often merchants choose A/B testing when they want to test completely different pages. That’s okay, so long as what is being analyzed as the variable is the complete difference. Like other forms of direct marketing, it’s best to test the big things. If you can’t see the difference in two different combinations being tested, it’s unlikely that visitors will.
• #23: Multivariate testing is an area of high growth, as it helps websites ensure that they are getting the most from the visitors arriving at their site. Search engine optimization and pay per click advertising bring visitors to a site and have been extensively used by many markerters. Multivariate testing allows marketers to ensure that visitors arriving at their website are being shown the right offers, content and layout to convert them to sale, registration or the desired action. In the example, headlines, offers, copy, images, background colors are tested. It’s easy to get carried away with multivariate analysis. So, it’s better to test a small number of variations to insure that there are at least 100 conversions per combination analyzed. Website visitors will vote with their clicks for which content they prefer. Multivariate testing is transparent to the visitor, and technology is capable of ensuring that each visitor is shown the same content on every visit. Both A/B and Multivariate Testing allow the market to decide a marketer’s best web page options. There is no room for guessing.
• #24: From blogs and landing pages, to official outlets on social media sites, marketers now manage a large extended web that thrives beyond the borders of the traditional web site. Great conversion optimization must leverage the dynamics of each layer — and coordinate the interaction between them. The different layers of this extended web require different methods of conversion optimization thrive in each: Core web : online applications Inner web : your traditional, navigable web site Outer web : landing pages, blogs, microsites Social web : outposts on social media sites Advertising networks : paid media on other sites For instance: The outer web is where the best practices of landing page optimization and post-click marketing shine. Users have more flexible expectations on landing pages, especially when responding to specific, targeted offers. You can engage in full-scale A/B testing of different layouts, content presentations, multi-step flows, calls to action, and behavioral segmentation choices — almost anything you can imagine.
• #33: Elusiveness continues to prevail however. The study found that the exact impact of social media tactics evade the grasp of CMOs. - 53% are unsure about their return on Twitter -50% are unable to assess the value of LinkedIn or industry blogs More specifically however, roughly 15% believe there is no ROI associated with Twitter and just over 10% cannot glean ROI from LinkedIn or Facebook. I believe this is the direct result of not tying activity to an end game, the ability to know what it is we want to measure before we engage. Doing so, allows us to define a strategy and a tactical plan to support activity that helps us reach our goals and objectives.
• #36: Social Engagement Index (SEI) - The SEI is a proxy for a brand social reach and is calculated by weighting the raw number of conversations by the reach of its participants.  The raw score is then calibrated into an index.  A score of 100 is the base brand score.  Anything above this indicates a greater net reach of social conversations compared to the average brand. Social Sentiment Engagement Index (SSEI) - The SSEI is a composite that combines measures of both engagement and sentiment.  We calculate engagement by measuring the raw number of social conversations factored upon the reach per conversation participant.  We then apply a function that accounts for the sentiment of positive and negative comments.  Finally we calibrate this into an index based upon 100 point brand score.  Anything above this indicates a greater net amount of positive engagements, while a score less than indicates more negative.  The further away from 100 a score falls the more intense the sentiment.
• #37: Social Sentiment Engagement Index (SSEI) - The SSEI is a composite that combines measures of both engagement and sentiment.  We calculate engagement by measuring the raw number of social conversations factored upon the reach per conversation participant.  We then apply a function that accounts for the sentiment of positive and negative comments.  Finally we calibrate this into an index based upon 100 point brand score.  Anything above this indicates a greater net amount of positive engagements, while a score less than indicates more negative.  The further away from 100 a score falls the more intense the sentiment.
• #38: Cost Per Social Impression (CPSM) - How much would you be willing to pay for a Tweet?  or a new fan or follower?  Clearly social media is in its infancy as a cross-channel media measurement tool, but already it's clear the social space is an excellent medium for measurement as it reflects and resonates brands spend in other channels.  In an effort to gauge how successful the brands were at converting their Super Bowl media spend to social engagement we've taken the potential reach of the conversation, using a popularity score as a multiplier, and divided it by the media spend.  In looking at a brands CPSM, the closer to $0.00 the better.