Talk to me, Not at me: Testing for a Better Customer Experience

Editor's Notes

• #6: I’m assuming that everyone has joined this webinar bc they have made the leap into the acceptance phase! Now more than ever we have the ability as digital marketers to see exactly what is working, what isnt and etc.
• #7: And everyone is also here because every company’s marketers regardless of whether it’s B2C or B2b, small or large, have the same question: How do I acquire new customers faster? How do I grow their lifetime value? And, how do I convert as many of them as possible to become advocates? How do I build brand advocates who in turn can influence new customers? In other words, create this virtuous cycle of customer success.
• #8: But, today’s mobile, social, digital world has made this really hard to create this virtuous cycle of customer success. First of all, your buyers are more empowered than ever before. They have 2900 messages per day that are vying for their attention so there is are lots of places that they can get information. And armed with this info, they are forming opinions and drawing conclusions well before they choose to interact with your brand. In fact if you look at some data points, from folks like Forrester, they say that anywhere from 66-90% of a buyers journey is self-directed before they interact with you. When they are interacting with your brand, at every stage of interaction, 87% percent of them demand something that is a meaningful, contextual, relevant experience (based on a recent report from Edelman). Given that there is lots of noise, the customer is self-directing their journey and the customer wants a personal interaction at every point of their journey, it’s now up to marketing, more than any other business function, to become the steward of the customer journey. In the past, sales was empowered and customer service interacted with the customers a lot, but given these trends, marketing is taking the lead, building a bond with customers and becoming the steward of the customer journey.
• #9: Because of the abundance of ads your audience sees on a day to day basis and the fact that such a small percentage of them will actually convert once they get to your website, it is obvious there needs to be a way to optimize! This is where A/B testing comes in – with A/B testing you can learn how to optimize the experience for each person who is seeing your ad or coming to your site and effectively engage with them to increase the chances of them converting. And the fact is – most marketers do not actually take the time to A/B test. Yes, it does take time to think of a test, run a test and analyze the results, but the overall results can be incredibly effective.
• #10: The role of testing is to keep you marketing messages out of the black hole if it’s relevant, they will come But how do you know if it’s relevant? How can you continuously learn about your customer Look by channel or work cross-functionally to look at how a customer engages with your brand everywhere You test out what you think you know about your customers
• #12: The last one these – in one subject line asked me if I was a smoker and pregnant – I delete immediately. By testing the different content themes by segment, they can start to deliver me better content and a customer experience that is relevant to me…
• #13: How do you use testing to drive your company goals? Well, start first with what are your company goals? Okay, so the first step of staying out of the mental black hole is to identify what type of test will correlate with your company goals. When thinking about what you should test, you should start by thinking about what will achieve the highest impact for your business. Or more simply – how you measure success online – is it online purchases, is it newsletter sign ups, is it lead generation or even just general brand awareness. You need to establish your objective ahead of time so you can decide what type of test you will run. And as a side note, your goal can include more than one of these examples. For example, my goals usually revolved around both lead generation for top of the funnel programs and conversion for middle of the funnel programs. + do you need to look cross functionally in order to test across channels?
• #14: Once you have identified the objectives you should be optimizing and testing, you can start thinking about where and what to test. For where to test, you will want to choose high-impact sites, pages or emails. An effective test is one that will make a difference to your business, so you will want to make sure your test is worth your time and you will learn from it. Next on what to test. The best tests to start with are ones that will reduce clutter. This can be an easy fix by just removing an additional sentence, for example. Clutter can cause confusion and reduce engagement so this is a good place to start. Also, you want to think about what is your goal or call to action and what will be a factor that will increase your audience’s focus to your goal. For example, if you want someone to click on a button to download an ebook, you can try making the button bigger, changing the font or color to see if that increases focus to your overall call to action.
• #15: Once you have all of your results compiled, it is time to start using them to increase engagement. What can you do with your results to increase engagement is simple. By the test you run, you can learn how your audience interacts with you – maybe you discovered that your audience tends to open more emails coming at 9am rather than at 12pm. You can use this data to enhance your communication results. Another way to increase engagement is to really start to understand not just how your audience interacts with you but how segments within your audience interacts with you. For example, we run separate tests here at Marketo based on company size. We found we get different results for enterprise and smb companies. You can also segment by job title, location or other segments.
• #16: many of the brands we have on this call today have their own version of this vision, it’s a great one. The key point that I’d like to make during the next few minutes is this: our customers want a personalized experience across all touchpoints, and paramount to this is the notion of testing. In a nutshell, personalization without testing is guessing!
• #17: National Express is the European version of a Greyhound Bus Line. They are a client of ours in Europe, and as one of their first tests, they experimented with the following scenario: On their homepage, they have a booking section -- The executive team wanted to better understand whether the simple change of a button mattered to booking conversion, so they created four variants of the CTA button. Let’s take a virtual show of hands for each variant – A, B, C, and D – which variant do you think drove the highest booking conversion?.... The point is, intuition often doesn’t work, and in the charlotte vision I showed a minute ago, yes all touchpoints are well coordinated based on charlotte’s shopping life cycle, but without testing, charlotte’s experience with any one of those touchpoints isn’t necessarily optimized.
• #22: In the national express experiment, we saw an example of a basic A/B/n test. Chances are, we each picked a different variant, as such, our visitors’ based on any set of factors might respond better or worse to one CTA or another. Basic A/B tests are a simple and quick way to better understand the tactical preferences of our different visitors. Of course, learnings from these tests can and should be extended to e-mail and other channels.
• #25: So too, for the more elaborate on-site campaigns, multivariate testing is integral – that’s the ability to determine in a statistically significant want the optimal combination of web elements across multiple pages, for each and every one of your visitors.
• #26: And of course, this is part art, part science – test design matters! In working with clients on the design and implementation side, I’ve seen time and time again that having a unique hypothesis behind each variant is a best practice in actually discovering meaningful and unique preferences among different visitor groups. In this simple example, we see food category as the distinction among variants – its understandable and likely that different visitors will have different preferences. Logically, your sweet-tooth will respond to a tempting cupcake image, and your steak-lover will respond to a sizzling steak. BUT I wouldn’t just assume. I’d test these hypotheses so that we can actually quantify how important these distinctions are to our visitors. Once we’ve proven these hypotheses at a high level, we’d drill down. Perhaps some sweet-tooths like cupcakes and others prefer pie! So there would be multiple variants of each recipe category, to ultimately understand preferences at a far more granular level. To bring this full loop, picture this: prospect X receives an e-mail inviting them to check out sweet recipes. They click the link and are shown pie, but leave the site without further engagement…. Lets follow up with her in a second e-mail and invite her back, and knowing she wasn’t thrilled with our pie-themed experience, let’s show her cake and cookie recipes!
• #27: Finally, before an example, I’d like to take a minute to talk abt a maturity curve we like to use in guiding our clients from a starting point in personalization to true 1:1 Omnicnnel personalization….
• #29: BMI Baby is a European client. They are a short-haul, low-cost airline that was acquired by British Airways. In this example, BMI ran a website test on the back of an e-mail campaign, where visitors generated into the campaign from having clicked on a link inviting them to “Get Inspired.”
• #31: The info we had on each visitor was then combined with any other data we had on them, and these profiles were consumed by our platform, and based on real-time testing, each visitor was presented with the optimal destination.
• #32: The results were great! Through this process campaign conversion increased by 36%! Additionally, a number of attributes were identified as having influence over a visitor’s best offer. This all happened real time, for thousands and thousands of BMI prospects and customers. No small feat!