SlideShare a Scribd company logo
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Executing and A/B Testing
Cross-Channel Personalization
Andy Zimmerman, Evergage Shanelle Mullin, ConversionXL
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Agenda
• The Cross-Channel Imperative
• Challenges
• Solution Requirements
• Examples
• A/B Test Process
• Prioritization
• Duration
• Results and Archiving
• Q & A
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
The Cross-Channel Imperative
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Challenge: Cross-Channel is Happening Now
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Challenge: Cross-Channel is Happening Now
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
70%
OF WALMART.COM
TRAFFIC LAST
HOLIDAY SEASON
WAS MOBILE
DEVICES.
Forrester: Your Digital Experience Technology Starts With A Customer Journey Map
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
69%
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
$1.5 Trillion retail sales generated
by cross-channel interactions
– Forrester, The Omnichannel Commerce Playbook for 2016
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
47% of bank customers engaged in
at least one cross-channel
interaction over 90 day period.
– Forrester, Design Better Cross-Channel Banking Journeys
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Top Goal is Customer Experience
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
4 Key Elements of a Great Customer Experience
4
1 Remember me.
2 Understand me.
3 Help me achieve my goals.
Surprise me. Delight me.
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Cross-Channel Personalization
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
77% of consumers have chosen,
recommended, or paid more for a
brand that provides a personalized
service or experience.
– Forrester’s North American Consumer Technographics® Brand
Compass Survey, Q3 2015
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
The Cross-Channel Challenge
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Challenge: Organizational Misalignment
– Forrester, The Omnichannel Commerce Playbook for 2016
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Challenge: Organizational Misalignment
– Forrester, The Omnichannel Commerce Playbook for 2016
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Challenge: Data is Siloed
Marketers are using SIX or more tools to collect performance data.
Marketing Analytics Tool
Website Analytics Tool
arketing Automation Tool
Operational Analytics Tool
ness Intelligence System
Sales Automation Tool
rketing or Online Surveys
rformance/Analytics Tool
CRM System
ERP System
0% 20% 40% 60% 80%
30%
42%
43%
44%
54%
54%
57%
57%
69%
79%
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Requirements for a Cross-Channel Personalization
Solution
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Track Behavior Across Channels
Website Email SearchMobile
Web App
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Synch Data with Other Systems
CRM
Marketing
Automation
Email Marketing
Data Warehouse
Data Lake
Call Center
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
One Unified Customer Profile
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Core Capabilities of Personalization Platform
Cross-channel Targeting
Algorithmic
Recommendations
Real-Time
Segmentation
A/B & MV Testing
Analysis & Attribution
Behavioral Tracking
Unified Customer
Profile
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Cross-Channel Personalization Maturity Evolution
Beginning
Intermediate Part
A
Intermediate Part
B
Advanced
Step 1 Digital Channel
2 or 3 Digital
Channels
Synch with other
systems
All channels,
including agents
Web
Mobile App
Open-time Email
Search
Web
Mobile App
Open-time Email
Search
CRM System
Email/MAP
Data Warehouse
Call Center
In-Store Kiosk
Reps/Agents
Benefit One Team Multiple Teams
Better data,
Targeting x-channel
Organization-wide
Challenge Getting Started
Cross-Team
coordination needed
Coordinate synch if
not out-of-box, APIs
Additional
Integrations
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Examples of Cross-Channel Personalization
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Ski Resort Example
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Ski Resort Example
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Ski Resort Example
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Rue La La Example
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
How to A/B Test
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Only 32% of small companies and
42% of large companies have a
structured, documented process.
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
1. Research
2. Prioritize
3. Experiment
4. Analyze and Learn
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Prioritization Problems
1. Ease and impact are too subjective
2. Too rigid.
3. Doesn’t encourage a data-driven culture
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Prioritization Fixes
1. Binary responses
2. Customizable variables
3. Ask everyone to bring data
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Grab the Template
bit.ly/pxlmodel
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Determining How Long to Test
1. Calculating sample size
2. Running for full weeks, usually two cycles
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Use the Calculator
bit.ly/evansample
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Ensuring a Representative Sample
1. Every day of the week
2. Various traffic sources
3. Ad, email, social, content schedule
4. External factors (e.g. holidays)
5. Comparison shoppers
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
On External Factors...
1. Season
2. Day of the Week
3. Press
4. PPC / SEM / SEO
5. Word of Mouth
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
On Sample Pollution...
1. Length pollution
2. Device pollution
3. Browser pollution
4. Cookie pollution
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Watch Out...
1. Regression to the mean
2. The novelty effect
3. Simultaneous testing
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Creating a Hypothesis
1. Is it designed to solve a conversion problem?
2. Is it measurable / testable?
3. Will you gain insight, win or lose?
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Skills That Help with Testing
1. HTML, CSS, JavaScript, jQuery
2. Design
3. Copywriting
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Get the Details
bit.ly/gaabresults
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
No Difference? Not So Fast...
1. Maybe the implementation sucked.
2. The variation might’ve won in a segment.
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Segmentation 101
1. Desktop vs. Mobile
2. New vs. Returning
3. Traffic Direct on the Page vs. Traffic via Internal Link
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Insights > Wins
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Why Archive?
1. To avoid repeating tests
2. To better communicate wins and insights
3. To improve the quality of future tests
4. To create an insights > wins culture
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Archive Methods
1. Iridion
2. Effective Experiments
3. GrowthHackers Projects
4. Experiment Engine
5. Testing Tool
6. Excel
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Conversion optimization training program.
evergage-webinar
WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
Questions?

More Related Content

PPTX
Implementing and Evaluating Web Accessibility
PPTX
Changing the Game: Always On Testing with Artificial Intelligence
 
PDF
[Elite Camp 2016] Martijn Scheijbeler - Optimization for Content Sites
 
PDF
What, why and how to A/B test with AI
PDF
[Elite Camp 2016] Karsten Lund - Master the Moment of Decision
 
PDF
A/B testing at Spotify
PDF
[Elite Camp 2016] Phil Nottingham - CRO with Video: Tips, Tricks and Tactics
 
PDF
[CXL Live 16] What to Test Next - Prioritizing Your Tests by Pauline Marol
 
Implementing and Evaluating Web Accessibility
Changing the Game: Always On Testing with Artificial Intelligence
 
[Elite Camp 2016] Martijn Scheijbeler - Optimization for Content Sites
 
What, why and how to A/B test with AI
[Elite Camp 2016] Karsten Lund - Master the Moment of Decision
 
A/B testing at Spotify
[Elite Camp 2016] Phil Nottingham - CRO with Video: Tips, Tricks and Tactics
 
[CXL Live 16] What to Test Next - Prioritizing Your Tests by Pauline Marol
 

Viewers also liked (14)

PDF
[Elite Camp 2016] Marie Polli - Taking Your Customers on a Journey That Match...
 
PDF
The power of customer loyalty in performance marketing - Anthony Quinn
PDF
Capta wlip credentials
PPTX
Criteo - NOAH13 London
PPTX
Creating an affiliate program for your small business
PDF
#RDVecom : Gael Cottet, responsable Canada Eulerian.inc : Les enjeux de la co...
PDF
15 Marketing and Production Tools Your SMB Shouldn't be Without
PDF
criteo-performance-advertising-playbook-2015
PPTX
Social Planning Framework
PDF
Enhanced Ecommerce Google Analytics Nordic Conference 2015
PDF
Performance Marketing: мост между потребителем и бизнес-результатами © Jason ...
PDF
Анна Свердлова - Actis Wunderman
PPT
Startup Metrics for Pirates (Nov 2012)
PDF
Digital Marketing Strategy 101
[Elite Camp 2016] Marie Polli - Taking Your Customers on a Journey That Match...
 
The power of customer loyalty in performance marketing - Anthony Quinn
Capta wlip credentials
Criteo - NOAH13 London
Creating an affiliate program for your small business
#RDVecom : Gael Cottet, responsable Canada Eulerian.inc : Les enjeux de la co...
15 Marketing and Production Tools Your SMB Shouldn't be Without
criteo-performance-advertising-playbook-2015
Social Planning Framework
Enhanced Ecommerce Google Analytics Nordic Conference 2015
Performance Marketing: мост между потребителем и бизнес-результатами © Jason ...
Анна Свердлова - Actis Wunderman
Startup Metrics for Pirates (Nov 2012)
Digital Marketing Strategy 101
Ad

Similar to How to Execute and A/B Test Cross-Channel Personalization (20)

PPTX
Putting Customers First: How To Build Data-Driven Strategies To Ensure Custom...
 
PPTX
Talk to me, Not at me: Testing for a Better Customer Experience
PPTX
Multi-Channel Experimentation & Personalization
PDF
Nancy's webinar
 
PDF
B2B Advertising, Personalization, and Landing Page Best Practices
PDF
DMEXCO 2019 Presentation - 3 steps towards effective personalization
PDF
Customer Experience Testing: The Key to Digital Success
PDF
Chicago Website Personalization Strategy Workshop
PDF
How to Keep Up Your Creative Testing After Budget Cuts
PDF
UX Bournemouth 2017
PDF
Creating Page Optimizations & Experiments in B2B
PPTX
Ratio Partners - Getting Started with Sitecore Personalisation
PPTX
A b-testing-101
PDF
Digital analytics: Optimization (Lecture 10)
PDF
How to drive great customer experiences for content ROI - government edition
PDF
Data-Driven UI/UX Design with A/B Testing
PDF
ROI on Personalized Experiences - Greg Kihlstrom, GK5A
PDF
Modernize 2018: Activating Customer Experiences for Modern Organisations - Ac...
PDF
Demandbase-Uberflip-Hexagon-Slides.pdf
PPTX
The ROI of UX
Putting Customers First: How To Build Data-Driven Strategies To Ensure Custom...
 
Talk to me, Not at me: Testing for a Better Customer Experience
Multi-Channel Experimentation & Personalization
Nancy's webinar
 
B2B Advertising, Personalization, and Landing Page Best Practices
DMEXCO 2019 Presentation - 3 steps towards effective personalization
Customer Experience Testing: The Key to Digital Success
Chicago Website Personalization Strategy Workshop
How to Keep Up Your Creative Testing After Budget Cuts
UX Bournemouth 2017
Creating Page Optimizations & Experiments in B2B
Ratio Partners - Getting Started with Sitecore Personalisation
A b-testing-101
Digital analytics: Optimization (Lecture 10)
How to drive great customer experiences for content ROI - government edition
Data-Driven UI/UX Design with A/B Testing
ROI on Personalized Experiences - Greg Kihlstrom, GK5A
Modernize 2018: Activating Customer Experiences for Modern Organisations - Ac...
Demandbase-Uberflip-Hexagon-Slides.pdf
The ROI of UX
Ad

More from CXL (20)

PDF
Pauline Marol - From Fantasy to Reality – How to Do AB Testing in a Small Com...
 
PDF
Jeff Sauer - Attribution is Bullshit
 
PDF
Sujan Patel - Content as a Growth Machine
 
PDF
Stephen Pavlovich - Testing Beyond the User Experience
 
PDF
Nicolas Visiers - User Experience Testing
 
PDF
Morgan Brown - Building a Growth Organization
 
PDF
Kristoffer Ewald - Modern Measurement Strategies
 
PDF
Guido Jansen -How to Involve the Whole Team in Optimization
 
PDF
Els Aerts - Advanced User Research
 
PDF
Daria Nepriakhina - Problem-Solution Fit
 
PDF
Christoph Luetke Schelhowe - Data for Everyone
 
PDF
Bob Ruffalo - How Impact Used ResearchXL to 3X Conversions
 
PDF
Andre Morys the magic of combining neuromarketing & data sciences
 
PDF
How Eye Tracking Research Helps You Run Better A/B Tests
 
PDF
[Elite Camp 2016] Craig Sullivan - Elite Camp Summary Session
 
PDF
[Elite Camp 2016] Thomas Barker - From Zero to In-House Optimisation Superstars
 
PDF
[Elite Camp 2016] Yehoshua Coren - Strategic And Tactical Implementation And ...
 
PDF
[Elite Camp 2016] Chris Mercer - How Top Marketers Are Using Google Tag Mana...
 
PDF
[Elite Camp 2016] Mogens Møller - How To Compete Against The Big Ones (Zaland...
 
PDF
[Elite Camp 2016] Johnathan Dane - The PPC Thermometer - Why You Should Care ...
 
Pauline Marol - From Fantasy to Reality – How to Do AB Testing in a Small Com...
 
Jeff Sauer - Attribution is Bullshit
 
Sujan Patel - Content as a Growth Machine
 
Stephen Pavlovich - Testing Beyond the User Experience
 
Nicolas Visiers - User Experience Testing
 
Morgan Brown - Building a Growth Organization
 
Kristoffer Ewald - Modern Measurement Strategies
 
Guido Jansen -How to Involve the Whole Team in Optimization
 
Els Aerts - Advanced User Research
 
Daria Nepriakhina - Problem-Solution Fit
 
Christoph Luetke Schelhowe - Data for Everyone
 
Bob Ruffalo - How Impact Used ResearchXL to 3X Conversions
 
Andre Morys the magic of combining neuromarketing & data sciences
 
How Eye Tracking Research Helps You Run Better A/B Tests
 
[Elite Camp 2016] Craig Sullivan - Elite Camp Summary Session
 
[Elite Camp 2016] Thomas Barker - From Zero to In-House Optimisation Superstars
 
[Elite Camp 2016] Yehoshua Coren - Strategic And Tactical Implementation And ...
 
[Elite Camp 2016] Chris Mercer - How Top Marketers Are Using Google Tag Mana...
 
[Elite Camp 2016] Mogens Møller - How To Compete Against The Big Ones (Zaland...
 
[Elite Camp 2016] Johnathan Dane - The PPC Thermometer - Why You Should Care ...
 

Recently uploaded (20)

PDF
Building a strong social media presence.
PDF
Master Fullstack Development Course in Chennai – Enroll Now!
PDF
MARG’s Door & Window Hardware Catalogue | Trending Branding Digital Solutions
PDF
Pay-Per-Click Marketing: Strategies That Actually Work in 2025
PPTX
UNIT 3 - 5 INDUSTRIAL PRICING.ppt x
PDF
Prove and Prioritize Profitability in Every Marketing Campaign - Zach Sherrod...
PPTX
Final Project parkville.............pptx
PPTX
Amazon - STRATEGIC.......................pptx
PPTX
Sumit Saxena IIM J Project Market segmentation.pptx
PDF
UNIT 1 -4 Profile of Rural Consumers (1).pdf
PDF
Mastering Bulk Email Campaign Optimization for 2025
PDF
EVOLUTION OF RURAL MARKETING IN INDIAN CIVILIZATION
PPTX
Tea and different types of tea in India
PPTX
Your score increases as you pick a category, fill out a long description and ...
PDF
AI & Automation: The Future of Marketing or the End of Creativity - Matthew W...
PPTX
Kimberly Crossland Storytelling Marketing Class 5stars.pptx
PDF
UNIT 1 -3 Factors Influencing RURAL CONSUMER BEHAVIOUR.pdf
PDF
Fly Emirates SEO case study by Rakesh pathak.pdf
PPTX
Presentation - MindfulHeal Digital Ayurveda GTM & Marketing Plan.pptx
PDF
How a Travel Company Can Implement Content Marketing
Building a strong social media presence.
Master Fullstack Development Course in Chennai – Enroll Now!
MARG’s Door & Window Hardware Catalogue | Trending Branding Digital Solutions
Pay-Per-Click Marketing: Strategies That Actually Work in 2025
UNIT 3 - 5 INDUSTRIAL PRICING.ppt x
Prove and Prioritize Profitability in Every Marketing Campaign - Zach Sherrod...
Final Project parkville.............pptx
Amazon - STRATEGIC.......................pptx
Sumit Saxena IIM J Project Market segmentation.pptx
UNIT 1 -4 Profile of Rural Consumers (1).pdf
Mastering Bulk Email Campaign Optimization for 2025
EVOLUTION OF RURAL MARKETING IN INDIAN CIVILIZATION
Tea and different types of tea in India
Your score increases as you pick a category, fill out a long description and ...
AI & Automation: The Future of Marketing or the End of Creativity - Matthew W...
Kimberly Crossland Storytelling Marketing Class 5stars.pptx
UNIT 1 -3 Factors Influencing RURAL CONSUMER BEHAVIOUR.pdf
Fly Emirates SEO case study by Rakesh pathak.pdf
Presentation - MindfulHeal Digital Ayurveda GTM & Marketing Plan.pptx
How a Travel Company Can Implement Content Marketing

How to Execute and A/B Test Cross-Channel Personalization

  • 1. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Executing and A/B Testing Cross-Channel Personalization Andy Zimmerman, Evergage Shanelle Mullin, ConversionXL
  • 2. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Agenda • The Cross-Channel Imperative • Challenges • Solution Requirements • Examples • A/B Test Process • Prioritization • Duration • Results and Archiving • Q & A
  • 3. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION The Cross-Channel Imperative
  • 4. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Challenge: Cross-Channel is Happening Now
  • 5. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Challenge: Cross-Channel is Happening Now
  • 6. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION 70% OF WALMART.COM TRAFFIC LAST HOLIDAY SEASON WAS MOBILE DEVICES. Forrester: Your Digital Experience Technology Starts With A Customer Journey Map
  • 7. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
  • 8. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION 69%
  • 9. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION $1.5 Trillion retail sales generated by cross-channel interactions – Forrester, The Omnichannel Commerce Playbook for 2016
  • 10. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION 47% of bank customers engaged in at least one cross-channel interaction over 90 day period. – Forrester, Design Better Cross-Channel Banking Journeys
  • 11. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Top Goal is Customer Experience
  • 12. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION 4 Key Elements of a Great Customer Experience 4 1 Remember me. 2 Understand me. 3 Help me achieve my goals. Surprise me. Delight me.
  • 13. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Cross-Channel Personalization
  • 14. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION 77% of consumers have chosen, recommended, or paid more for a brand that provides a personalized service or experience. – Forrester’s North American Consumer Technographics® Brand Compass Survey, Q3 2015
  • 15. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION The Cross-Channel Challenge
  • 16. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Challenge: Organizational Misalignment – Forrester, The Omnichannel Commerce Playbook for 2016
  • 17. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Challenge: Organizational Misalignment – Forrester, The Omnichannel Commerce Playbook for 2016
  • 18. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Challenge: Data is Siloed Marketers are using SIX or more tools to collect performance data. Marketing Analytics Tool Website Analytics Tool arketing Automation Tool Operational Analytics Tool ness Intelligence System Sales Automation Tool rketing or Online Surveys rformance/Analytics Tool CRM System ERP System 0% 20% 40% 60% 80% 30% 42% 43% 44% 54% 54% 57% 57% 69% 79%
  • 19. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Requirements for a Cross-Channel Personalization Solution
  • 20. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Track Behavior Across Channels Website Email SearchMobile Web App
  • 21. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Synch Data with Other Systems CRM Marketing Automation Email Marketing Data Warehouse Data Lake Call Center
  • 22. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION One Unified Customer Profile
  • 23. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Core Capabilities of Personalization Platform Cross-channel Targeting Algorithmic Recommendations Real-Time Segmentation A/B & MV Testing Analysis & Attribution Behavioral Tracking Unified Customer Profile
  • 24. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Cross-Channel Personalization Maturity Evolution Beginning Intermediate Part A Intermediate Part B Advanced Step 1 Digital Channel 2 or 3 Digital Channels Synch with other systems All channels, including agents Web Mobile App Open-time Email Search Web Mobile App Open-time Email Search CRM System Email/MAP Data Warehouse Call Center In-Store Kiosk Reps/Agents Benefit One Team Multiple Teams Better data, Targeting x-channel Organization-wide Challenge Getting Started Cross-Team coordination needed Coordinate synch if not out-of-box, APIs Additional Integrations
  • 25. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Examples of Cross-Channel Personalization
  • 26. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Ski Resort Example
  • 27. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Ski Resort Example
  • 28. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Ski Resort Example
  • 29. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Rue La La Example
  • 30. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION How to A/B Test
  • 31. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
  • 32. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Only 32% of small companies and 42% of large companies have a structured, documented process.
  • 33. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
  • 34. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION 1. Research 2. Prioritize 3. Experiment 4. Analyze and Learn
  • 35. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
  • 36. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Prioritization Problems 1. Ease and impact are too subjective 2. Too rigid. 3. Doesn’t encourage a data-driven culture
  • 37. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
  • 38. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Prioritization Fixes 1. Binary responses 2. Customizable variables 3. Ask everyone to bring data
  • 39. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Grab the Template bit.ly/pxlmodel
  • 40. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
  • 41. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
  • 42. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
  • 43. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Determining How Long to Test 1. Calculating sample size 2. Running for full weeks, usually two cycles
  • 44. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
  • 45. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
  • 46. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
  • 47. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Use the Calculator bit.ly/evansample
  • 48. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Ensuring a Representative Sample 1. Every day of the week 2. Various traffic sources 3. Ad, email, social, content schedule 4. External factors (e.g. holidays) 5. Comparison shoppers
  • 49. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION On External Factors... 1. Season 2. Day of the Week 3. Press 4. PPC / SEM / SEO 5. Word of Mouth
  • 50. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION On Sample Pollution... 1. Length pollution 2. Device pollution 3. Browser pollution 4. Cookie pollution
  • 51. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Watch Out... 1. Regression to the mean 2. The novelty effect 3. Simultaneous testing
  • 52. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Creating a Hypothesis 1. Is it designed to solve a conversion problem? 2. Is it measurable / testable? 3. Will you gain insight, win or lose?
  • 53. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
  • 54. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Skills That Help with Testing 1. HTML, CSS, JavaScript, jQuery 2. Design 3. Copywriting
  • 55. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
  • 56. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Get the Details bit.ly/gaabresults
  • 57. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION No Difference? Not So Fast... 1. Maybe the implementation sucked. 2. The variation might’ve won in a segment.
  • 58. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION
  • 59. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Segmentation 101 1. Desktop vs. Mobile 2. New vs. Returning 3. Traffic Direct on the Page vs. Traffic via Internal Link
  • 60. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Insights > Wins
  • 61. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Why Archive? 1. To avoid repeating tests 2. To better communicate wins and insights 3. To improve the quality of future tests 4. To create an insights > wins culture
  • 62. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Archive Methods 1. Iridion 2. Effective Experiments 3. GrowthHackers Projects 4. Experiment Engine 5. Testing Tool 6. Excel
  • 63. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Conversion optimization training program. evergage-webinar
  • 64. WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION Questions?