Measuring Digital Return on Experience
2 likes830 views
Emily Leahy discusses InterContinental Hotels Group's (IHG) journey to establish a Digital Return on Experience (ROX) program. The program aims to design, implement, and manage digital customer experiences through routine performance practices. Key aspects discussed include establishing measurement practices using scorecards and analytics, optimizing experiences through testing and small enhancements, and structuring teams and processes to work more efficiently. Leahy shares lessons learned about gaining buy-in for change, establishing routines early, and finding allies within the organization. The overall goal is to continually improve the digital customer experience and key metrics like revenue, loyalty contributions, and brand perception.
Measuring Digital Return on Experience
- 1. InterContinental Hotels Group Measuring Digital Return on Experience What We’ve Learned On Our Journey So Far Emily Leahy Manager, User Experience Strategy
- 2. “…the only poster where the family was shown safely outside.” Smoke Detectors Save Lives Always a Strategist
- 3. 3 What the VP thinks I do What my boss thinks I do What the rest of the UX team thinks I do What I think I do What I actually doWhat Product Managers think I do Emily Leahy Manager, UX Strategy
- 4. 4 Problem Large companies often make huge investments in large-scale redesigns where we change everything, making it hard to tell what worked, what didn’t, and we miss the opportunity to learn. Time Success Redesign 1 Redesign for Responsive, Unique Brand Experiences 2012 2015
- 5. 5 • Formally founded in 2003 • Globally Based in Denham, UK • World’s largest hotel company (by rooms) • 12 brands in over 100 countries with over 4,942 hotels and more than 474,290 rooms • 13 major dot coms • $1,858m total revenue About IHG UX Strategy Delegation
- 6. 6 And how do we do that at a company like IHG? Time Success 2012 2015 Redesign 1 Redesign for Responsive, UBE Design updates in a year KPILift Home Search Results Hotel Details Rooms & Rates Payment Problem How do we get from here to there?
- 7. 7 Problem One small example Before After Result Points and Cash $
- 8. 8 Problem One small example Before After Result Quick, change it back! Points and Cash $
- 9. 9 Quiz Time!
- 10. 10 A.Change it back and hope that solves the problem without any unintended consequences. Let’s try hope… again. Quiz Time!
- 11. 11 B.Dig our heels in and point back to all the usability testing and analytics that supported the research-based design decision. Hey, we used, “The Force!” Quiz Time!
- 12. 12 C. Test the hypothesis, “Removing the rate preference dropdown decreased Points and Cash revenue,” before proceeding. Search your feelings, you know it to be true Quiz Time!
- 13. The Digital Return on Experience (ROX) program supports IHG’s Web Channel performance objectives by routinely conducting the practices required to design, implement, and manage digital customer experiences in a disciplined way.
- 14. Performance Practices People & Processes Performance What we do with it Digital ROX Program structure
- 15. 15 Tools help us focus on IHG’s business metrics and that “automate the mundane” and help us spot opportunities faster Measurement
- 16. Performance What matters to the business? Web Channel Metrics • Web Channel Revenue • Loyalty Contribution • Ancillary Revenue • Brand Perception • Digital Self Service And of course, digital guest experience…
- 17. Performance Our measurement universe Scorecards Ad Hoc Analytics VoC Reports + Web Channel Checkout Revenue Revenue Contribution Average Daily Rate Revenue Per Available Room Booking Conversion Rate/Funnel Web Rewards Club Enrollments Credit Card Sign Ups Brand Perception Gap Social Brand/Web Sentiment Visits vs. 2+ Page Visits Property page views Customer escalations + = Digital ROX KPIs, by Brand & Region Global , Multi- Brand Data
- 18. Demand'KPI' Prior'Period'Performance' Current'Month' Trend' Score'Calcula8on' Score' Dimensional'Score' Health'Score' A" Homepage"to"Room"Rate"Conversion" 4%" 5%" 6%" (5%74%)/(6%74%)*"100" 50" Demand"Performance" Score"*"Dimension" Weight" " 78.5%x25%" Demand" Dimensional"Score"" +"" Conversion" Dimensional"Score"" +"" SaLsfacLon" Dimensional"Score"" +"" Revenue" Dimensional"Score" " 24.11%"+"18.75%"+" 19.58%"+"8.9%" B" Homepage"to"Rewards"Club"Enroll"Conversion" 4%" 7%" 6%" (7%74%)/(6%74%)*"100" 150" C" Search"Results"to"Room"Rate"Conversion" 4%" 6%" 6%" (6%74%/6%74%)"*"100" 100" D" Search"Results"to"Rewards"Club"Conversion" 4%" 5%" 6%" (5%74%/6%74%)"*"100" 50" E" Hotel"Details"to"Room"Rate"Conversion" 4%" 6%" 6%" (6%74%/6%74%)"*"100" 100" F" Hotel"Details"to"Rewards"Club"Conversion" 4%*" 5%*" 4%*" (5%/4%)"*"100" 125" G" Site"Exits"to"Chase"Enrollment"over"Site"Visits" 4%**" 4%**" 4%**" (1/1)*"100" 100" Demand'Performance'Score' A+B+C+D+E+F+G/(#"KPIs"*"100)" 96.43%' 24.11%' Conversion'KPI'' H" Room"Rate"to"Booking"ConfirmaLon"Conversion" 4%" 5%" 6%" (5%74%/6%74%)"*"100" 50" Conversion"Performance" Score""*"Dimension" Weight"I" Rewards"Club"Submission"Conversion" 4%" 6%" 6%" (6%74%/6%74%)"*"100" 100" Conversion'Performance'Score' H+I/(#"KPIs"*"100)" 75%' 18.75%' Sa8sfac8on'KPI'' J" Rate"Experience" 7" 8" 9" (877/977)"*"100" 50" SaLsfacLon"Performance" Score"*"Dimension" Weight" K" Purpose"of"Visit"CompleLon" 8" 10" 10" (1077/1077)"*100" 100" L" Discovery" 6" 5" 6" (576/676)"*"100" 750" M Relevancy" 7" 7.7" 7.5" (7.777/7.577)"*"100" 140" N" Convenience" 6" 7.3" 7" (7.376/776)"*"100" 130" O" Bobom"Line" 8" 8" 8.5" (878/8.578)"*"100" 0" Sa8sfac8on'Performance'Score' (J+K+L+M+N+O)/(#"KPIs"*"100)" 78.33%' 19.58%' Revenue'KPI'' P" Web"Channel"Checkout"Revenue" $75M" $80M" $90M" ($80M7$75M/$90M7$75M)"*"100" 33" Revenue"Performance" Score"*"Dimension" Weight" Q" Number"of"Bookings" 120K" 100K" 150K" (100k7120K/150K7120K)"*"100" 767" R" Revenue"per"Booking" $280" $520" $450" (5207280/4507280)"*"100" 141" Revenue'Performance'Score' (P+Q+R)/(#"KPIs"*"100)" 35.6%' 8.9%' 71.34%' Performance The fancy math
- 20. 20 Performance What we do with it Strategy Hypotheses Optimization Plans Stakeholder Input Competitor Analysis Industry Trends User Session Replays 3rd Party Research Call Center Data Vendor Vision / Recommendations Usability Testing Findings Site Analytics Online Chat Log A/B Testing Small Enhancements Winners Okay, let’s get to work…
- 21. 21 Performance What we do with it Where should we start? Strategy Hypotheses Optimization Plans Stakeholder Input Competitor Analysis Industry Trends User Session Replays 3rd Party Research Call Center Data Vendor Vision / Recommendations ROX Scorecard Metrics Usability Testing Findings Site Analytics Online Chat Log A/B Testing Small Enhancements Winners
- 22. 22 Performance What this helped us do • Optimization plan for membership enrollment form • Optimization plan for login • A/B tests for booking conversion • Strategy for key ancillary revenue projects • Strategy for IHG.com • And the list grows daily
- 23. 23 • Optimization plan for membership enrollment form • Optimization plan for login • A/B tests for booking conversion • Strategy for key ancillary revenue projects • Strategy for IHG.com • And the list grows daily Performance What this helped us do Increases Loyalty Contribution Enables Digital Self Service Increases Channel Revenue Grows Ancillary Revenue Improves Brand Perception Builds a Sustainable Program … But you said “routine” and “disciplined”
- 24. 24 Routines that keep us focused on our vision and achieving results that build buy-in. Practices
- 25. Practices How do we get it all done? 5 weeks of website analysis of booking funnel lead to: • 4 Small Enhancements • 8 Online Test Ideas evolve to enhancements • 5+% Lift in funnel entry • 8-15% lift in each step of funnel • 24% increase in funnel conversion • Leads to $8.6MM incremental/month once deployed to all traffic • +114,000 new bookings and +$51.6M Incremental Revenue (6 mths after implementation) 12 Quick Wins* Page Quick Win Approach Priority All New CTA Button Verbiage, Color and Visual Treatment Test First 1 All Image/Content Refresh focus on benefits and brand Test First 1 Search Results Explain sister hotels in search results Test First 1 Room Rate More visibility of refining search options Test First 1 Room Rate Sense of Urgency Messaging Test First 1 Room Rate Minimize confusion and create differentiation of Best Flexible Rate/ Deals by moving to bottom of page Test First 1 Guest Info Move Credit Card offer to bottom of page Test First 1 Guest Info Remove Learn More link Implement 1 All Targeted content for returning visitors Implement 2 Room Rate Default to Currency Selector to current country currency Implement 2 Guest Info Make Hotel Stay Summary easier to find and review Test First 2 Guest Info Edit Hotel Stay Summary On Page Implement 3 *Quick'wins'apply'to'the'current'state'of'the'desktop'website'for'Crowne'Plaza'
- 26. Practices What to do when - high level Heuristic Assessments & Design Exploration/Concepts The$quarterly$and$monthly$feedback$loop$will$7e$into$an$orchestra7on$workflow$and$set$ cadence$of$research,$analysis,$planning$and$execu7on$processes$ Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Digital Health Scorecard !!Staybridge,!Candlewood,!IC,!CP! !!Holiday!Inn!Family!Brands! !!IHG,!Indigo,!Hualuxe,!! !!Kimpton,!EVEN!Brand Site Experience Research 1:1 Usability Tests (Deep Dive) Small Enhancements Small Enhancements (AEM) Online Testing Quarterly!Brand!Summit! !!Staybridge,!Candlewood,!IC,!CP! !!Holiday!Inn!Family!Brands! Monthly!Performance!Review! Quick Win Analysis & Design Explorations
- 27. Practices What to do when - details r#Year# th&Deep&Dive&1:1&Modera9on& Year# taybridge,&IC,&Crowne&Plaza& mily&Brands& ndigo,&Even& taybridge,&IC,&Crowne&Plaza& mily&Brands& ws#Per#Year# s&Analysis&Per&Brand& r#Year## ntly&(6&–&9&Week&Test&Cycles)& s#Releases#Per#Year# Story&Points&(3&Developers)& Story&Points&(3&Developers)& ments#Release#Per#Year# Story&Points&(2&Developers)& Story&Points&(2&Developers)& calendar#that#extends#12#months#out#will#drive#increase#producHvity#
- 28. 28 Increase bandwidth in key areas including design, research, and project management while promoting changes that make us more efficient and effective. People & Processes
- 29. People & Processes Where we want to be DXD#Director# Performance*Strategy* Performance*Design*Studio* Op,miza,on#Lead#(Mgr)# Op,miza,on#Analyst# Interac,ve## Project#Manager#(Mgr)# #Studio#Manager#(Mgr)# +2# New# Small*Enhancements* &*Online*Tes:ng* Brand*Campaigns* GSM*&*Brand*Priori:es* #####Hybrid#x3# #####Copywriter#x2# #####Designer#x5# ####UEA#+#Hybrid#x6# #####Designer/AD#x7# #####Content#### #####Str/Sp# x5# #####Copywriter#x1# DXD*PEOPLE*COUNT*CHANGE* 38****!****44* CX#Strategy# Lead#(Mgr)# Experience# Researcher# +#Research#Vendor# +2# New# #####Hybrid#x2# +2# New# SLT#Sponsor# DXD* ########Experience#Lead#(Mgr)#–#Ryan,#John,#Mike,#Kim# Ideal UX Team
- 30. 30 People & Processes Many hands make light work Area Allies Measurement Analytics Team, Brand Teams, Product Managers, Testing & Optimization Quarterly Brand Summits Product Management, Testing & Optimization, Mobile, Ent.Content Strategy Monthly Performance Reviews Product Management, Testing & Optimization, Small Enhancements
- 31. 31 People & Processes From our perspective Design Documentation Ticketing Post-Launch Ticketing Design Direction Design Prototype Ticketing Post-Launch Optimization Launch Core Documentation Design Support Currently Proposed High-level design process
- 32. 32 People & Process From the organization’s perspective Idea% Project%Charter% Crea.ve%Brief% Crea.ve% Concepts% AEM%Specs%% &%AX%Wires% Wireframes%&% Annota.ons% Visual% Design% Solu.on%Needs%Document% Sprint%0% % …% Defini.on% Solu.on%Design% Implementa.on% DXD% DXD% DXD% C.Ops% W%D% GT% DXD%P%M%Biz%Stakeholder% Prototype% Tes.ng% DXD% Sprint%1% Sprint%X%
- 33. Automate the mundane Results Orientation Sustainable Program Performance Practices People & Processes
- 34. 34 What we’ve learned A journey we are on 1. You’ll always be a rebel, but the old ways exist for a reason 2. Be clear and repeat often why you’re asking for change (sometimes you’re even reminding yourself) 3. Use challenging moments as a way to show why a different approach is necessary 4. Establish routines as early as possible 5. Find allies within your organization