Ahead of the Curve: How 23andMe Improved UX with Performance Edge
0 likes405 views
This document summarizes a webinar presented by Courtney Ball, Antonio Contreras, and Whelan Boyd from 23andMe on how the company improved user experience through A/B testing and implemented Performance Edge to optimize page load times. The webinar discussed how website performance directly impacts business metrics like conversion rates, highlighted best practices for measuring performance, and explained how standard A/B testing can negatively affect page speeds. It then introduced Performance Edge as a solution that delivers only necessary code variations to visitors, significantly reducing file sizes and browser execution times. 23andMe's case study showed how implementing Performance Edge through Cloudflare Workers dramatically improved their page load performance metrics and allowed them to scale up testing without slowing down
Ahead of the Curve: How 23andMe Improved UX with Performance Edge
- 1. 1 Ahead of the curve How 23andMe improved UX with Performance Edge + Courtney Ball, Antonio Contreras, and Whelan Boyd
- 2. 2 Introductions Courtney Ball Sr. Manager, Web Marketing Antonio Contreras Sr. Web Developer Whelan Boyd Sr. Product Manager
- 3. ● Customize the widgets on your page to your preference ● This webinar is recorded and you will receive the link with the slides in the next few days ● We will have time for questions! Please submit them in the Q&A box on the left side of the screen Housekeeping
- 4. 4 Agenda 1. Why performance matters 2. How to think about measuring performance 3. How experimentation affects performance 4. Performance Edge 5. Case study: 23andMe 6. Q&A
- 6. 6 The business impact of speed 13% increase in sales when page load time was cut by half Found that 53% of mobile visits were abandoned if page took > 3 seconds to load 7% increase in conversion when page load time reduced by 850 ms Source: https://developers.google.com/web/fundamentals/performance/why-performance-matters/
- 8. 8 Measure impact to user experience Source: Google’s Web Fundamentals First Paint (FP) First Contentful Paint (FCP) First Meaningful Paint (FMP) Time to Interactive (TTI)
- 9. 9 Best practices for measuring Free Online Tools Synthetic Traffic Easy to run Actionable Insights Real User Monitoring Production Traffic Segment by Device, Location, etc. Robust Analytics
- 11. 11 Standard Web Experiments 1. Snippet size increases with more experiments 2. For specific visitors, the downloaded snippet includes all project variations vs just active variations 3. Larger snippet results in more time to download and execute 4. Resource-constrained browsers have to execute several tasks Performance Challenges for Web Experiments Standard CDN Script Experiment A variation 1: CTA: Get Started variation 2: CTA: Try it out Experiment B variation 1: CTA: Promo Code variation 2: CTA: No Promo Code Task load: URL Targeting Audience Targeting Traffic Splitting Apply Variations Event Tracking
- 13. 13 Only returns visitor-specific variations Standard Web Experiments Task load: Experiment Delivery Network Apply Variations Event Tracking Key experiment decisions are moved to the Edge Much smaller file to download and execute in the browser with less tasks to perform Performance Edge Experiments Standard CDN Script Experiment A variation 1: CTA: Get Started variation 2: CTA: Try it out Experiment B variation 1: CTA: Promo Code variation 2: CTA: No Promo Code URL Targeting Audience Targeting Traffic Splitting Apply Variations Event Tracking CTA: Get Started Script URL Targeting Audience Traffic Splitting
- 14. 14 Why is it so fast? Less data Far smaller file loading over the wire Reduce download time Less code Only code necessary for active variations Fraction of the execution time in browser Faster machines Powerful Cloudflare Workers execute expensive tasks Located extremely close to end visitor
- 15. 15 Proper Measurement Make things smaller Performance Edge ● User-focused Metrics ● Synthetic and real traffic ● Lighter download ● Shorter execution ● Quick implementation with any CDN ● Push logic to the Edge Summary
- 17. 17 23andMe: Intro Courtney Ball Sr. Manager, Web Marketing Antonio Contreras Sr. Web Developer
- 18. 18 23andMe: Overview Company size: 500+ employees Location: Sunnyvale, California Mission: Help people access, understand and benefit from the human genome. What we offer: 23andMe gives individuals easy and affordable access to their genetic information, offering a growing list of reports on traits, wellness, carrier status and genetic ancestry. Consumer Research Therapeutics
- 19. 23andMe: Testing Overview TV AUDIO DISPLAY SOCIAL Testing focus for 2020: ● Optimize kit mix. ● Optimize mobile experience. ● Optimize personalization through audience targeting. 23andMe Customer Acquisition / Marketing Funnel SEARCH SITE Consideration Conversion Awareness
- 20. 20 23andMe: Approach to improving user experience through a site redesign Create new designs (multiple options) Consumer insights conducts qualitative research Identify key areas of opportunity through a site audit A/B test against existing page via Optimizely Iterative testing (if needed)
- 21. 21 Test: Redesigned Product Page Test Variant #1Control Round 1
- 22. 22 Test: Redesigned Product Page Test Variant #1Control Round 1 Winner
- 23. 23 Test: Redesigned Product Page Control Test Variant #1 Round 2 Test Variant #2
- 24. 24 Test: Redesigned Product Page Control Test Variant #1 Round 2 Test Variant #2: Winner
- 25. 25 Why we needed performance edge as we scaled A growing point of internal tension… 1. More testing = slower page speed 2. More emphasis placed on performance Eagerness to try new tactics to improve site performance ahead of the holiday season. + Frequency A/B Tests 2016 2017 2018 2019 2020 Year NumberofTestsLaunched 20 40 60 0
- 26. 26 How we implemented Performance Edge at 23andMe 1. We had an existing Worker on Cloudflare that we were using to load our Optimizely JavaScript 2. We updated the existing Worker for Optimizely Edge and created a separate Worker for the old Optimizely JavaScript 3. We used an integration environment to test performance and root out errors Cloudflare CDN / Worker Client / Browser 23andMe Web Server
- 27. 27 Speed improvements with Performance Edge Our performance statistics improved across the board: Standard Snippet ⚡Optimizely Edge Snippet First Contentful Paint Mobile: 2.7s Desktop: 2.5s Mobile: 1.2s Desktop: 1.1s First Meaningful Paint Mobile: 3.0s Desktop: 2.7s Mobile: 1.6s Desktop: 1.5s Time to Interactive Mobile: 6.7s Desktop: 6.2s Mobile: 4.4s Desktop: 5.4s Key Benefits: dramatic performance improvements on mobile and we can continue scaling our testing program without slowing down our site!
- 28. 28 Questions? Ask them in the webinar console