Generating Quality Hypotheses For Higher Uplifts | Masters of Conversion by VWO
0 likes1,132 views
The document discusses strategies for enhancing conversion rates through the formulation of quality hypotheses based on user behavior and data analysis. It outlines a research plan involving user testing, heuristic reviews, and web analytics to identify areas for improvement on a website. The proposed changes aim to better address user needs and motivations, ultimately leading to significant uplifts in conversion and revenue metrics.
![Because we saw [prior research] we believe [condition] for [users] will
encourage them to [behavior].
We will know this is successful when we see [effects] happen to [metrics].
Experiment hypotheses template
Prior research always informs hypotheses](https://image.slidesharecdn.com/mocantoniabogatu-bitdefender-200203081408/85/Generating-Quality-Hypotheses-For-Higher-Uplifts-Masters-of-Conversion-by-VWO-6-320.jpg)
Generating Quality Hypotheses For Higher Uplifts | Masters of Conversion by VWO
- 1. Generate quality hypotheses for higher uplifts Antonia Bogatu Conversion Optimization Manager
- 2. Why am I talking about this? Conversion Optimization Manager @ Bitdefender - 3 years experience client side in cybersecurity - Agency experience - Pre-CRO: e-commerce and lead gen business I wish I knew everything I’m about to share
- 3. The discovery of what matters How do you get from friction to delight?
- 4. 1. What are we solving for? 2. Whose problems are we solving? 3. What do they need? 4. What do they think they want?Why? 5. How are they making a decision?Why? 6. What are they thinking when they see our offer? 7. What are they thinking when they don’t choose our offer? 8. Does the value we provide match what they are looking for? 9. Where is the site losing conversions? 10. What are people doing or not doing on the website? 11. What leads more people to do the desired behavior? Asking questions
- 5. Data & Conversion Insights + Behavior Analysis Segmented Surveys (Guerilla) User Testing Heuristic Review Web Analytics Clickstream data Form Analysis User session recordings What data turns into hypotheses? On-site exit polls Qualitative + Quantitative Research Data
- 6. Because we saw [prior research] we believe [condition] for [users] will encourage them to [behavior]. We will know this is successful when we see [effects] happen to [metrics]. Experiment hypotheses template Prior research always informs hypotheses
- 7. ● Using PXL by ConversionXL (source) ● Hypotheses are evaluated by their quality and ease of implementation using the questions on the next slide ● RESULT (14- 0) The higher the score of a hypothesis, the sooner it will be tested. Experiment hypotheses scoring and prioritization
- 8. Scoring criteria Is the treatment: ● above the fold?(1 or 0) ● noticeable within 5 sec? (2 or 0) ● adding or removing an element?(2 or 0) ● designed to increase user motivation?(1 or 0) ● running on (a) high traffic page(s)? (1 or 0) ● addressing an issue discovered via user testing? (1 or 0) ● addressing an issue discovered via qualitative feedback (surveys, polls, interviews)? (1 or 0) ● addressing insights found via digital analytics? (1 or 0) ● supported by mouse tracking heat maps or eye tracking? (1 or 0) ● easy to implement? (less than 4 hrs = 3, up to 8 hrs = 2, under 2 days = 1, more = 0) Experiment hypotheses scoring and prioritization
- 9. Hands on ... Getting 85.4% uplift when redesigning a key page.
- 11. Existing content - Above The Fold: ● Award news with CTA for offer and secondary comparison page link Solutions page
- 12. Solutions page Existing content - Second screen: - Product finder and in-page navigation
- 13. Solutions page Existing content - Third screen: - Awards and recognition
- 14. Solutions page Existing content - Fourth screen: - Main solutions
- 15. Solutions page Existing content - Fifth screen: - Dedicated SMB and smart home solutions
- 16. Solutions page Existing content - Sixth screen: - Trust and social proof
- 17. Web analytics - Identify current page performance - Identify user engagement with product filter Heuristic Review - Evaluate the page for clarity, relevance, friction, distraction and value Research plan
- 18. On site exit polls - Find out if the page has the information they needed by asking: “Were you able to find the information you were looking for?”- Yes/No, followed by “What information was missing from the page?” in case the answer was No Clickmaps and scrollmaps - Find out what are users engaging with and what content is within their reach (% percentage) User recordings - What behaviors are common and are there any patterns? Research plan
- 19. User testing - Recruit 5 users for desktop and 5 users for mobile - Same tasks for both desktop and mobile - focusing on acquisition Tasks: 1. Go to www.bitdefender.com. Look at the homepage for five seconds then look away. Try to recall the one thing that stands out most. What does this company do? What do you remember? 2. Think of protecting your devices online. Think aloud about the questions you might have about how to achieve this. With those questions and your objective in mind, find a security solution that fits your needs and purchase what you just found. Go as far as you can **without entering payment information**. Which questions/issues were not clear to you? Which are you still unsure about? Research plan
- 20. Tasks (continued): 3. Is there any reason you would think twice about buying / signing up for one of the solutions? Why or why not? 4. Is there anything about Bitdefender’s website or tone that you feel is really missing? What would have made you like or enjoy this product's sales experience more? Research plan
- 21. Follow-up questions: 1. What frustrated you most about this site? 2. If you had a magic wand, how would you improve this site? 3. What kind of doubts and hesitations might have stopped you from buying? 4. What almost stopped you from buying? Research plan
- 22. Web analytics ● Data showed clear room for improvement ● Interaction with the product finder without purchases completed from it Insights from Research
- 23. Heuristic Review Context ● Users went to this page mainly after seeing the homepage Insights from Research
- 24. Heuristic Review ● Little relevance ATF: users were looking for solutions but what was displayed ATF was not showing solutions, but awards and an offer Insights from Research
- 25. Heuristic Review ● The page needed to showcase all our Consumer offering to be relevant, but the page had only 3 products in the fourth screen thus not meeting user expectations ● Not enough value - short description only Insights from Research
- 26. Heuristic Review ● The product finder added some friction to the page: no way to navigate back and forth and no clickability signaling for the options Insights from Research
- 27. On page exit poll - Desktop ● “Were you able to find the information you were looking for?”- Yes/No - 35%/65% ● “What information was missing from the page?” in case the answer was No Insights from Research
- 28. Clickstream data - clickmaps Desktop ● Some interaction with the product finder and the CTA ● No clicks on solutions Buy CTA Insights from Research
- 29. Clickstream data - clickmaps Mobile ● The most clicked item on the page was the menu - consistent with little relevance ● Little to no interaction elsewhere on the page Insights from Research
- 30. Clickstream data - scroll maps Desktop ● The illusion of completeness - “false bottom” was strongly manifesting after the first one and a half screens we had less than 50%users scrolling Insights from Research
- 31. Clickstream data - scroll maps Mobile ● Similar scrolling patterns to desktop ● Having the first section promote the free app meant even less people scrolled down Insights from Research
- 32. User recordings Desktop ● Lack of relevance made users go back and forth to the homepage ● Eventually they went to main menu as they couldn’t find solutions Insights from Research
- 33. User recordings Mobile ● Lack of affordance made the interaction with the product filter incomplete ● Scrolling up and down - the page did not provide the content they were looking for Insights from Research
- 34. User testing Desktop ● Users reads through the first screen with no intent to click ● Little interaction with product finder ● Due to the previous version of HP and the little relevance the page provided it looked like users lacked choice Insights from Research
- 35. User testing Mobile ● Most users couldn’t find paid solutions ● Those who found them couldn’t compare solutions ● They lacked RTBs, only with a short description and price shown Insights from Research
- 36. ● If we quickly established trust in the first screen then our users would trust the brand more and would feel more compelled to choose a security solution from us so our conversion rate would go up. ● If we made product selection easier by showing users solutions by platform, users would have an easier time finding products so conversion rate would go up. ● If we added benefits to the displayed products then users would be more compelled to choose a product so our conversion rate would increase. ● If we added a link to the product page for each product then the user could get more information to figure out if the product matches their needs so our conversion rate would go up. Hypotheses generated from research
- 37. ● If we added a link to the product page for each product than users would be able to choose longer engagements thus increasing the AOV. ● If we showed the most popular option in each category of solutions, users would be persuaded to choose it so our conversion rate would go up. ● If we displayed product from the most complete to the basic ones then we would persuade users to opt for the best protection and thus our RPV would go up. ● If we displayed the services in the solutions page we would increase awareness for them resulting in increased conversion rate for them. ● If we made the in-page navigation sticky - thus more available - users would be able to use it so they would more easily filter choices and our conversion rate would go up. Hypotheses generated from research
- 38. ● Using PXL and web analytics data - traffic and conversion rates ● It would take more than one year to test all hypotheses ● Decision: Combine all hypotheses into one big page redesign Hypotheses prioritization
- 39. Having learnt so much about how users interacted with the page: ● Strong UVP above the fold, citing our no. 1 RTB (from surveys) ● Product filter ATF - it would scroll to dedicated solutions for the platform Challenger
- 40. ● Trust area with all relevant awards (and sources linked) ● LP banner area - feature the best offer for the month Challenger
- 41. ● Products & Services - filtered by device type - including a few of the most important features to help with making a decision - “Buy button” and “Learn more” ● Sticky navigation featuring platform types, product choice widget, Trials and renewals Challenger
- 42. ● Trust area - showing how Bitdefender compares to other cybersecurity in terms of protection and performance Challenger
- 43. ● Product selector was the most used feature Results
- 44. ● People started scrolling more so more people saw products (from less than 25% to 75%) Results 75% 25% 50% 75% 50% 25%
- 45. ● More users choosing a product on this page ● More product page views +85.4% uplift in CR, +77.4% uplift in RPV Results
- 46. Win & implement Prioritization Testing Learn & iterate successful not successful Data & Conversion Insights + Behavior Analysis Repeat Optimization is an ongoing process +
- 47. Thoughts? Questions? Thank you! Find me on LinkedIn (https://www.linkedin.com/in/antoniabogatu/)
- 48. Next Sessions Announcing in January 2020 vwo.com/webcast/ Happy New Year :)