Empowering Product Teams: Build Delightful Products with Customer Validation
- 1. Build Delightful Products with Customer Validation John Little Hannah Flynn With: Moderated by: TO USE YOUR COMPUTER'S AUDIO: When the webinar begins, you will be connected to audio using your computer's microphone and speakers (VoIP). A headset is recommended. Webinar will begin: 11:00 am, PST TO USE YOUR TELEPHONE: If you prefer to use your phone, you must select "Use Telephone" after joining the webinar and call in using the numbers below. United States: +1 (213) 929-4232 Access Code: 424-277-812 Audio PIN: Shown after joining the webinar --OR-- Empowering Product Teams Webinar Series
- 2. Revulytics gives any software producers deep and actionable insight into who is using their software products and how they are being used, and the out-of-box analytics that enable them to grow incremental revenue, convert and retain customers, and make decisions about licensing and cloud transformation strategies.
- 3. Empowering Product Teams Webinar Series Click on the Questions panel to interact with the presenters www.productmanagementtoday.com/webinar-series/empowering-product-teams
- 4. About John Little John Little is the VP of Product Marketing at Centercode and co-creator of the Customer Validation Framework. His background is in engineering and project management, with an emphasis on leadership starting with skills gained in the Marine Corps. After 4 years working customer feedback into product development at Western Digital, he teamed up with Centercode to build a better approach to customer testing. He loves to talk about product development and customer feedback. His favorite leadership advice (stolen from the Corps) is “know yourself and seek self-improvement.” About Hannah Flynn Hannah went to The University of Chicago, where she majored in Environmental Studies with a concentration in Economics and Policy. She now works with Aggregage on social media strategy and webinar production for Product Management Today and B2B Marketing Zone. Empowering Product Teams Webinar Series
- 5. 5 Empowering Product Teams Webinar Series 5 Agenda 1 Need for Early Product Feedback 2 The Modern Approach Process 3 Analyzing and Presenting Findings 4 Q&A
- 6. 6 Empowering Product Teams Webinar Series Customer Validation… • First, strategically identify your top product areas that need attention prior to release. • Second, turn user data into strategic priorities that help the right teams make the right decisions.
- 7. 7 Empowering Product Teams Webinar Series Accomplished Through 1 Choosing an Achievable Feedback Goal 2 Organizing Experiences with Topics 3 A Simple Customer Acceptance Survey 4 Categorizing & Prioritizing Results 7
- 8. 8 Empowering Product Teams Webinar Series The Challenging State of Technology Today Highly Sophisticated Software, Hardware, Services Deeply Connected Alexa, Homekit, SSO, APIs... Continuously Evolving OTA, Updates Forever! 30+ Smart Pet Collars Highly Competitive 8
- 9. 9 Empowering Product Teams Webinar Series Targeted Users Market Audience (Demographics) Actual Products Post-Prototype Real Environments Homes, Offices (Technographics) Over Time Weeks or Months Defining Attributes of Customer Validation 9
- 10. 10 Empowering Product Teams Webinar Series Quality Release Customer Validation Lifecycle 10 Launch Confident (110%) All relevant results and recommendations have been implemented. The product is ready to go to market. Test Ready (60-80%) The product has been vetted by professional testing resources and deemed ready for initial usage by end-users. Alpha Beta Near Feature Complete (80-90%) Engineering and Product are confident that most severe issues have been resolved and the product is ready for representative customers. Delta Ongoing Iteration (X.x) The product team continues to evolve and improve the product, focusing testing on new features and quality enhancements
- 11. PROFIT Effective Customer Validation Impact & ROI QUALITY COST DEV TIME INTERNAL BIAS TIME TO MARKET RETURNS SUPPORT COST QUALITY CX/UX REVIEWS CUST SAT PLANNING SALES 11
- 12. 12 Empowering Product Teams Webinar Series Key Customer Validation Challenges 12 Project design and scoping Persistent user engagement Collecting useful feedback Deriving actionable recommendations Allocating qualified resources Recruiting ideal testers
- 13. 13 Empowering Product Teams Webinar Series Quick Question for You Based on Your Experience…
- 14. 14 Empowering Product Teams Webinar Series Fix Improve Promote Primary Deliverable: Actionable Recommendations 14 Bugs, Issues, Failures Feature Requests, Suggestions Highlights, Mini- case Studies
- 15. 15 Empowering Product Teams Webinar Series Process Walkthrough 15
- 16. 16 Empowering Product Teams Webinar Series For me, a Smart Camera Purchased in a box Mobile app Remote connection Think about a product
- 17. 17 Empowering Product Teams Webinar Series Generate and Share Impact Analysis Customer Validation Process Overview 17 Recruitment and Selection; Project KickoffPrep Weekly Planning Test Scoping and Product Mapping Directed Engagement Closure Incentives, Collection, and Wrap-Up Ongoing Feedback Tester Support
- 18. 18 Empowering Product Teams Webinar Series Test Scoping and Product Topic Mapping 18 Topics represent the individual features, functions, and experiences you want tested Product Setup The process of connecting the product for first time use. Find a spot to plug the device into both the network and wall. High M Account Creation The process of creating a personal account to customize the product. Log into www.acme.com and create a new user account. Low S Topic Out of the Box Description The initial experience opening the product package. Activity Receive and unbox the product. Priority Med Size S
- 19. 19 Empowering Product Teams Webinar Series Tester Recruitment 19 US Resident, Own 2+ Smart Home Products Android iOS . Smart Home Interest + Experience Age 18-25 Age 26-35 Age 36-50 . Customer Base + Recruited Core Requirements Technographic Segmentations Additional Traits Demographic Segmentations Executed via a 5-10 Question "Qualification Survey"
- 20. 20 Empowering Product Teams Webinar Series Announce Weekly Topic Activities Engage & Support Testers 1 2 3 4 20 Weekly Engagement Process
- 21. 21 Empowering Product Teams Webinar Series 21 Ongoing Feedback Collection All Ongoing Feedback should be aligned with Topics Test quality, interoperability, and real-world performanceIssues1 Measure acceptance, prioritize backlog, generate new ideasIdeas2 Gather delights, strengths, and real-world usage insights Emulate real-world discussion; drive focused collaborationDiscussion4 Praise3
- 22. 22 Empowering Product Teams Webinar Series Announce Weekly Topic Activities Engage & Support Testers Customer Acceptance Survey 1 2 3 4 22 Weekly Engagement Process
- 23. 23 Empowering Product Teams Webinar Series 1. Please rate your satisfaction of unboxing the product 23 It sounds like you ran into issues that lead to a negative experience. If you haven't already done so, please take a moment to report any of these Issues right now.
- 24. 24 Empowering Product Teams Webinar Series 2. Please rate your satisfaction of setting up the product. 24 It looks like you see room for improvement in this area. If you haven't already done so, please take a moment to share any Ideas that would lead to a higher score.
- 25. 25 Empowering Product Teams Webinar Series 3. Please rate your satisfaction of creating your account. 25 Awesome! Would you please take a moment to share your Praise regarding whatever specifically earned the 5th star in your rating. Thanks!
- 26. 26 Empowering Product Teams Webinar Series Announce Weekly Topic Activities Gather & Present Results Engage & Support Testers Customer Acceptance Survey 1 2 3 4 26 Weekly Engagement Process
- 27. 27 Empowering Product Teams Webinar Series Product, Stakeholder, Tester Readiness Product, Stakeholders, Testers | https://centercode.com/gotime NDA / Participant Agreement https://centercode.com/nda Product Logistics Mobile: https://testfairy.com Introduction to Customer Validation https://centercode.com/cv101 27
- 28. 28 Empowering Product Teams Webinar Series Impact Scoring 28
- 29. 29 Empowering Product Teams Webinar Series Quick Question for You Based on Your Experience…
- 30. 30 Empowering Product Teams Webinar Series Impact Scoring is a prioritization system which considers the relative Weight and Popularity of each piece of tester feedback, based specifically your Objectives. Feedback Impact Scoring 30 i.e. Severity * Frequency
- 31. 31 Empowering Product Teams Webinar Series Severity Frequency Item Example Type Issue Topic Unboxing Severity Critical Status Reproduced Item Duplicates Votes Comments Views
- 32. 32 Empowering Product Teams Webinar Series Issue Idea Praise Weight 2 1 1 Low Medium High Weight 1 2 4 Cosmetic Minor Major Critical Weight 1 2 4 6 Reproduced Weight 2 Feedback Type Topic Area Issue Severity Other Status
- 33. 33 Empowering Product Teams Webinar Series Example Use Case Consider a hardware product, where an antenna is bending during the unboxing process, which results in an issue reported by multiple testers. 33
- 34. 34 Empowering Product Teams Webinar Series Issue Idea Praise 2 1 1 Low Medium High 1 2 4 Cosmetic Minor Major Critical 1 2 4 6 Reproduced 2 Weighted Average = 2.5 (3) Type of Issue Topic Area Issue Severity Other Status
- 35. 35 Empowering Product Teams Webinar Series Frequency Weight Low 1 Med 2 High 4 Very High 6 Adding Issue Frequency Duplicates Votes Comments Views
- 36. 36 Empowering Product Teams Webinar Series Frequency Weight Low 1 Med 2 High 4 Very High 6 Adding Issue Frequency Duplicates Votes Comments Views
- 37. 37 Empowering Product Teams Webinar Series 1 2 3 4 Low (1) 2 3 4 5 Med (2) 3 4 5 6 High (4) 5 6 7 8 Very High (6) 7 8 9 10 Severity Frequency
- 38. 38 Empowering Product Teams Webinar Series 1 2 3 4 Low (1) 2 3 4 5 Med (2) 3 4 5 6 High (4) 5 6 7 8 Very High (6) 7 8 9 10 Severity Frequency Issue Impact = 7
- 39. 39 Empowering Product Teams Webinar Series Severity Frequency Impact
- 40. 40 Empowering Product Teams Webinar Series Leveraging Impact Score Prioritize all tester feedback Identify product problem areas Identify areas of improvement Identify praise-worthy feature sets Benchmark products against each other Identify most impactful testers 40
- 41. 41 Empowering Product Teams Webinar Series Presenting Results 41
- 42. 42 Empowering Product Teams Webinar Series 42 CUSTOMER ACCEPTANCE SCORECARDS Topic Grading Scale The comparison of your topic Customer Acceptance score is based on Centercode's collection of Customer Acceptance Scores across all Beta Tests. See Appendix for more details. OUT OF THE BOX D+68% PRODUCT SETUP A-92% ACCOUNT CREATION B85% Greater than 90% A 82% - 90% B 70% - 81% C 62% - 69% D Less than 62% F Week 1 Topics
- 43. 43 Empowering Product Teams Webinar Series Topic Overview Scorecard Sample 43 D+ 68% Recommendations to FIX 3 TOPIC: OUT OF THE BOX Customer Acceptance Score This score is BELOW AVERAGE. Recommendations to IMPROVE 4 Recommendations to PROMOTE 1 RecommendationsUnboxing Task TASK Review the packaging of the product. DESCRIPTION Review the product packaging and information without opening the box. Get to know the product as if you were considering it prior to purchase.
- 44. 44 Fix Recommendation Sample Tester Quotes SOURCE FEEDBACK BUG-0015 Antenna broke pulling it out of the box Reinforce the Antenna in Packaging Some testers reported frustration with the antenna bending or breaking while removing it from the packaging. A damaged antenna can lead to a poor perception of product quality as well as performance issues. “While trying to remove the product from the box the antenna snapped off. I did not even apply that much pressure.” “I did not realize that the antenna was broken off until I heard it banging around in the box. Very cheaply made. Plus the packaging was so hard to remove, it is no wonder it broke off." 20% RESPONDENTS 7 IMPACT SCORE Critical SEVERITY FIX RECOMMENDATION Pro Tip: Create an “elevator pitch” for each significant issue
- 45. 45 Improve Recommendations Sample Tester Quotes SOURCE FEEDBACK SUG-0041 Twist Ties Were Difficult to unravel “Everything was going great as I pulled stuff out of the box, except the twist ties that held the ethernet cord together were already broken.” “I enjoyed pulling everything out of the box. Things were neat and easily accessible. But, the twist ties around the cords looked like twist ties on bread and took the feel of quality out of most the experience.” 23% RESPONDENTS 4 IMPACT SCORE Minor SEVERITY IMPROVE RECOMMENDATION Provide Higher Quality Twist Ties Testers feel that the twist ties are very cheap compared to the rest of the contents in the packaging. Some testers reported that twist ties were broken upon first inspection. This gives testers the mindset of a product of lesser quality and can result in a poor display.
- 46. 46 Promote Recommendation Sample Tester Quotes SOURCE FEEDBACK PRA-0026 Onboarding was ultra fast “WOW! That was probably the simplest product setup I've ever done. I was up and running in less than 5 minutes." 30% RESPONDENTS 5 IMPACT SCORE PROMOTE RECOMMENDATION Quick, Intuitive Setup and Installation Testers felt the onboarding experience as a whole was very simple and was done in a matter of minutes with no major errors. This should be shared with your marketing team to fuel customer testimonials and product messaging. "I'm pretty impressed with how quickly I was able to get set up. I assumed it was going to take a decent amount of my time, but was pleasantly surprised. Documentation, packaging, pretty much everything was easy to get through. I would expect my mother could pick this thing up and install it. “I couldn't think of a way to make the setup experience any better. I finished just as soon as a started."
- 47. 47 Empowering Product Teams Webinar Series Fix Improve Promote Deliver Actionable Recommendations 47 Engineering, QA, & Support Product Mgmt & UX/CX Marketing & Sales
- 48. 48 Empowering Product Teams Webinar Series Wrapping It Up 48
- 49. 49 Empowering Product Teams Webinar Series Map Product With Topics & Activities 49 Topics represent the individual - Features - Functions - Experiences you want tested Topic Out of the Box Description The initial experience opening the product package. Activity Receive and unbox the product. Priority Med Size S
- 50. 50 Empowering Product Teams Webinar Series Announce Weekly Topic Activities Gather & Present Results Engage & Support Testers Customer Acceptance Survey 1 2 3 4 50 Consistent Weekly Engagement
- 51. 51 Empowering Product Teams Webinar Series Fix Improve Promote Deliver Actionable Recommendations 51 Engineering, QA, & Support Product Mgmt & UX/CX Marketing & Sales
- 52. 52 Empowering Product Teams Webinar Series Goals Recap • Strategically identify your top product areas that need attention prior to release. • Turn user data into strategic priorities that help the right teams make the right decisions.
- 53. 53 Empowering Product Teams Webinar Series Achieve your goals by 1 Choosing an Achievable Feedback Goal 2 Organizing Experiences with Topics 3 Using a Simple Customer Acceptance Survey 4 Prioritizing Results with Satisfaction and Impact Scores 53
- 54. 54 Empowering Product Teams Webinar Series Centercode.com/CV101 Introduction to Customer Validation
- 55. Empowering Product Teams Webinar Series Q&A Hannah Flynn With: Moderated by: VP of Product Marketing, Centercode Linkedin page: /in/jslittle/ Twitter ID: @centercode Website: centercode.com John Little Site editor, Product Management Today Linkedin page: /in/hannahmflynn Twitter ID: @prodmgmttoday Email: hannah@aggregage.com Website: productmanagementtoday.com www.productmanagementtoday.com/webinar-series/empowering-product-teams