Stan Gaidar: How far can you go improving user experience with AI?
1 like128 views
The document discusses improving user experience through AI and identifies popular use cases for AI/ML in product management, such as personalized recommendations and predictive analytics. It emphasizes the importance of explainability in AI to build user trust and outlines considerations for building AI solutions versus buying them. Key takeaways include the necessity of personalization, transparency with users, and the critical role of data quality in AI/ML project success.
Stan Gaidar: How far can you go improving user experience with AI?
- 1. How Far Can You Go Improving User Experience With AI? Stanislav Gaidar | Kyiv PMDay 2021
- 2. About Me • Product Manager in a non-product company • Product Strategy, Analytics, UX, AI/ML for business • Startups to Fortune 500 • PM at Crystal Engine by Provectus • Reforge member, Amplitude, GA certified expert linkedin.com/in/stan-gaidar/
- 3. Agenda 1. Popular use cases and how to identify yours 2. What's feasible and what still belongs to science fiction 3. AI/ML Explainability - make your users feel comfortable 4. Okay, I want to build a solution. What to start with and what to expect?
- 4. How can I help you, Username? Popular use cases
- 5. Products, their features and offers inventory. User behavior data, clickstream Customer
- 6. Personalized Customer Activation ‣ Expose product value early on ‣ Increase conversion rate Personalized Next Best Action ‣ Reduce friction ‣ Increase user engagement Personalized Notifications ‣ Build a relationship ‣ Increase click-through rate
- 7. More Use Cases • Recommend Product / Offer / Content Personalization Predictive Analytics • Churn prediction • Conversion/purchase probability • Customer classification • Customer journey insights • Fraud detection • Chat bots • Content moderation • Intelligent search • and more Whatever your fantasy goes
- 9. How To Generate Ideas Acquisition Activation Retention Revenue Referral # of New Customers CAC Conversion Rate Retention Rate Stickiness MAU LTV NPS K-Factor Metrics Stages / Loops Touchpoints Data Ability Triggers Motivation Context User Actions KPI ML Use-Case
- 10. How To Kill Prioritize Ideas Personalization Criteria Use case 1 Use case 2 Use case 3 KPI / ROI Market / Audience Size Data Availability Implementation Complexity
- 12. It works with 99.9% accuracy, right? What's feasible and what's not
- 14. Complex logic required Scalability is a challenge Personalization needed Responsive Can be solved with traditional methods Does not require adapting to new data Requires 100% accuracy Requires full interpretability The Choice Is Yours ML Non-ML (according to Amazon)
- 15. ML Efficiency Environments that are sufficiently regular to be predictable with plenty of data, and were quick feedback is possible Low-validity environments Environments where most crucial events are rare Self-driving cars Long-term investments Startups success Political affairs 70-95% Accuracy Some accidents occur... No better than tossing a coin
- 16. Can one predict behavior of a Product Manager? Business problem: Increase chart sharing and prevent making duplicates Solution: Recommend next best chart with ML Case Study - Amplitude
- 17. Can one predict behavior of a Product Manager? Probably, not yet. Case Study - Amplitude
- 18. Is this app really smart?! AI/ML Explainability
- 19. Explainability for Consumers As a consumer I want to know: 1. Why the system recommends me this 2. What the system has learned about my profile
- 20. Transparency Builds Trust • Explain to your users what stands behind the AI-powered UI elements. • Give users control, so they'll feel safe. • Ask for feedback — e.g. flag incorrect recommendation. This will improve your models. • Being open with users cushions the requirements for model precision.
- 21. Let It Learn Sr. Product Manager s.gaidar@provectus.com Sr. Product Manager s.gaidar@provectus.com Sr. Product Manager s.gaidar@provectus.com
- 22. Explainability for Business Churn Prediction
- 23. Explainability for Business Text classification: Medical / Non-medical Automated content moderation
- 24. Okay, let's build one! Should I hire a data scientist?
- 25. Build vs Buy SaaS, low-code platforms ✓ Intellectual property ✓ No extra fees ✓ High quality of predictions for specialized data ✓ Rigorous testing scenario ✓ More control over the product - Upfront cost - Time - Infrastructure requirements ✓ Low development cost ✓ Quick deploy ✓ High quality of predictions for generic data cases ✓ Hands-off management - Data customization - Lower quality of outcome for specific data cases - Costs rising with system growing - Vendor lock Ready-to-use Custom-built in-house, outsourced
- 26. Roadmap
- 28. People PM Software Engineer ML Engineer Product Designer Data PO Data Engineer DevOps
- 29. Takeaways 1. Augment your customer journey map with a data layer to generate ideas. 2. Personalization is already a must-have. 3. Be transparent with users to build trust. AI/ML explainability doesn't come by default. 3. ML code is just a small piece of the project. 4. AI/ML project success depends mostly on data quality and availability, no matter you buy it or build.
- 30. Thank you for your time! Let's talk now