From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)
Download as PPTX, PDF9 likes1,677 views
The document discusses the process of transforming recommendation algorithms into a commercial product. It describes how research tools needed to be adapted to meet business requirements like robustness, low response times, and easy integration. User interactions were leveraged instead of ratings to provide recommendations. Testing involved A/B tests to increase revenue and measure key performance indicators. The resulting product was tailored to different industries and incorporated contextual factors and business rules.
From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)
- 1. From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience 13.09.2012.
- 2. The kick-start 2 From a toolkit of recommendation algorithms into a real business 13.09.2012.
- 3. Facing with real needs What we had What clients wanted • rating prediction algorithms • recommendations that • coded in various languages bring revenue • blending mechanism • robustness • accuracy oriented • low response time • easy integration • reporting 3 From a toolkit of recommendation algorithms into a real business 13.09.2012.
- 4. What we do? users content of service provider recommender 4 From a toolkit of recommendation algorithms into a real business 13.09.2012.
- 5. Explicit vs implicit feedback No ratings but interactions sparse vs. dense matrix requires different learning 5 From a toolkit of recommendation algorithms into a real business 13.09.2012.
- 6. Increase revenue: A/B tests against the original solution internally 6 From a toolkit of recommendation algorithms into a real business 13.09.2012.
- 7. Robustness Management LAN SNMP Nagios Monitoring HP OpenView Aggregator HTTP HTTP Platform OSS/BSS / SQL / SQL IMPRESS IMPRESS SOAP Application Server #1 Application Server #2 IMPRESS Frontend web server #1 Backend LAN Reco LAN HTTP Load Balancer HTTP(S) Firewall SQL SQL CSV over FTP TV Service LAN IMPRESS Frontend web server #2 Database #1 Database #2 Reporting Subsystem End users 7 From a toolkit of recommendation algorithms into a real business 13.09.2012.
- 8. Time requirements • Response time: few ms (max 200) • Training time: maximum few hours • regular retraining • incremental training • Newsletters: • nightly batch run 8 From a toolkit of recommendation algorithms into a real business 13.09.2012.
- 9. Productization IMPRESS RECO AD•APT for for for IPTV, CATV and satellite e-commerce ad networks and ad server providers Recommends Recommends Recommends Personally Personally Relevant Relevant Personally Relevant products & services ads Linear TV, VOD, catch-up TV and more Gravity personalization platform 9 From a toolkit of recommendation algorithms into a real business 13.09.2012.
- 10. The 5% question – Importance of UI Francisco Martin (Strands): „the algorithm is only 5% in the success of the recommender system” • placement below or above the fold scrolling easy to recognize floating in • title not misleading explanation like • widget carrousel static 10 From a toolkit of recommendation algorithms into a real business 13.09.2012.
- 11. Recommendation scenario Item2Item recommendation logic: the ad’s profile will be matched to the profile model of available ads 11 From a toolkit of recommendation algorithms into a real business 13.09.2012.
- 12. Marketing channels Changing the order of two boxes: 25% CTR increase 12 From a toolkit of recommendation algorithms into a real business 13.09.2012.
- 13. Cannibalization • Goal: increase user engagement • Measurements • average visit length • average page views • Effect of accurate recommendations: • use of listing page ↓ • use of item page ↑ • Overall page view: remains the same • Secondary measurements • Contacting • CTR increase 13 From a toolkit of recommendation algorithms into a real business 13.09.2012.
- 14. Evolution: increased user engagement • not a cold start problem • parameter optimization and user engagement 14 From a toolkit of recommendation algorithms into a real business 13.09.2012.
- 15. KPIs – may change during testing 15 From a toolkit of recommendation algorithms into a real business 13.09.2012.
- 16. Complete personalization: coupon-world • Newsletter (daily + occassionally) • Ranking all offers on the website • top1 item • category preferences • user metadata (gender, age, …) • user category preferences (seldom given) • item metadata • context • customer vs. vendor 16 From a toolkit of recommendation algorithms into a real business 13.09.2012.
- 17. Business rules – driving/overriding ranking 17 From a toolkit of recommendation algorithms into a real business 13.09.2012.
- 18. Contexts 18 From a toolkit of recommendation algorithms into a real business 13.09.2012.
- 19. Context at TV program recommendation • TV (EPG program & video-on-demand) explicit and implicit identification of the user in the household time-dependent recommendation 19 From a toolkit of recommendation algorithms into a real business 13.09.2012.
- 20. (offline) Some results (online) Improvement using season iTALS iTALSx Dataset Recall@20 MAP@20 Recall@20 MAP@20 Grocery 64,31% 137,96% 89,99% 199,82% TV1 14,77% 43,80% 28,66% 85,33% TV2 -7,94% 10,69% 7,77% 14,15% LastFM 96,10% 116,54% 40,98% 254,62% Improvement using Seq iTALS iTALSx Dataset Recall@20 MAP@20 Recall@20 MAP@20 Grocery 84,48% 104,13% 108,83% 122,24% TV1 36,15% 55,07% 26,14% 29,93% 20 From a toolkit of recommendation algorithms into a real business 13.09.2012.
- 21. Anecdotes • Item2item recommendations – bookstore • Placebo effect • buyer vs. seller 21 From a toolkit of recommendation algorithms into a real business 13.09.2012.
- 22. Conclusion • Offline and online testing • From simple to sophisticated • Many more potential fields of application 22 From a toolkit of recommendation algorithms into a real business 13.09.2012.