Practical Recommendation System - Scalable Machine Learning

Sonpvh – VNG.R&D.12.2017
Zalo.Recommendation.System.Review

1. Introduction
2. Goals of Recommendation system
3. The high level architecture of Recommender system
4. Basic models of recommender Systems
5. Experiences
6. Q&A
1

BIG
4
Xavier 2014 [1]
• 80M Users
• A Thousand of shops, articles, OA, Music …
• >20K product
• Billion interactions times
• …

VALUE
5
Xavier 2014 [1]
ZingMp3: >30%
traffic
ZOA: improve
>30% total click
and follow

PERSONAL
EXPERIENCES
6
 “Mining” the user’s preferences
 Improve the user experiences

8
1. Prediction version of problem (Matrix completion problem):
1. m users, n items, r ratings: (m x n) matrix
2. Given specified values for training, then predict missing rating values
2. Ranking version of problem (top-k recommendation problem)
1. Recommend the top-k items/users for a particular user/item
Charu 2016 [2]

9
1. The ultimate goal is Increasing PROFIT
1. Direct: Increasing product sales, total click (product …), total time spend on pages (music,
news…)
2. Indirect: Increasing total connection (Facebook).
3. The common operational and technical goals is:
 Relevance
 Novelty
 Serendipity
 Diversity
Charu 2016 [2]

14
Bogers, T., & Van
Den Bosch [6]

18
1. Architecture: pipeline for big data
1. Ex: Mp3 - daily (1M user x 100k song), 50G App + 8G Web log
2. Streaming, batch
2. Algorithm: Scalable
1. This is not Machine Learning, this is Scalable Machine Learning
2. Combine multi-model
3. Monitor system
4. Evaluation

19
Machine
learning
Online
Evaluatio
n
Offline
Evaluatio
n
80%-90% Time
“garbage in garbage out”

20
Cron.d
LuigiZDB
(3) Extract Column
(4) Union ...
(1) call Luigi(5) call java
(6)Streamdata
(7) upload DB
(8)visualize

22
1. Memory-based methods (neighborhood-
based CF)
1. User-based CF
2. Item-based CF
2. Model-based methods: predictive models
1. Decision trees
2. Rule-based model
3. Bayesian methods
4. Latent factor models
5. Clustering
6. LDA
7. SVD/MF
8. Deep learning
9. … Charu 2016 [2]

23
1. Recommend similar items based on their “description”
 Required to analyzing the content of items which user has rated in the past
 Usually keyword, prices..
2. Knowledge-based recommender systems
1. Useful in the context of item are not purchased often (house, car … )
2. Useful for search engine, conversational systems, navigation-based recommender
3. Demographic Recommender System

24
1. User preferences changes …
2. Context aware:
1. Location-based recommender systems
2. Time-sensitive recommender systems
3. Social recommender system => social network analysis
3. Features learning

25
1. Trustworthy Recommender system
2. Social feedback analysis
3. Group recommender systems
4. Multi-Criteria Recommender system
5. Active learning
6. Privacy in recommender system

27
Machine
learning
Online
Evaluatio
n
Offline
Evaluatio
n
1. Log 2. Feature 3. Indexes
Chart …
4. Model
6. Indexes
5. Problems
What do they want?

28
1. Advertisement (ZaloFeed)
2. Search engine (ZingMp3)
3. Next Song (ZingMp3)
4. Personal playlist (ZingMp3)
5. Personal suggestion (OA, Product)
6. Trade-off
1. Accuracy vs Diversity (short term vs long term)
2. Discovery vs Continuation
3. Freshness vs stability
4. Efficiency vs Accuracy

29
1. Business KPI – Indexes
2. Casual Effect Measurement
3. User behavior Analysis

30
1. Candidate generation or ranking prediction
2. Classification or CF
3. Cost function?
4. Performance: Time, Accuracy …

31Andrew NG [7]
1. Explaining recommendation system - Yifan[10]
2. Complex model is good?
3. Deep learning vs Boosting

32
Machine
learning
Online
Evaluatio
n
Offline
Evaluatio
n
1. Log 2. Feature 3. Indexes
Chart …
4. Model
6. Indexes
5. Problems
What do they want?

33
1. What kind of Rating?
1. Explicit or Implicit
2. Preferences vs confidences
3. Impression or No negative feedback
4. How to treat the missing data?
5. Appropriate evaluation measurement
1. Youtube: total time listen [11]
2. [10]
3. …
Yifan Hu[10]
𝑟𝑎𝑛𝑘

34
1. People laws and rich-get-richer phenomena [13]
2. The effect of long tail – stanford [12]

35
1. Take care about bias
2. Take care about data snooping

37
1. Keep moving and think about average
2. Monitoring

38
1. Pick the right features really help
1. Spotify: same playlist
2. Youtube: pair count
3. Tencent: co-items
4. ZingMp3:  same playlist, co-items

41
1. This is not Machine Learning, this is Scalable Machine Learning
2. Build a robust pipeline for big data
3. Monitoring – control the data qualification
4. Evaluation – user behaviors and casual effect
5. Define indexes and feature are really important
6. Keep in touch with product man and log man
7. Start with simple, improve with patient
8. …

42
ZingMp3: >30%
traffic
ZOA: improve
>30% total click
and follow
Product and personal
playlist is coming soon

1. https://www.slideshare.net/xamat/recommender-systems-machine-learning-
summer-school-2014-cmu
2. Recommender System The Text Book – Charu C. Aggarwal
3. Amatriain, X. (2013). Mining large streams of user data for personalized
recommendations. ACM SIGKDD Explorations Newsletter, 14(2), 37.
https://doi.org/10.1145/2481244.2481250
4. Amatriain, X. (2013). Big & personal: data and models behind netflix
recommendations. Proceedings of the 2nd International Workshop on Big Data,
Streams and Heterogeneous Source Mining Algorithms, Systems, Programming
Models and Applications - BigMine ’13, 1–6.
https://doi.org/10.1145/2501221.2501222
5. https://buildingrecommenders.wordpress.com/2016/10/10/mendeley-suggest-
architecture/
6. Bogers, T., & Van Den Bosch, A. (2009). Collaborative and content-based
filtering for item recommendation on social bookmarking websites. CEUR
Workshop Proceedings (Vol. 532). https://doi.org/10.1007/978-0-387-85820-3
7. Andrew NG https://www.youtube.com/watch?v=n1ViNeWhC24 45

8. https://hackernoon.com/spotifys-discover-weekly-how-machine-learning-finds-
your-new-music-19a41ab76efe
9. TencentRec: Real-time Stream Recommendation in Practice
10. Collaborative filtering for implicit Feedback Datasets – Yifan Hu
11. https://research.google.com/pubs/pub45530.html -youtube
12. http://bid.berkeley.edu/cs294-1-spring13/index.php/About_People - Stanford
13. http://www.cs.cornell.edu/home/kleinber/networks-book/networks-book-ch18.pdf
46

Practical Recommendation System - Scalable Machine Learning

More Related Content

Similar to Practical Recommendation System - Scalable Machine Learning (20)

Recently uploaded (20)

Practical Recommendation System - Scalable Machine Learning