Modeling Social Data, Lecture 8: Recommendation Systems

Recommendation Systems
APAM E4990
Modeling Social Data
Jake Hofman
Columbia University
March 13, 2015
Jake Hofman (Columbia University) Recommendation Systems March 13, 2015 1 / 29

Personalized recommendations

http://netﬂixprize.com

http://netﬂixprize.com/rules

http://netﬂixprize.com/faq

Netﬂix prize: results
http://en.wikipedia.org/wiki/Netflix_Prize

Netﬂix prize: results
See [TJB09] and [Kor09] for more gory details.

http://bit.ly/beyond5stars

Recommendation systems
High-level approaches:
• Content-based methods
(e.g., wgenre: thrillers = +2.3, wdirector: coen brothers = +1.7)
• Collaborative methods
(e.g., “Users who liked this also liked”)

Netﬂix prize: data
(userid, movieid, rating, date)

Netﬂix prize: data
(movieid, year, title)

Recommendation systems
High-level approaches:
• Content-based methods
(e.g., wgenre: thrillers = +2.3, wdirector: coen brothers = +1.7)
• Collaborative methods
(e.g., “Users who liked this also liked”)

Collaborative ﬁltering
Memory-based
(e.g., k-nearest neighbors)
Model-based
(e.g., matrix factorization)
http://research.yahoo.com/pub/2859

Problem statement
• Given a set of past ratings Rui that user u gave item i
• Users may explicitly assign ratings, e.g., Rui ∈ [1, 5] is number
of stars for movie rating
• Or we may infer implicit ratings from user actions, e.g.
Rui = 1 if u purchased i; otherwise Rui = ?

Problem statement
• Make recommendations of several forms
• Predict unseen item ratings for a particular user
• Suggest items for a particular user
• Suggest items similar to a particular item
• . . .

Problem statement
• Make recommendations of several forms
• Predict unseen item ratings for a particular user
• Suggest items for a particular user
• Suggest items similar to a particular item
• . . .
• Compare to natural baselines
• Guess global average for item ratings
• Suggest globally popular items

k-nearest neighbors
Key intuition:
Take a local popularity vote amongst “similar” users

k-nearest neighbors
User similarity
Quantify similarity as a function of users’ past ratings, e.g.
• Fraction of items u and v have in common
Suv =
|ru ∩ rv |
|ru ∪ rv |
= i Rui Rvi
i (Rui + Rvi − Rui Rvi )
(1)
Retain top-k most similar neighbors v for each user u

k-nearest neighbors
User similarity
Quantify similarity as a function of users’ past ratings, e.g.
• Angle between rating vectors
Suv =
ru · rv
|ru| |rv |
= i Rui Rvi
i R2
ui j R2
vj
(1)
Retain top-k most similar neighbors v for each user u

k-nearest neighbors
Predicted ratings
Predict unseen ratings ˆRui as a weighted vote over u’s neighbors’
ratings for item i
ˆRui = v Rvi Suv
v Suv
(2)

k-nearest neighbors
Practical notes
We expect most users have nothing in common, so calculate
similarities as:
for each item i:
for all pairs of users u, v that have rated i:
calculate Suv (if not already calculated)

k-nearest neighbors
Practical notes
Alternatively, we can make recommendations using an item-based
approach [LSY03]:
• Compute similarities Sij between all pairs of items
• Predict ratings with a weighted vote ˆRui = j Ruj Sij / j Sij

k-nearest neighbors
Practical notes
Several (relatively) simple ways to scale:
• Sample a subset of ratings for each user (by, e.g., recency)
• Use MinHash to cluster users [DDGR07]
• Distribute calculations with MapReduce

Matrix factorization
Key intuition:
Model item attributes as belonging to a set of unobserved “topics
and user preferences across these “topics”

Linear model
Start with a simple linear model:
ˆRui = b0
global average
+ bu
user bias
+ bi
item bias
(3)

Linear model
For example, we might predict that a harsh critic would score a
popular movie as
ˆRui = 3.6
global average
+ −0.5
user bias
+ 0.8
item bias
(3)
= 3.9 (4)

Low-rank approximation
Add an interaction term:
ˆRui = b0
global average
+ bu
user bias
+ bi
item bias
+ Wui
user-item interaction
(5)
where Wui = pu · qi = k PukQik
• Puk is user u’s preference for topic k
• Qik is item i’s association with topic k

Loss function
Measure quality of model ﬁt with squared-loss:
L =
(u,i)
ˆRui − Rui
2
(6)
=
(u,i)
PQT
ui
− Rui
2
(7)

Optimization
The loss is non-convex in (P, Q), so no global minimum exists
Instead we can optimize L iteratively, e.g.:
• Alternating least squares: update each row of P, holding Q
ﬁxed, and vice-versa
• Stochastic gradient descent: update individual rows pu and qi
for each observed Rui

Alternating least squares
L is convex in rows of P with Q ﬁxed, and Q with P ﬁxed, so
alternate solutions to the normal equations:
pu = Q(u)T
Q(u)
−1
Q(u)T
r(u) (8)
qi = P(i)T
P(i)
−1
P(i)T
r(i) (9)
where:
• Q(u) is the item association matrix restricted to items rated
by user u
• P(i) is the user preference matrix restricted to users that have
rated item i
• r(u) are ratings by user u and r(i) are ratings on item i

Stochastic gradient descent
Alternatively, we can avoid inverting matrices by taking steps in
the direction of the negative gradient for each observed rating:
pu ← pu − η
∂L
∂pu
= pu + Rui − ˆRui qi (10)
qi ← qi − η
∂L
∂qi
= qi + Rui − ˆRui pu (11)
for some step-size η

Practical notes
Several ways to scale:
• Distribute matrix operations with MapReduce [GHNS11]
• Parallelize stochastic gradient descent [ZWSL10]
• Expectation-maximization for pLSI with MapReduce
[DDGR07]

Datasets
• Movielens
http://www.grouplens.org/node/12
• Reddit
http://bit.ly/redditdata
• CU “million songs”
http://labrosa.ee.columbia.edu/millionsong/
• Yahoo Music KDDcup
http://kddcup.yahoo.com/
• AudioScrobbler
http://bit.ly/audioscrobblerdata
• Delicious
http://bit.ly/deliciousdata
• . . .

References I
AS Das, M Datar, A Garg, and S Rajaram.
Google news personalization: scalable online collaborative
filtering.
page 280, 2007.
R Gemulla, PJ Haas, E Nijkamp, and Y Sismanis.
Large-scale matrix factorization with distributed stochastic
gradient descent.
2011.
Yehuda Koren.
The bellkor solution to the netflix grand prize.
pages 1–10, Aug 2009.
G Linden, B Smith, and J York.
Amazon. com recommendations: Item-to-item collaborative
filtering.
IEEE Internet computing, 7(1):76–80, 2003.

References II
A Toscher, M Jahrer, and RM Bell.
The bigchaos solution to the netﬂix grand prize.
2009.
M. Zinkevich, M. Weimer, A. Smola, and L. Li.
Parallelized stochastic gradient descent.
In Neural Information Processing Systems (NIPS), 2010.

Modeling Social Data, Lecture 8: Recommendation Systems

More Related Content

Viewers also liked (20)

Similar to Modeling Social Data, Lecture 8: Recommendation Systems (20)

More from jakehofman (20)

Recently uploaded (20)

Modeling Social Data, Lecture 8: Recommendation Systems