Collaborative filtering for recommendation systems in Python, Nicolas Hug
11 likes8,124 views
This document provides an introduction and overview of collaborative filtering in Python using the Surprise library. It begins with a taxonomy of recommendation system approaches, focusing on collaborative filtering which leverages user-item interactions. It then describes the neighborhood method of collaborative filtering, including computing similarity measures between users/items and aggregating neighbors' ratings. The document introduces the Surprise library, explaining its features like built-in algorithms, dataset handling, and evaluation tools. It provides code examples for basic usage, implementing custom algorithms, and evaluating models through cross-validation.
![LET'S CODE!
def predict_rating(u, i):
'''Return estimated rating of user u for item i.
rating_hist is a list of tuples (user, item, rating)'''
# Retrieve users having rated i
neighbors = [(sim[u, v], r_vj)
for (v, j, r_vj) in rating_hist if (i == j)]
# Sort them by similarity with u
neighbors.sort(key=lambda tple: tple[0], reversed=True)
# Compute weighted average of the k-NN's ratings
num = sum(sim_uv * r_vi for (sim_uv, r_vi) in neighbors[:k])
denum = sum(sim_uv for (sim_uv, _) in neighbors[:k])
return num / denum
14](https://image.slidesharecdn.com/3-170615114205/85/Collaborative-filtering-for-recommendation-systems-in-Python-Nicolas-Hug-26-320.jpg)
![LET'S CODE!
def predict_rating(u, i):
'''Return estimated rating of user u for item i.
rating_hist is a list of tuples (user, item, rating)'''
# Retrieve users having rated i
neighbors = [(sim[u, v], r_vj)
for (v, j, r_vj) in rating_hist if (i == j)]
# Sort them by similarity with u
neighbors.sort(key=lambda tple: tple[0], reversed=True)
# Compute weighted average of the k-NN's ratings
num = sum(sim_uv * r_vi for (sim_uv, r_vi) in neighbors[:k])
denum = sum(sim_uv for (sim_uv, _) in neighbors[:k])
return num / denum
14](https://image.slidesharecdn.com/3-170615114205/85/Collaborative-filtering-for-recommendation-systems-in-Python-Nicolas-Hug-27-320.jpg)
![DATASET HANDLING
# Built-in datasets (movielens, Jester)
data = Dataset.load_builtin('ml-100k')
# Custom datasets (from file)
file_path = os.path.expanduser('./my_data')
reader = Reader(line_format='user item rating timestamp',
sep='t')
data = Dataset.load_from_file(file_path, reader=reader)
# Custom datasets (from pandas dataframe)
reader = Reader(rating_scale=(1, 5))
data = Dataset.load_from_df(df[['uid', 'iid', 'r_ui']],
reader)
24](https://image.slidesharecdn.com/3-170615114205/85/Collaborative-filtering-for-recommendation-systems-in-Python-Nicolas-Hug-48-320.jpg)
![CUSTOM PREDICTION ALGORITHM
class MyRatingPredictor(AlgoBase):
def train(self, trainset):
'''Fit your data here.'''
self.mean = np.mean([r_ui for (u, i, r_ui)
in trainset.all_ratings()])
def estimate(self, u, i):
return self.mean
29](https://image.slidesharecdn.com/3-170615114205/85/Collaborative-filtering-for-recommendation-systems-in-Python-Nicolas-Hug-53-320.jpg)
![CUSTOM PREDICTION ALGORITHM
class MyRatingPredictor(AlgoBase):
def train(self, trainset):
'''Fit your data here.'''
self.mean = np.mean([r_ui for (u, i, r_ui)
in trainset.all_ratings()])
def estimate(self, u, i):
return self.mean
trainset object:
Iterators over the rating history (user-wise, item-wise, or unordered)
Iterators over users and items ids
Other useful methods/attributes
29](https://image.slidesharecdn.com/3-170615114205/85/Collaborative-filtering-for-recommendation-systems-in-Python-Nicolas-Hug-54-320.jpg)
![class MyKNN(AlgoBase):
def train(self, trainset):
self.trainset = trainset
self.sim = ... # Compute similarity matrix
def estimate(self, u, i):
neighbors = [(self.sim[u, v], r_vi) for (v, r_vj)
in self.trainset.ur[u]]
neighbors = sorted(neighbors,
key=lambda tple: tple[0],
reverse=True)
sum_sim = sum_ratings = 0
for (sim_uv, r_vi) in neighbors[:self.k]:
sum_sim += sim
sum_ratings += sim * r
return sum_ratings / sum_sim
30](https://image.slidesharecdn.com/3-170615114205/85/Collaborative-filtering-for-recommendation-systems-in-Python-Nicolas-Hug-55-320.jpg)
![GRID-SEARCH
param_grid = {'n_epochs': [5, 10], 'lr_all': [0.002, 0.005],
'reg_all': [0.4, 0.6]}
grid_search = GridSearch(SVD, param_grid,
measures=['RMSE', 'FCP'])
data = Dataset.load_builtin('ml-100k')
data.split(n_folds=3)
grid_search.evaluate(data) # will evaluate each combination
print(grid_search.best_score['RMSE'])
print(grid_search.best_params['RMSE'])
algo = grid_search.best_estimator['RMSE'])
34](https://image.slidesharecdn.com/3-170615114205/85/Collaborative-filtering-for-recommendation-systems-in-Python-Nicolas-Hug-61-320.jpg)
![MODEL PERSISTENCE
Can also dump the predictions to analyze and
carefully compare algorithms with pandas
# ... grid search
algo = grid_search.best_estimator['RMSE'])
# dump algorithm
file_name = os.path.expanduser('~/dump_file')
dump.dump(file_name, algo=algo)
# ...
# Close Python, go to bed
# ...
# Wake up, reload algorithm, profit
_, algo = dump.load(file_name)
38](https://image.slidesharecdn.com/3-170615114205/85/Collaborative-filtering-for-recommendation-systems-in-Python-Nicolas-Hug-65-320.jpg)
![MINIMIZATION BY (STOCHASTIC) GRADIENT DESCENT
Our parameter is for all and . The function to be minimized is:
def compute_SVD():
'''Fit pu and qi to known ratings by SGD'''
p = np.random.normal(size=F)
q = np.random.normal(size=F)
for iter in range(n_max_iter):
for u, i, r_ui in rating_hist:
err = r_ui - np.dot(p[u], q[i])
p[u] = p[u] + learning_rate * err * q[i]
q[i] = q[i] + learning_rate * err * p[u]
def estimate_rating(u, i):
return np.dot(p[u], q[i])
53](https://image.slidesharecdn.com/3-170615114205/85/Collaborative-filtering-for-recommendation-systems-in-Python-Nicolas-Hug-91-320.jpg)
Collaborative filtering for recommendation systems in Python, Nicolas Hug
- 10. OUTLINE 1. THE NEIGHBORHOOD METHOD (K-NN) 2. OVERVIEW OF ( ) SURPRISE http://surpriselib.com 8
- 11. OUTLINE 1. THE NEIGHBORHOOD METHOD (K-NN) 2. OVERVIEW OF ( ) SURPRISE http://surpriselib.com 3. MATRIX FACTORIZATION 8
- 39. OUTLINE 1. THE NEIGHBORHOOD METHOD (K-NN) 2. OVERVIEW OF ( ) 3. MATRIX FACTORIZATION SURPRISE http://surpriselib.com 17
- 57. TYPICAL EVALUATION PROTOCOL Hide some of the (they become ) Use the remaining to predict the 32
- 58. TYPICAL EVALUATION PROTOCOL Hide some of the (they become ) Use the remaining to predict the RMSE = The average error, where big errors are heavily penalized (squared) 32
- 59. TYPICAL EVALUATION PROTOCOL Hide some of the (they become ) Use the remaining to predict the RMSE = The average error, where big errors are heavily penalized (squared) Do that many times with different subsets: this is cross-validation 32
- 66. OUTLINE 1. THE NEIGHBORHOOD METHOD (K-NN) 2. OVERVIEW OF ( ) 3. MATRIX FACTORIZATION SURPRISE http://surpriselib.com 39
- 70. BEFORE SVD: PCA Here are 400 greyscale images (64 x 64): Put them in a 400 x 4096 matrix : 42
- 71. PCA also gives you the . In advance: you don't need all the 400 guys BEFORE SVD: PCA PCA will reveal 400 of those creepy typical guys: These guys can build back all of the original faces 43
- 75. PCA ON A RATING MATRIX? SURE! Assume all ratings are known. Transpose the matrix Exact same thing! PCA will reveal typical movies. Note: in practice, the factors semantic is not clearly defined. 45
- 77. SVD IS PCA2 PCA on gives you the typical users PCA on gives you the typical movies SVD gives you both in one shot! is diagonal, it's just a scaler. This is our matrix factorization! 46
- 82. SO HOW TO COMPUTE AND ? Columns of are the eigenvectors of Columns of are the eigenvectors of Associated eigenvalues make up the diagonal of There are very efficient algorithms in the wild 48
- 83. SO HOW TO COMPUTE AND ? Columns of are the eigenvectors of Columns of are the eigenvectors of Associated eigenvalues make up the diagonal of There are very efficient algorithms in the wild Alternate option: find the s and the s that minimize the reconstruction error (With some orthogonality constraints). There are also very efficient algorithms in the wild 48
- 84. SO HOW TO COMPUTE AND ? Columns of are the eigenvectors of Columns of are the eigenvectors of Associated eigenvalues make up the diagonal of There are very efficient algorithms in the wild Alternate option: find the s and the s that minimize the reconstruction error (With some orthogonality constraints). There are also very efficient algorithms in the wild So, piece of cake right? 48
- 85. OOPS! 49
- 86. WE ASSUMED THAT WAS DENSE But it's not! 99% are missing and are not even defined So and are not defined either There is no SVD 50
- 87. WE ASSUMED THAT WAS DENSE But it's not! 99% are missing and are not even defined So and are not defined either There is no SVD Two options: Fill the missing entries with a simple heuristic " Let's just not give a damn " -- Simon Funk (ended- up top-3 of the Netflix Prize for some time) 50
- 88. Dense case: find the s and the s that minimize the total reconstruction error (With orthogonality constraints) Sparse case: find the s and the s that minimize the partial reconstruction error (Forget about orthogonality) APPROXIMATION 51
- 89. Dense case: find the s and the s that minimize the total reconstruction error (With orthogonality constraints) Sparse case: find the s and the s that minimize the partial reconstruction error (Forget about orthogonality) APPROXIMATION Basically the same, except we don't have all the ratings (and we don't care) 51
- 91. MINIMIZATION BY (STOCHASTIC) GRADIENT DESCENT Our parameter is for all and . The function to be minimized is: def compute_SVD(): '''Fit pu and qi to known ratings by SGD''' p = np.random.normal(size=F) q = np.random.normal(size=F) for iter in range(n_max_iter): for u, i, r_ui in rating_hist: err = r_ui - np.dot(p[u], q[i]) p[u] = p[u] + learning_rate * err * q[i] q[i] = q[i] + learning_rate * err * p[u] def estimate_rating(u, i): return np.dot(p[u], q[i]) 53
- 95. SOME REFERENCES Can't recommend enough (pun intended) Aggarwal's Recommender Systems - The Textbook Jeremy Kun's (great insights on and )blog PCA SVD 56
- 96. THANKS! 57