Deep Learning for Recommender Systems - Budapest RecSys Meetup
19 likes7,418 views
1. Deep learning techniques such as convolutional neural networks, recurrent neural networks, and autoencoders can be applied to recommender systems. 2. Convolutional neural networks are commonly used to extract features from images, audio, and video that can then be used for recommendation. Recurrent neural networks can model user sessions as sequences of clicks. 3. Autoencoders learn lower-dimensional representations of items that capture similarities and can be used to make recommendations, especially for cold start problems where little is known about new users or items.
![Convolutional Networks
[Krizhevsky 2012]](https://image.slidesharecdn.com/deeplearningforrecsys-budapestrecsysmeetup-160613084451/85/Deep-Learning-for-Recommender-Systems-Budapest-RecSys-Meetup-27-320.jpg)
![Convolutional Networks
[Faster R-CNN: Ren, He, Girshick, Sun 2015] [Farabet et al., 2012]](https://image.slidesharecdn.com/deeplearningforrecsys-budapestrecsysmeetup-160613084451/85/Deep-Learning-for-Recommender-Systems-Budapest-RecSys-Meetup-28-320.jpg)
![Convolutional Networks
[Faster R-CNN: Ren, He, Girshick, Sun 2015] [Farabet et al., 2012]](https://image.slidesharecdn.com/deeplearningforrecsys-budapestrecsysmeetup-160613084451/85/Deep-Learning-for-Recommender-Systems-Budapest-RecSys-Meetup-29-320.jpg)
![Convolutional Networks
AlexNet [Krizhevsky et al 2014]](https://image.slidesharecdn.com/deeplearningforrecsys-budapestrecsysmeetup-160613084451/85/Deep-Learning-for-Recommender-Systems-Budapest-RecSys-Meetup-38-320.jpg)
![Session-based recommendation
with Recurrent Neural Networks
RNN (GRU) with ranking loss function
ICLR 2016 [B. Hidasi, et.al.]
Treat each user session as sequence of clicks](https://image.slidesharecdn.com/deeplearningforrecsys-budapestrecsysmeetup-160613084451/85/Deep-Learning-for-Recommender-Systems-Budapest-RecSys-Meetup-52-320.jpg)
![Session-based recommendation
with Recurrent Neural Networks
RNN (GRU) with ranking loss function
ICLR 2016 [B. Hidasi, et.al.]
Treat each user session as sequence of clicks](https://image.slidesharecdn.com/deeplearningforrecsys-budapestrecsysmeetup-160613084451/85/Deep-Learning-for-Recommender-Systems-Budapest-RecSys-Meetup-53-320.jpg)
Deep Learning for Recommender Systems - Budapest RecSys Meetup
- 1. Deep Learning for Recommender Systems Alexandros Karatzoglou Senior Research Scientist @ Telefonica Research first.lastname@gmail.com @alexk_z
- 2. Telefonica Research Machine Learning HCI Network & Systems Mobile Computing http://www.tid.es
- 3. Why Deep? ImageNet challenge error rates (red line = human performance)
- 4. Why Deep?
- 5. Inspiration for Neural Learning Early aviation attempts aimed at imitating birds, bats
- 6. Neural Model
- 9. Learning
- 10. Stochastic Gradient Descent Generalization of (Stochastic) Gradient Descent
- 15. Backpropagation
- 16. Backpropagation Does not work well in plain a “normal” multilayer deep network Vanishing Gradients Slow Learning SVM’s easier to train 2nd Neural Winter
- 17. Modern Deep Networks Ingredients: Rectified Linear Activation function a.k.a. ReLu
- 19. Modern Deep Networks Ingredients: Mini-batches: Stochastic Gradient Descent Compute gradient over many (50 -100) data points (minibatch) and update.
- 20. Modern Deep Networks Ingredients: Softmax output:
- 21. Modern Deep Networks Ingredients: Categorical cross entropy loss
- 22. Modern Feedforward Networks Ingredients: Batch Normalization
- 23. Modern Feedforward Networks Ingredients: Adagrad a.k.a. adaptive learning rates
- 28. Convolutional Networks [Faster R-CNN: Ren, He, Girshick, Sun 2015] [Farabet et al., 2012]
- 29. Convolutional Networks [Faster R-CNN: Ren, He, Girshick, Sun 2015] [Farabet et al., 2012]
- 30. Convolutional Networks Self Driving Cars Convolutional example slides from Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 6 75
- 31. Convolutional Networks Standford CS231n: Convolutional Neural Networks for Visual Recognition
- 38. Convolutional Networks AlexNet [Krizhevsky et al 2014]
- 39. d d D-tour → Matrix Factorization 2 5 4 5 4 4 1 5 5 4 1 2 5 2 4 1
- 40. Convolutional Networks for enhancing Collaborative Filtering VBPR: Visual Bayesian Personalized Ranking from Implicit Feedback He, etl AAAI 2015
- 41. Convolutional Networks for Music feature extraction Deep Learning can be used to learn item profiles e.g. music Map audio to lower dimensional space where it can be used directly for recommendation Useful in recommending music from the long tail (not popular) A solution to the cold start problem
- 42. Convolutional Networks for Music feature extraction A. van den Oord, S. Dielmann, B. Schrauwen Deep content- based music recommendation NIPS 2014
- 45. Recurrent Neural Networks Long Short Term Memory
- 47. Recurrent Neural Networks PANDARUS: Alas, I think he shall be come approached and the day When little srain would be attain'd into being never fed, And who is but a chain and subjects of his death, I should not sleep. Second Senator: They are away this miseries, produced upon my soul, Breaking and strongly should be buried, when I perish The earth and thoughts of many states. DUKE VINCENTIO: Well, your wit is in the care of side and that. Second Lord: They would be ruled after this chamber, and my fair nues begun out of the fact, to be conveyed, Whose noble souls I'll have the heart of the wars. Clown: Come, sir, I will make did behold your worship. VIOLA: I'll drink it.
- 52. Session-based recommendation with Recurrent Neural Networks RNN (GRU) with ranking loss function ICLR 2016 [B. Hidasi, et.al.] Treat each user session as sequence of clicks
- 53. Session-based recommendation with Recurrent Neural Networks RNN (GRU) with ranking loss function ICLR 2016 [B. Hidasi, et.al.] Treat each user session as sequence of clicks
- 54. Autoencoders
- 55. Autoencoders
- 56. Autoencoders
- 57. Personalized Autoencoders Collaborative Denoising Auto-Encoders for Top-N Recommender Systems Wu et.al. WSDM 2016
- 58. (Some) Deep Learning Software Theano: Python Library TensorFlow: Python Library Keras: High Level Python Library (Theano &TF) MXNET: R, Python, Julia
- 59. Thanks ● Some slides or parts of slides are taken from other excellent talks and papers on Deep Learning (e.g. Yan Lecun, Andrej Karpathy and other great deep learning researchers)