Iterative Methodology for Personalization Models Optimization
0 likes83 views
The document outlines an iterative methodology for optimizing personalization models using user profile data, emphasizing the importance of user profiling for personalization, lookalikes modeling, and interest targeting. It details a profile lab process involving offline and online profile transformations, testing, and model performance evaluation. Additionally, it discusses embedding techniques for enhancing machine learning models and clustering approaches to improve data integration and user segmentation.
![Profile Lab - Advanced Flow (WIP)
Uuid, click, adid, document ids of interactions
11233455, true, 99837377, [11234342, 13424234, 3254534]
56456546, false, 3434888, [11234342, 23432444, 1213333, 23432423]
34564363, true, 11113333,[35245555, 463321111, 19938222]
…….](https://image.slidesharecdn.com/newdatascienceguild-november2018-181107212233/85/Iterative-Methodology-for-Personalization-Models-Optimization-54-320.jpg)
Iterative Methodology for Personalization Models Optimization
- 1. Iterative Methodology for Personalization Models Optimization
- 10. 1. 2. 3.
- 15. - -
- 16. User Profile Lab Model Learning Framework
- 18. 1. 2.
- 19. 1.
- 20. 1.
- 21. 1.
- 22. → 1.
- 23. 1. 2. a. b.
- 24. 1 2 4 5 3
- 25. 1 2 3 4 5
- 28. 1 4
- 29. - Listings - Real Listings - Clicks - User Profile - Ad Profile 1
- 30. 1
- 31. 4
- 32. 4 1 2
- 33. 4 1 2
- 34. 4 1 2
- 35. 4 1 2
- 37. =
- 38. <
- 40. User Profile Lab Model Learning Framework
- 42. Why User Profile Is Important ● Personalization ● Lookalikes modeling ● Interest Targeting ● and more….
- 43. User Profile Data Model DocId Timestamp Feature Confidence 12 0 Sport Cars 1 42 -1 Sport Cars 1 43 -21 World Cup 1 55 -21 World Cup 1 Offline Profile
- 44. User Profile Data Model DocId TS Feature Co nf 12 0 Sport Cars 1 42 -1 Sport Cars 1 43 -21 World Cup 1 55 -21 World Cup 1 Category Conf Sports Cats 2 Soccer 2 Serving ProfileOffline Profile
- 45. User Profile - Boost Recent X2 DocId TS Feature Co nf 12 0 Sport Cars 1 42 -1 Sport Cars 1 43 -21 World Cup 1 55 -21 World Cup 1 Category Conf Sports Cats 2 Soccer 1 Serving ProfileOffline Profile
- 46. Motivation - User Profile Tweaks ● Is this hypothesis true? ● What is the decay schema? ● Linear in time? ● Exponential in time? ● Potentially many trial & error cycles
- 47. Profile Lab - Basic Flow ● Static dataset of offline profiles ● Sequence of docids for each user ● Static feature mapping all docids ● Lean algo block needed to be implemented ● Transform offline profile to online ● Apply algo piece to generate online profile ● Generate KPIs
- 48. Profile Lab - Cross User KPI
- 49. Profile Lab - Cross User KPI
- 50. Profile Lab - Cross User KPI
- 51. Profile Lab - Cross User KPI
- 52. Profile Lab - Cross User KPI
- 53. Profile Lab - Cross User KPI ● Run cross user test 20K time ● Average and normalize result frequencies
- 54. Profile Lab - Advanced Flow (WIP) Uuid, click, adid, document ids of interactions 11233455, true, 99837377, [11234342, 13424234, 3254534] 56456546, false, 3434888, [11234342, 23432444, 1213333, 23432423] 34564363, true, 11113333,[35245555, 463321111, 19938222] …….
- 55. Profile Lab - Advanced Flow (WIP) ● Static supervised datasets of offline profiles ● Perso - with click ● LAL - in/out segment ● Model is built ● Model performance KPIs (auc etc)
- 57. Motivation: Named Entities & Wikitags
- 58. Motivation: Named Entities & Wikitags ● Very high cardinality ~300-400K ● Precise user taste ● Big potential in perso ● Big money for user segmentation ● Hard to leverage as is
- 59. Machine Learning High Applied Statistics High Student Loans Medium Teaching Medium Technical Tutorial High Recurrent Neural Nets High Andrew Ng High Motivation: Hard To Leverage
- 60. Gender Royal Age Tech Rich Prince Harry 1 0.9 -0.05 0.3 0.9 Queen Elizabeth -1 0.99 0.9 -0.8 0.9 Apple inc. 0.1 -0.03 0.5 0.9 0.9 Machine Learning Students -0.7 0.02 -0.5 0.8 -0.6 Facebook 0 0.01 0.2 0.9 0.87 Embeddings: Dense Representation
- 61. Embeddings: Dense Representation ● Given a high confidence concept in a doc ● Context is other concepts ● Lots of training data in our DocStore ● Many existing libraries: word2vec, glove, starspace etc ● Good embedding model
- 62. Machine Learning High Applied Statistics High Student Loans Medium Teaching Medium Technical Tutorial High Recurrent Neural Nets High Andrew Ng High Embeddings Based Models
- 63. Embeddings Based Models ● Major change in prod model architecture ● High dev costs ● Potential issue with Elastic Static embedding cluster is a good fallback
- 64. Clustering Phase ● Cluster embedding vectors ● Cluster id = doc feature ● Concept vector => cluster id ● Easy integration with current architecture
- 65. Clustering - Many Hyperparameters ● Train embedding model : |D| docs, |C| coordinates ● Quick sanity over embeddings model ● Select most frequent N concepts ● Apply sk-learn clustering analysis method A ● Benchmark clusters using common metrics ● Qualitative cluster analysis look good? ● No: Try different |D|, |C|, N, A ● Yes: Implement test and run on lab
- 66. Clustering - Many Hyperparameters ● Starspace embeddings / Word2Vec ● Embedding Dim - 50, 100, 300 ● Number of Clusters - 1000, 2000, 5000, 10000 ● Clustering Algorithms - k-means, DB-Scan
- 67. Clustering Phase ● Pakistani Cricketers ● Philipino Celebrities ● Norwegian Politics ● Badminton ● Potato Dishes ● Spanish Football
- 68. Clustering Phase
- 69. Clustering Phase
- 70. Results - WEC vs WikiTags First WEC model
- 71. ● Word2Vec on 3 days data ● Only features which conf > 0.3 ● No long tail clustering (freq>100) Results - WEC vs Categories
- 72. Training with Modeling Framework
- 73. Training with Modeling Framework
- 74. Thank You