Insight Consulting Project
0 likes152 views
The document describes a pipeline developed to analyze textual features on content URLs and their relationship to user engagement. It scrapes URLs, processes text, extracts keywords, and models features to classify engagement as yes/no using logistic regression. Validation randomly splits data into training and test sets to generate precision and recall scores. The pipeline is delivered to the company in Python code to implement, along with extracted top keywords for different ad types.
![Modeling
Attempted linear regression.
Classify engagement as yes/no.
- Features are bags of words from content URL.
Word Count
0 1 2 3 4 5 6 7 8 9 10
Probability[%]
0
0.2
0.4
0.6
0.8
1
Logistic Classification Model
Ad Clicked
Ad Not Clicked
Logistic Classification Model
5 / 11](https://image.slidesharecdn.com/demo3-151008152105-lva1-app6892/85/Insight-Consulting-Project-5-320.jpg)
![Backup
Feature Frequency/Documents
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
RelativeNumberofDocuments[%]
4−
10
3−
10
2−
10
1−
10
1
Ad Type 1Ad Type 1
9 / 11](https://image.slidesharecdn.com/demo3-151008152105-lva1-app6892/85/Insight-Consulting-Project-9-320.jpg)
Insight Consulting Project
- 1. FeatureRank - An Insight Data Science Consulting Project Kuhan Wang October 8th, 2015 1 / 11
- 2. Consulting Scenario Company X wishes to maximize user engagement through optimal placement of advertisements on content URLs. Ad Type: Tourism Keyword: Cuba Keyword: Package Tour Keyword: Airplane Ad Type X Keyword 1 Keyword 2 Keyword 3 Keyword N . . . Example: Tourism ads not ideal on investment content URL. 2 / 11
- 3. A Pipeline to Analyze Textual Features Developed and implemented a pipeline to analyze importance of textual feature on content URLs relative to engagement. Scrape URL Process Text Model Features Extract Keywords Update Keywords Collect Data, Reiterate Begin 3 / 11
- 4. User Engagement Data Occurrences Counts Summary of Engagement Data Page Loaded Ad Viewed Ad Clicked Summary of Engagement Data 4 / 11
- 5. Modeling Attempted linear regression. Classify engagement as yes/no. - Features are bags of words from content URL. Word Count 0 1 2 3 4 5 6 7 8 9 10 Probability[%] 0 0.2 0.4 0.6 0.8 1 Logistic Classification Model Ad Clicked Ad Not Clicked Logistic Classification Model 5 / 11
- 6. Validation Randomly split data into training/test sets. - Generate distribution of validation scores. Precision 0.55 0.6 0.65 0.7 0.75 0.8 0.85 Recall 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 NumberofMCToys 0 5 10 15 20 25 30 Distribution of Precision vs Recall 〉Precision, Recall〈 6 / 11
- 7. Deliverables Extracted keywords: Rank Ad Type 1 Ad Type 2 Ad Type 3 Ad Type 4 1 debt coordinator mortgage gold 2 gift administrative home 0 3 profit minimum procurement stock 4 check minimum wage loan fund 5 balance reports trustee event Pipeline in Python is delivered to company for implementation. 7 / 11
- 8. About Myself PhD Particle Physics, McGill University, researcher on the Large Hadron Collider. Lead the search for black holes and string objects as part of the ATLAS Collaboration. About project and myself at http://kuhanw.zohosites.com/. 8 / 11
- 9. Backup Feature Frequency/Documents 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 RelativeNumberofDocuments[%] 4− 10 3− 10 2− 10 1− 10 1 Ad Type 1Ad Type 1 9 / 11
- 10. 10 / 11
- 11. FeatureRank Kuhan Wang1 1. Insight Data Science October 2, 2015 Abstract FeatureRank is a software tool for extracting correlations between text ngram features and user engagement, thereby optimizing the placement of financial widgets on URL articles. 1 Directory Structure • / processing.py Pre-processing to parse relevant information from engagement csv files. crawl.py A simple web crawler that pulls the title and < p > tag text from URLs. FeatureRank.py Driver file to execute main functions. feature_extraction_model.py The core program that contains the machine learning algorithms. post_processing.py Post processing to produce evaluation metrics and ngram rankings. 11 / 11