Predicting Pre-click Quality for Native Advertisements
6 likes31,178 views
The document discusses measuring and modeling pre-click ad quality, emphasizing the importance of factors like aesthetic appeal, relevance, and trustworthiness over traditional metrics such as click-through rate (CTR). It introduces offensive feedback rate (OFR) as a more effective metric for identifying bad ad quality and provides a methodology for evaluating ad preferences. The findings include insights on engineering ad quality features and the efficacy of models based on user interactions and ad characteristics.
Predicting Pre-click Quality for Native Advertisements
- 2. Facebook Suggested Post Twi3er Promoted Tweet Yahoo Sponsored Content Na#ve adver#sing
- 3. Offensive ads disengage the users! D. G. Goldstein, R. P. McAfee, and S. Suri. The cost of annoying ads. WWW 2013. A. Goldfarb and C. Tucker. Online display adver#sing: Targe#ng and obtrusiveness. MarkeIng Science 2011.
- 4. • How to measure? • What makes an ad preferred by users? • How to model? Pre-‐click ad quality
- 5. • How to measure? • What makes an ad preferred by users? • How to model? Pre-‐click ad quality
- 6. How to measure the pre-‐click quality? • Is CTR (click-‐through rate) a good pre-‐click metric? – A compounding metric: • Relevance: how ads match user interests. • Quality: nature of the ad product and ad creaIve design decision. • Pre-‐click metrics solely measure on ad quality? – Let us elicit from the users (crowdsourcing)
- 9. CTR vs. Offensiveness (OFR) Bad ads a&rac)ng clicks (clickbaits?) • Correlation between CTR and OFR (very weak) – Spearman: 0.155 – Pearson: -0.043 • Quantile analysis – High OFR distribute across ads with various CTR – Higher CTR more ads with higher OFR
- 10. • How to measure? • What makes an ad preferred by users? • How to model? Pre-‐click ad quality
- 11. What makes an ad preferred by users? ● Methodology ○ Pair-‐wise ad preference + reasons ○ Sample ads with various CTR (whole quality spectrum) ○ Quality based comparison within category (verIcal) ● Underlying preference reasons ○ Aesthe#c appeal > Product, Brand, Trustworthiness > Clarity > Layout ○ VerIcal Differences personal finance (clarity) beauty and educaIon (product) within verIcal comparison
- 12. Can we engineer ad quality features? brand readability, senIment aestheIc, visual User Reasons Engineerable Ad Crea#ve Features Brand Brand (domain pagerank, search term popularity) Product/Service Content (YCT, adult detector, image objects) Trustworthiness Psychology (senIment, psychological incenIves) Content Coherence (similarity between Itle and desc) Language Style (formality, punctuaIon, superlaIve) Language Usage (spam, hatespeech, click bait) Clarity Readability (Flesch reading ease, num of complex words) Layout Readability (num of sentences, words) Image ComposiIon (Presence of objects, symmetry) Aesthe;c appeal Colors (H.S.V, Contrast, Pleasure) Textures (GLCM properIes) Photographic Quality (JPEG quality, sharpness) ○ By mining ad copy (Itle and descripIon), image and adverIser informaIon ○ Cold-‐start features
- 13. We also use historical features User Behavior Engineerable Features Click CTR (click-‐through rate) Post-‐click Bounce Rate Average Dwell Time We mine user interacIons with the ads
- 14. Feature correla#on with OFR The offensive ads tend to: start with number maintain lower image JPEG quality be less formal express negaIve senIment in the ad Itle
- 15. • How to measure? • What makes an ad preferred by users? • How to model? Pre-‐click ad quality
- 16. Data NaIve mobile iOS and Android app 28,664 ads (Sampled from March 01-‐18, 2015) Ad feedback data obtained from Yahoo news stream Classifier Logis;c Regression as a binary classifier posiIve examples: high quanIle of OFR ads negaIve examples: all others EvaluaIon 5-‐fold Cross-‐validaIon Metric: AUC (Area Under the ROC Curve) Pre-‐click model: Data and evalua#on brand readability, sentiment aesthetic, visual
- 17. Overview of model performance Models based on each feature category: product > trustworthiness > brand > aestheIc appeal > clarity > layout Model summary: • cold start: AUC (0.77) • User behavior: AUC (0.70) • cold start + user behavior: AUC (0.79)
- 18. A/B Tes#ng online evalua#on • Baseline System – Score(ad) = bid * pCTR • Pre-‐click Quality System – Eliminate the ad from ad ranking if P(Offensive|ad) > 𝛿 – 𝛿 is determined by other constraints (e.g. eCPM) Mobile: OFR (-‐17.6%) Desktop: OFR (-‐8.7%)
- 19. Take-‐away messages • How to measure pre-‐click ad quality? – Offensive feedback rate as a metric – Capture bad quality be3er than CTR • What makes an ad preferred by users (reasons)? – AestheIc appeal > Product, Brand, Trustworthiness > Clarity > Layout • How to model? – Mining ad copy features from ad text, image and adverIser – EffecIve in the predicIon
- 20. Ques#ons? Ad feedback Offensive Feedback Rate vs. CTR brand readability, senIment aestheIc, visual PredicIve model by mining ad features