SlideShare a Scribd company logo
Bug Report, Feature Request, or Simply
Praise? On Automatically Classifying App
Reviews
Authors: Walid Maalej and Hader Nabil
IEEE RE - 2015
Presented by:
Oğuzhan Çalıkkasap
App Market
• Apple App Store
• Google Play Store
https://www.statista.com/statistics/276623/number-of-apps-available-in-leading-app-stores/
2
Motivation
• Better reviews, higher sales
• Rich source
Information about bugs
Ideas for new features
• Benefitted by
App vendors
Developers
3
Purpose
• Classify reviews into four categories
Bug reports
Feature requests
User experiences
Ratings
4
Paper Outline
• Introduce probabilistic techniques for review classification
• Compare accuracy of the review classification techniques
• Derive insights into review analytics tool design
5
Review Classification Techniques
1. String matching
2. Bag of Words
3. Text Preprocessing
4. Review Metadata
5. Sentiment Analysis
6. Binary and Multiclass Classifiers
6
Research Method and Data
• Collect reviews from app store, extract their metadata
• Manually label reviews and create a truth set
• Implement each classifier at different parts of truth set
• Evaluate classifiers’ accuracies
• Data: Review text, title, app name, category, store, submission
date, star rating
7
Evaluation Data 8
Evaluation Metrics
• Precision: Fraction of reviews that are classified correctly for type i
• Recall: Fraction of reviews of type i which are classified correctly
• F1: Harmonic mean
9
𝑇𝑃𝑖
𝑇𝑃𝑖 + 𝐹𝑃𝑖
𝑇𝑃𝑖
𝑇𝑃𝑖 + 𝐹𝑁𝑖
Research Results - Features 10
Research Results - Classifiers 11
Study Outcomes
• Probabilistic approaches always outperformed the basic classifiers
• Combination of text classifiers, metadata, NLP and sentiments usually
resulted in high precision – above about 70%
• Does not mean combination of techniques always rank best
• No clear trend like ‘more NLP leads to a better result’
• Binary classifiers more accurate than multiclass classifiers
• No single classifier works best for all review types and data sources
• Naive Bayes seems to be a more appropriate classifier for review
classification
12
Thank you for listening 13
https://mast.informatik.uni-hamburg.de/wp-content/uploads/2015/06/review_classification_preprint.pdf
Original paper:

More Related Content

PPTX
App analytics march2015
PPT
Seo company in bangalore quadraincorp
PPTX
App analytics
PPTX
33 Tactics to Engage and Retain More Customers- IRCE 2016
PPT
Search Engine Optimization
PDF
Application Insights and Jupyter Notebook(Opensource) combo to analyze large ...
PDF
LIA Introduction
PDF
Webinar: Personalized Retail Search & Recommendations with Fusion
App analytics march2015
Seo company in bangalore quadraincorp
App analytics
33 Tactics to Engage and Retain More Customers- IRCE 2016
Search Engine Optimization
Application Insights and Jupyter Notebook(Opensource) combo to analyze large ...
LIA Introduction
Webinar: Personalized Retail Search & Recommendations with Fusion

What's hot (7)

PPT
Genevieve De La Cruz .Net Portfolio
PPTX
Final Presentation
PDF
_Search? en toute simplicité : Elastic + App Search
PPTX
Birds-Other
PPTX
Mobipedia presentation
PPTX
Psm digital-olympus-slides-09.19.18
PDF
Webinar: Ecommerce, Rules, and Relevance
Genevieve De La Cruz .Net Portfolio
Final Presentation
_Search? en toute simplicité : Elastic + App Search
Birds-Other
Mobipedia presentation
Psm digital-olympus-slides-09.19.18
Webinar: Ecommerce, Rules, and Relevance
Ad

Similar to Oguzhan nlp presentation (20)

PPTX
[DSC DACH 24] Evalution and Observability of Gen AI application - Igor Nikola...
PDF
Webinar: Increase Conversion With Better Search
PDF
App testing and publishing
PDF
How To Implement Your Online Search Quality Evaluation With Kibana
PDF
Using microsoft application insights to implement a build, measure, learn loop
PDF
UCL M.Sc. Technology Entrepreneurship 2015 - Launching Digital Products
PPTX
Optimizing Dev Portals with Analytics and Feedback
PDF
FineAI Recommendation Engine
PPTX
ppt.pptx
PDF
Mobile App Analytics
PPTX
Measuring the Right App Metrics - Guide for Beginners
PPTX
Smartone v1.0
PDF
Modern Perspectives on Recommender Systems and their Applications in Mendeley
PDF
Maruti gollapudi cv
PPTX
Sentiment analysis with variaous Modeling
PDF
Modern Perspectives on Recommender Systems and their Applications in Mendeley
PDF
Option 2015- Getting Started with Optimizely for Mobile
PPTX
Book Recommendation System using Machine Learning
PDF
Cultural change of testing
PPTX
Google Tag Manager and Google Analytics
[DSC DACH 24] Evalution and Observability of Gen AI application - Igor Nikola...
Webinar: Increase Conversion With Better Search
App testing and publishing
How To Implement Your Online Search Quality Evaluation With Kibana
Using microsoft application insights to implement a build, measure, learn loop
UCL M.Sc. Technology Entrepreneurship 2015 - Launching Digital Products
Optimizing Dev Portals with Analytics and Feedback
FineAI Recommendation Engine
ppt.pptx
Mobile App Analytics
Measuring the Right App Metrics - Guide for Beginners
Smartone v1.0
Modern Perspectives on Recommender Systems and their Applications in Mendeley
Maruti gollapudi cv
Sentiment analysis with variaous Modeling
Modern Perspectives on Recommender Systems and their Applications in Mendeley
Option 2015- Getting Started with Optimizely for Mobile
Book Recommendation System using Machine Learning
Cultural change of testing
Google Tag Manager and Google Analytics
Ad

Recently uploaded (20)

PDF
Pre independence Education in Inndia.pdf
PDF
Abdominal Access Techniques with Prof. Dr. R K Mishra
PPTX
human mycosis Human fungal infections are called human mycosis..pptx
PDF
Anesthesia in Laparoscopic Surgery in India
PDF
grade 11-chemistry_fetena_net_5883.pdf teacher guide for all student
PDF
Black Hat USA 2025 - Micro ICS Summit - ICS/OT Threat Landscape
PDF
STATICS OF THE RIGID BODIES Hibbelers.pdf
PDF
BÀI TẬP BỔ TRỢ 4 KỸ NĂNG TIẾNG ANH 9 GLOBAL SUCCESS - CẢ NĂM - BÁM SÁT FORM Đ...
PPTX
master seminar digital applications in india
PDF
VCE English Exam - Section C Student Revision Booklet
PDF
TR - Agricultural Crops Production NC III.pdf
PDF
01-Introduction-to-Information-Management.pdf
PPTX
Microbial diseases, their pathogenesis and prophylaxis
PDF
Module 4: Burden of Disease Tutorial Slides S2 2025
PDF
RMMM.pdf make it easy to upload and study
PPTX
PPH.pptx obstetrics and gynecology in nursing
PDF
Supply Chain Operations Speaking Notes -ICLT Program
PDF
Saundersa Comprehensive Review for the NCLEX-RN Examination.pdf
PPTX
Introduction_to_Human_Anatomy_and_Physiology_for_B.Pharm.pptx
PDF
FourierSeries-QuestionsWithAnswers(Part-A).pdf
Pre independence Education in Inndia.pdf
Abdominal Access Techniques with Prof. Dr. R K Mishra
human mycosis Human fungal infections are called human mycosis..pptx
Anesthesia in Laparoscopic Surgery in India
grade 11-chemistry_fetena_net_5883.pdf teacher guide for all student
Black Hat USA 2025 - Micro ICS Summit - ICS/OT Threat Landscape
STATICS OF THE RIGID BODIES Hibbelers.pdf
BÀI TẬP BỔ TRỢ 4 KỸ NĂNG TIẾNG ANH 9 GLOBAL SUCCESS - CẢ NĂM - BÁM SÁT FORM Đ...
master seminar digital applications in india
VCE English Exam - Section C Student Revision Booklet
TR - Agricultural Crops Production NC III.pdf
01-Introduction-to-Information-Management.pdf
Microbial diseases, their pathogenesis and prophylaxis
Module 4: Burden of Disease Tutorial Slides S2 2025
RMMM.pdf make it easy to upload and study
PPH.pptx obstetrics and gynecology in nursing
Supply Chain Operations Speaking Notes -ICLT Program
Saundersa Comprehensive Review for the NCLEX-RN Examination.pdf
Introduction_to_Human_Anatomy_and_Physiology_for_B.Pharm.pptx
FourierSeries-QuestionsWithAnswers(Part-A).pdf

Oguzhan nlp presentation

  • 1. Bug Report, Feature Request, or Simply Praise? On Automatically Classifying App Reviews Authors: Walid Maalej and Hader Nabil IEEE RE - 2015 Presented by: Oğuzhan Çalıkkasap
  • 2. App Market • Apple App Store • Google Play Store https://www.statista.com/statistics/276623/number-of-apps-available-in-leading-app-stores/ 2
  • 3. Motivation • Better reviews, higher sales • Rich source Information about bugs Ideas for new features • Benefitted by App vendors Developers 3
  • 4. Purpose • Classify reviews into four categories Bug reports Feature requests User experiences Ratings 4
  • 5. Paper Outline • Introduce probabilistic techniques for review classification • Compare accuracy of the review classification techniques • Derive insights into review analytics tool design 5
  • 6. Review Classification Techniques 1. String matching 2. Bag of Words 3. Text Preprocessing 4. Review Metadata 5. Sentiment Analysis 6. Binary and Multiclass Classifiers 6
  • 7. Research Method and Data • Collect reviews from app store, extract their metadata • Manually label reviews and create a truth set • Implement each classifier at different parts of truth set • Evaluate classifiers’ accuracies • Data: Review text, title, app name, category, store, submission date, star rating 7
  • 9. Evaluation Metrics • Precision: Fraction of reviews that are classified correctly for type i • Recall: Fraction of reviews of type i which are classified correctly • F1: Harmonic mean 9 𝑇𝑃𝑖 𝑇𝑃𝑖 + 𝐹𝑃𝑖 𝑇𝑃𝑖 𝑇𝑃𝑖 + 𝐹𝑁𝑖
  • 10. Research Results - Features 10
  • 11. Research Results - Classifiers 11
  • 12. Study Outcomes • Probabilistic approaches always outperformed the basic classifiers • Combination of text classifiers, metadata, NLP and sentiments usually resulted in high precision – above about 70% • Does not mean combination of techniques always rank best • No clear trend like ‘more NLP leads to a better result’ • Binary classifiers more accurate than multiclass classifiers • No single classifier works best for all review types and data sources • Naive Bayes seems to be a more appropriate classifier for review classification 12
  • 13. Thank you for listening 13 https://mast.informatik.uni-hamburg.de/wp-content/uploads/2015/06/review_classification_preprint.pdf Original paper: