ARdoc: App Reviews Development Oriented Classifier
Download as PPTX, PDF2 likes595 views
ARDoc is a classifier designed to help developers identify useful feedback from app user reviews, achieving precision, recall, and F-measure rates between 84% and 89%. The tool employs text analysis and machine learning to filter and analyze reviews for software maintenance and evolution. It integrates resources like Stanford CoreNLP and Apache Lucene to enhance its functionality.
![Users Submit Many
Reviews Regularly
iOS apps receive on average 23
reviews per day
Facebook for iOS receive more
than 4000 reviews per day
[ Pagano et al. - RE 2013 ]
4](https://image.slidesharecdn.com/fse-demo-161116160028/85/ARdoc-App-Reviews-Development-Oriented-Classi-er-4-320.jpg)
ARdoc: App Reviews Development Oriented Classifier
- 1. ARdoc: App Reviews Development Oriented Classifier Sebastiano Andrea Emitza Corrado Gerardo Harald Panichella Di Sorbo Guzman Visaggio Canfora Gall
- 3. Reviews Include Useful Information for Developers Pagano et al. – RE2013 Chen et al. – ICSE2014 Galvis Carreno et al. – ICSE2013 3
- 4. Users Submit Many Reviews Regularly iOS apps receive on average 23 reviews per day Facebook for iOS receive more than 4000 reviews per day [ Pagano et al. - RE 2013 ] 4
- 5. Past Work Chen et al – ICSE 2014 ARMiner: an approach to help app developers discover the most informative user reviews i. text analysis and machine learning to filter out non- informative reviews ii. topic analysis to recognize topics treated in the reviews classified as informative 5
- 7. Identifying Useful Reviews i. The awful button in the page doesn’t work ii. A button in the page should be added 7
- 8. Identifying Useful Reviews i. The awful button in the page doesn’t work ii. A button in the page should be added 8 BUG DESCRIPTION
- 9. Available Sources for identifying Useful Reviews i. The awful button in the page doesn’t work ii. A button in the page should be added 9 sentiment lexicon structure Natural Language Parsing Sentiment Analysis Text Analysis
- 10. 10 ARdoc: App Reviews Development Oriented Classifier
- 11. ARdoc’s Architecture 11 Stanford CoreNLP Apache Lucene API
- 12. ARdoc’s Architecture 12 Stanford CoreNLP Apache Lucene API WEKA
- 13. Taxonomy & Examples 14 Panichella et al. “How can I improve my app? Classifying user reviews for software maintenance and evolution” – ICSME 2015
- 14. ARdoc’s DEMO 15 Stanford CoreNLP Apache Lucene API WEKA
- 17. ARdoc Classification Accuracy? 18 3 Apps
- 18. ARdoc Classification Accuracy? 19 3 Apps Minesweeper PowernAPP Picturex
- 19. ARdoc Classification Accuracy? 20 3 Apps Minesweeper PowernAPP Picturex https://www.scribd.com/document/323048838/ARdoc-Appendix
- 20. ARdoc Classification Accuracy 21 Minesweeper PowernAPP Picturex https://www.scribd.com/document/323048838/ARdoc-Appendix 3 Apps 2) ARdoc classifies useful feedback with a precision ranging between 84% and 89%, a recall ranging between 84% and 89%, and an F-Measure ranging between 84% and 89% 2) ARdoc classifies useful feedback with a precision ranging between 84% and 89%, a recall ranging between 84% and 89%, and an F-Measure ranging between 84% and 89%
- 21. ARdoc Classification Accuracy 24 Minesweeper PowernAPP Picturex https://www.scribd.com/document/323048838/ARdoc-Appendix 3 Apps 2) ARdoc classifies useful feedback with a precision ranging between 84% and 89%, a recall ranging between 84% and 89%, and an F-Measure ranging between 84% and 89% 2) ARdoc classifies useful feedback with a precision ranging between 84% and 89%, a recall ranging between 84% and 89%, and an F-Measure ranging between 84% and 89% 2) ARdoc classifies useful feedback with a precision ranging between 84% and 89%, a recall ranging between 84% and 89%, and an F-Measure ranging between 84% and 89%
- 22. Conclusion & Future Work 25 1) ARdoc a novel tool able to mine relevant feedback for real world developers interested in accomplishing software maintenance and evolution tasks. 2) ARdoc classifies useful feedback with a precision ranging between 84% and 89%, a recall ranging between 84% and 89%, and an F-Measure ranging between 84% and 89%
- 23. Conclusion & Future Work 26 1) ARdoc a novel tool able to mine relevant feedback for real world developers interested in accomplishing software maintenance and evolution tasks. 2) ARdoc classifies useful feedback with a precision ranging between 84% and 89%, a recall ranging between 84% and 89%, and an F-Measure ranging between 84% and 89% & Di Sorbo et al. “What Would Users Change in My App? Summarizing App Reviews for Recommending Software Changes” – FSE 16/11//2016 (Session 11)
- 24. Thanks for the Attention! 27 Stanford CoreNLP Apache Lucene API WEKA Questions?
Editor's Notes
- #2: Hi, I’m Andrea Di Sorbo, I’m a ph.D. student at University of Sannio. In this paper we investigated possible ways for classifying user reviews in according to software maintenance tasks with the purpose of helping developers improving their apps.
- #3: Well, the context of our study is App Stores, such as Apple App Store and Google Play, where we know that users can download apps, give ratings and write reviews about the mobile apps they're using.
- #4: Indeed previous studies demonstrated that about one third of the information contained in user reviews is helpful for developers, giving feedback containing requests of implementation of new features, bug descriptions or requests of improvement about existing functionalities.
- #5: For example, a study by Pagano (RE2013) showed that mobile apps receive approximately 23 reviews per day and popular apps, as Facebook, receive on average more than 4000 reviews per day.
- #6: To handle this problem Chen at. al proposed AR-Miner, an approach to help app developers discover the most informative user reviews, which uses i) text analysis and machine learning to filter out non-informative reviews and ii) topic analysis to recognize topics treated in "informative" reviews.
- #8: We argue that text lexicon represents just one of the possible dimensions that can be exploited to detect informative reviews.
- #9: In the first review the user exposes a problem, while in the second one the user suggests the implementation of a new feature
- #10: Thus, understanding the intention in user reviews could add precious information for accomplishing software maintenance and evolution tasks. We believe that exists three different dimensions that can be explored to determine the intention of a given user review: the sentiment, the structure, and text lexicon.
- #14: Thus our final taxonomy was composed by only four categories of sentences: feature request, problem discovery, information seeking and information giving.
- #15: Relying on the techniques previously discussed, we can associate a label to each sentence in the review. These are some example of useful feedback by users. These example sentences contain relevant information for improving an app: the first two sentences could suggest developers new functionalities to implement, while the third and fourth sentences indicate bugs that need to be fixed.