Filtering Bug Reports for Fix-Time Analysis
- 1. Proceedings of the 16th European Conference on Software Maintenance and Reengineering Filtering Bug Reports for Fix-Time Analysis Ahmed Lamkanfi, Serge Demeyer Antwerp Systems and Software Modelling Ansymo 1 /13
- 2. Bug Report Fix-Time Prediction Predicting Eclipse Bug Lifetimes A Comparative Exploration of FreeBSD Bug Lifetimes Predicting the Fix Time of Bugs Panjer et al. Bougie et al. Giger et al. 2 /13
- 3. Bug Report Fix-Time Prediction Predicting Eclipse Bug Lifetimes A Comparative Exploration of FreeBSD Bug Lifetimes Predicting the Fix Time of Bugs Panjer et al. Bougie et al. Giger et al. 2 /13
- 4. Bug Report Fix-Time Prediction Predicting Eclipse Bug Lifetimes A Comparative Exploration of FreeBSD Bug Lifetimes Predicting the Fix Time of Bugs Panjer et al. Bougie et al. Giger et al. 2 /13
- 5. History of all reported bugs Bug Database Uncover facts about history Make predictions about future 3 /13
- 6. History of all reported bugs Bug Database Uncover facts about history Make predictions about future Fix-time of a bug? ✓ Time between opening and resolving a bug. 3 /13
- 7. Cases under Study: Eclipse and Mozilla Project Nr. of Bugs Period Platform 76.456 Oct. 2001 - Oct. 2007 PDE 11.117 Oct. 2001 - Oct. 2007 JDT 41.691 Oct. 2001 - Oct. 2007 CDT 11.468 Oct. 2001 - Oct. 2007 GEF 1.587 Oct. 2001 - Oct. 2007 Core 143.542 Mar. 1997 - Jul. 2008 Bugzilla 19.135 Mar. 2003 - Jul. 2008 Firefox 79.272 Jul. 1999 - Jul. 2008 Thunderbird 23.408 Jan. 2000 - Jul. 2008 SeaMonkey 85.143 Nov. 1995 - Jul. 2008 4 /13
- 8. 3000 1000 300 100 30 Fix−Time (logarithmic scale) 10 3 1 0.3 0.1 0.03 0.01 0.003 0.001 0.0003 0.0001 Platform PDE JDT CDT GEF Projects 5 /13
- 9. 3000 1000 300 100 30 Fix−Time (logarithmic scale) 10 3 1 0.3 0.1 0.03 0.01 0.003 0.001 0.0003 0.0001 Core Bugzilla Firefox Thunderbird Seamonkey Projects 6 /13
- 10. Summary of the Box-Plots Project Smallest Fix-time Platform 10 seconds PDE 12 seconds JDT 10 seconds CDT 9 seconds GEF 8 seconds Core 11 seconds Bugzilla 3 seconds Firefox 13 seconds Thunderbird 18 seconds SeaMonkey 14 seconds 7 /13
- 11. Ask a developer! 8 /13
- 12. Ask a developer! ➡“the developer has already the necessary code changes ready to fix a bug, then files a bug to make sure it's getting tracked in the system” 8 /13
- 13. Filtering out unreliable reports? ✓ How does this impact the accuracy? 9 /13
- 14. Filtering out unreliable reports? ✓ How does this impact the accuracy? Small experiment ✓ Based on experiment from “Predicting the Fix Time of Bugs” from Giger et al. (2010) 9 /13
- 15. Train from the history of bug reports ✓ Fields are extracted from the reports day opened, month opened, platform, reporter, ➡ severity,... ✓ Naïve Bayes classifiers learns the characteristics from the reports ✓ 10-fold cross validation 10/13
- 16. Train from the history of bug reports ✓ Fields are extracted from the reports day opened, month opened, platform, reporter, ➡ severity,... ✓ Naïve Bayes classifiers learns the characteristics from the reports ✓ 10-fold cross validation Bugs are grouped in two sets ✓ Fast: fixtime ≤ median ✓ Slow: fixtime > median 10/13
- 17. Evaluation: ✓ Receiver Operating Characteristic(ROC) curve ✓ Area Under Curve(AUC): 0.5 is random prediction; 1.0 perfect classification 11/13
- 18. Evaluation: ✓ Receiver Operating Characteristic(ROC) curve ✓ Area Under Curve(AUC): 0.5 is random prediction; 1.0 perfect classification Two-fold experiment ✓ With and without the filtering of bug reports ✓ Threshold for filtering set to 1/2 of the first quartile 11/13
- 19. Accuracy Results Project AUC Before AUC After Platform 0.692 0.700 PDE 0.641 0.661 JDT 0.646 0.649 CDT 0.693 0.708 GEF 0.663 0.732 Core 0.663 0.686 Bugzilla 0.722 0.733 Firefox 0.623 0.653 Thunderbird 0.657 0.645 SeaMonkey 0.698 0.706 12/13
- 20. Conclusions ✓ More investigation needed when dealing with real-world data ✓ Some bugs are fixed conspicuously fast! ✓ More preprocessing/filtering may lead to improved results 13/13