Promise 2011: "Are Change Metrics Good Predictors for an Evolving Software Product Line?"
Download as PPTX, PDF1 like780 views
This research examines the effectiveness of change metrics as predictors for failure-prone files within an evolving software product line, specifically focusing on the Eclipse product line. The findings indicate that there exists a stable set of change metrics that reliably predict such failures, with performance improving as the product line evolves. Additionally, while some predictors remain consistent across all products, others may only be relevant for specific releases.
![BackgroundProduct line – “A family of products designed to take advantage of their common aspects and predicted variabilities” [Weiss and Lai 1999]e.g., Nokia cellphones, HP printers, etc.Products - Commonalities – Shared by all products. e.g., PlatformVariabilities – Differentiate the productsHigh-reuse variation](https://image.slidesharecdn.com/promisepresentation-110920162503-phpapp01/85/Promise-2011-Are-Change-Metrics-Good-Predictors-for-an-Evolving-Software-Product-Line-2-320.jpg)
![Related WorkEclipse as a product line. [Chastek, McGregor, and Northrop, 2007], [Linden, 2009], [Krishnan et al., 2011]. Summary of previous work3Failure-prone file -A file with one or more non-trivial post-releasebugs recorded in the Eclipse Bugzilla database.Important/Good predictor – Predictor providing high information gain for classification of failure-prone filesDept. of Computer Science, Iowa State University, PROMISE, September 20, 2011](https://image.slidesharecdn.com/promisepresentation-110920162503-phpapp01/85/Promise-2011-Are-Change-Metrics-Good-Predictors-for-an-Evolving-Software-Product-Line-8-320.jpg)
Promise 2011: "Are Change Metrics Good Predictors for an Evolving Software Product Line?"
- 1. Are Change Metrics Good Predictors for an Evolving Software Product Line?Sandeep Krishnan, ISUChris Strasburg, ISU & Ames Laboratory Robyn R. Lutz, ISU & JPL, California Institute of TechnologyKaterina Goseva-Popstojanova, WVU1This research is supported by NSF grants 0916275 and 0916284Dept. of Computer Science, Iowa State University, PROMISE, September 20, 2011
- 2. BackgroundProduct line – “A family of products designed to take advantage of their common aspects and predicted variabilities” [Weiss and Lai 1999]e.g., Nokia cellphones, HP printers, etc.Products - Commonalities – Shared by all products. e.g., PlatformVariabilities – Differentiate the productsHigh-reuse variation
- 3. JDT, PDE, Mylyn, Webtools, etc.
- 4. Reused in more than three products and for more than six years.
- 6. CDT, Datatools, Java EE tools.
- 7. Reused in three or fewer products and for more than four years.2Dept. of Computer Science, Iowa State University, PROMISE, September 20, 2011
- 8. Related WorkEclipse as a product line. [Chastek, McGregor, and Northrop, 2007], [Linden, 2009], [Krishnan et al., 2011]. Summary of previous work3Failure-prone file -A file with one or more non-trivial post-releasebugs recorded in the Eclipse Bugzilla database.Important/Good predictor – Predictor providing high information gain for classification of failure-prone filesDept. of Computer Science, Iowa State University, PROMISE, September 20, 2011
- 9. Product Line EvolutionProduct line evolution in two dimensions4New ReleasesP1R1P1R2P1R3P1RnP2R1P2R2P2R3P2RnNew ProductsPnR1PnR2PnR3PnRnDept. of Computer Science, Iowa State University, PROMISE, September 20, 2011
- 10. MotivationCan we leverage the reducing amount of change in product lines to better predict failure-prone files?5Dept. of Computer Science, Iowa State University, PROMISE, September 20, 2011
- 11. Eclipse case study6BlockerEclipse ClassicCriticalEclipse JavaMajorEclipse C/C++NormalEclipse JavaEEMinor
- 12. Research QuestionsAs a product evolves, do any change metrics serve as good predictors of failure-prone files?Is there a subset of change metrics which are good predictors across all product line members?Does our ability to predict failure-prone files improve as product line evolves?7Dept. of Computer Science, Iowa State University, PROMISE, September 20, 2011
- 13. FindingsThe change metrics provide good classification of the failure-prone files in the Eclipse product line.As each product evolves, there is a stable set of change metrics that are prominent predictors of failure-prone files across its releases.There is a subset of change metrics that is among the prominent predictors of all the products across most of the releases.As the product line matures, prediction performance improves for each of the four Eclipse products.8Dept. of Computer Science, Iowa State University, PROMISE, September 20, 2011
- 14. Data Source9Data TimelineData TimelineSource of failure reports-Source of change reports – CVS repository of Eclipse.Dept. of Computer Science, Iowa State University, PROMISE, September 20, 2011
- 15. Approach10Get prediction results + best predictorsWeka J48 decision tree learnerDept. of Computer Science, Iowa State University, PROMISE, September 20, 2011
- 16. Replication Results11Learner performance compared to previous resultsClassification performance comparison for Eclipse Classic 2.0, 2.1, and 3.0PC- Percentage of correctly classified instancesTPR- True positive rateFPR- False positive rateDept. of Computer Science, Iowa State University, PROMISE, September 20, 2011
- 17. 12Top five predictors for earlier releases of Eclipse ClassicReplication ResultsTop predictors from this studyRevisions, Weighted_ageTop predictors from previous studyMax_changeset, Bugfixes, RevisionsDept. of Computer Science, Iowa State University, PROMISE, September 20, 2011
- 18. 13Learner performance improves as single product evolvesExtension ResultsDept. of Computer Science, Iowa State University, PROMISE, September 20, 2011
- 19. 14Top five predictors for later releases of Eclipse ClassicExtension ResultsRevisions is good predictor for later releases also.Max_changeset is a good predictor also.Dept. of Computer Science, Iowa State University, PROMISE, September 20, 2011
- 20. 15Learner performance improves as product line evolvesExtension ResultsPercentage of correctly classified instances increases across releases for each productDept. of Computer Science, Iowa State University, PROMISE, September 20, 2011
- 21. 16Learner performance improves as product line evolvesExtension ResultsPercentage of true positives shows improvement across releases for each productDept. of Computer Science, Iowa State University, PROMISE, September 20, 2011
- 22. 17Learner performance improves as product line evolvesExtension ResultsPercentage of false positives shows reduces across releases for each productDept. of Computer Science, Iowa State University, PROMISE, September 20, 2011
- 23. 18Top five predictors for four products of Eclipse Product LineExtension ResultsNo common set of predictors across each product and each release.Max_changeset, Revisions and Authors are prominent predictors for all products.Some predictors are prominent for only one product.Dept. of Computer Science, Iowa State University, PROMISE, September 20, 2011
- 24. 19Dept. of Computer Science, Iowa State University, PROMISE, September 20, 2011
- 25. Thank You!20Our data is available at http://www.cs.iastate.edu/~lss/PROMISE11Data.tar.gzDept. of Computer Science, Iowa State University, PROMISE, September 20, 2011