Wcre12b.ppt
0 likes49 views
TRIS is a fast and accurate approach for splitting and expanding source code identifiers. It treats identifier splitting as an optimization problem and uses a two-phase strategy: 1) building a dictionary of terms and transformation rules, and 2) applying these to split identifiers. A case study found TRIS outperformed other approaches like Camel Case splitting, Samurai, and TIDIER in accuracy on two software projects. TRIS also had higher accuracy than Samurai and GenTest on a large dataset of identifier split correctness.
![Research Context
Researchers agree that the identifier semantics areResearchers agree that the identifier semantics are
important:important:
Help program comprehension: [CT99, DP05, LFB06]…;
Improve software quality: [MPF08, LMFB06, JH06]…;
3/25
WCRE’12, Kingston
Suggest clues: [TGM96, LMFB07]...
Composed identifiers:Composed identifiers:
Camel Case: MyLocalAccount, User_Address
Contraction based: pntr ctr, usrAdrss , imagEdge
Good and possibly known to the developers:
hmmm, ixoth , pqrstuvwxyz](https://image.slidesharecdn.com/wcre12b-170828010430/85/Wcre12b-ppt-3-320.jpg)
![Current Practices
Camel CaseCamel Case--based approaches [based approaches [LFB06LFB06]]
Samurai bySamurai by EnslenEnslen [ELK09]:[ELK09]:
Mines term frequencies in large source code bases and relies on two
assumptions:
4/25
WCRE’12, Kingston
1) A substring composing an identifier is also likely to be used in
other parts of the program or in other programs.
2) Given two possible splits of a given identifier, the split that
most likely represents the developer’s intent partitions the
identifier into terms occurring more often in the program.
Abbreviations not treated, no quantification of howAbbreviations not treated, no quantification of how
close the match is to the unknown string.close the match is to the unknown string.](https://image.slidesharecdn.com/wcre12b-170828010430/85/Wcre12b-ppt-4-320.jpg)
![Current Practices
TIDIER [TIDIER [GMAGMA11]:11]:
Inspired by speech recognition techniques and uses Dynamic
Time Warping to match terms to domain concepts.
Exploits context in the form of specialized dictionaries and
mimics the process of transforming words via contraction
rules.
5/25
WCRE’12, Kingston
GenTestGenTest/Normalize [LBM10, LB11]:/Normalize [LBM10, LB11]:
GenTest generates all possible splittings and evaluates a
scoring function against each proposed splitting.
Normalize uses a machine translation technique, the
maximum coherence model.
TIDIER,TIDIER, GenTestGenTest and Normalize are demandingand Normalize are demanding
in terms of computation time.in terms of computation time.](https://image.slidesharecdn.com/wcre12b-170828010430/85/Wcre12b-ppt-5-320.jpg)
![References
[LB11 ] D. Lawrie and D. Binkley, “Expanding identifiers to normalize source
code vocabulary,” International Conference on Software Maintenance , 2011, pp.
113–122.
[LBM10] D. Lawrie, D. Binkley, and C. Morrell, “Normalizing source code
vocabulary,” Working Conference on Reverse Engineering, 2010, pp. 112–122.
[GMA11] L. Guerrouj, D. P. Massimiliano, A. Giuliano, and Y.-G. Guéhéneuc,
“Tidier: An identifier splitting approach using speech recognition techniques,”
19/25
WCRE’12, Kingston
“Tidier: An identifier splitting approach using speech recognition techniques,”
Journal of Software Maintenance and Evolution: Research and Practice, 2011.
[MGD10] N. Madani, L. Guerrouj, M. Di Penta, Y. Guéhéneuc, and G. Antoniol.
“Recognizing words from source code identifiers using speech recognition
techniques”. 14th European Conference on Software Maintenance and
Reengineering, March 2010.
[ELK09] E. Enslen, E. Hill, L. Pollock, and K. Vijay-Shanker, “Mining source
code to automatically split identifiers for software analysis”. Mining Software
Repositories, International Workshop on, vol. 0, pp. 71-80, 2009.](https://image.slidesharecdn.com/wcre12b-170828010430/85/Wcre12b-ppt-19-320.jpg)
![References
[MPF08] A. Marcus, D. Poshyvanyk, and R. Ferenc. “Using the conceptual
cohesion of classes for fault prediction in object-oriented systems”. IEEE
Transactions on Software Engineering, 34(2):287-300, 2008.
LMFB07] Dawn Lawrie, Christopher Morrell, Henry Feild, and David Binkley.
“Effective identifier names for comprehension and memory”. Innovations in
Systems and Software Engineering, 3(4):303-318, 2007.
[LFB06] D. Lawrie, H. Feild, and D. Binkley. “Syntactic identifier conciseness
20/25
WCRE’12, Kingston
[LFB06] D. Lawrie, H. Feild, and D. Binkley. “Syntactic identifier conciseness
and consistency”. 6th International Workshop on Source Code Analysis and
Manipulation, pages139-148, Sept 27-29, 2006.
[LMFB06] D. Lawrie, C. Morrell, H. Feild, and D. Binkley. “What’s in a name? a
study of identifiers”. 14th International Conference on Program
Comprehension, pages 3-12, Athens, Greece, 2006.
[JH06] Z. Ming Jiang and Ahmed E. Hassan. “Examining the evolution of code
comments in postgresql”. 2006 International Workshop on Mining Software
Repositories, pages 179-180, 2006.](https://image.slidesharecdn.com/wcre12b-170828010430/85/Wcre12b-ppt-20-320.jpg)
![References
[DP05] Florian Deissenbock and Markus Pizka. “Concise and consistent
naming”. Proceedings of the International Workshop on Program
Comprehension, May 2005.
[Ian00] Ian Sommerville. Software Engineering. Addison-Wesley, sixth edition,
2000.
21/25
WCRE’12, Kingston
[TGM96] Armstrong Takang, Penny A. Grubb, and Robert D. Macredie. “The
effects of comments and identifier names on program comprehensibility: an
experiential study”. Journal of Program Languages, 4(3):143–167, 1996.
[HNey84] H. Ney, “The use of a one-stage dynamic programming algorithm for
connected word recognition”. Acoustics, Speech and Signal Processing, IEEE
Transactions on, vol. 32, no. 2, pp. 263-271, Apr 1984.](https://image.slidesharecdn.com/wcre12b-170828010430/85/Wcre12b-ppt-21-320.jpg)
Wcre12b.ppt
- 1. TRIS: a Fast and Accurate IdentifiersTRIS: a Fast and Accurate Identifiers Splitting and ExpansionSplitting and ExpansionSplitting and ExpansionSplitting and Expansion AlgorithmAlgorithm WCRE 2012, Kingston, CanadaWCRE 2012, Kingston, Canada Latifa Guerrouj, Philippe Galinier, Yann-Gaël Guéhéneuc, Giuliano Antoniol, and Massimiliano Di Penta
- 2. Content Research Context Current Practices TRIS in Essence 2/25 WCRE’12, Kingston TRIS in Essence Case Study Threats to Validity Conclusion
- 3. Research Context Researchers agree that the identifier semantics areResearchers agree that the identifier semantics are important:important: Help program comprehension: [CT99, DP05, LFB06]…; Improve software quality: [MPF08, LMFB06, JH06]…; 3/25 WCRE’12, Kingston Suggest clues: [TGM96, LMFB07]... Composed identifiers:Composed identifiers: Camel Case: MyLocalAccount, User_Address Contraction based: pntr ctr, usrAdrss , imagEdge Good and possibly known to the developers: hmmm, ixoth , pqrstuvwxyz
- 4. Current Practices Camel CaseCamel Case--based approaches [based approaches [LFB06LFB06]] Samurai bySamurai by EnslenEnslen [ELK09]:[ELK09]: Mines term frequencies in large source code bases and relies on two assumptions: 4/25 WCRE’12, Kingston 1) A substring composing an identifier is also likely to be used in other parts of the program or in other programs. 2) Given two possible splits of a given identifier, the split that most likely represents the developer’s intent partitions the identifier into terms occurring more often in the program. Abbreviations not treated, no quantification of howAbbreviations not treated, no quantification of how close the match is to the unknown string.close the match is to the unknown string.
- 5. Current Practices TIDIER [TIDIER [GMAGMA11]:11]: Inspired by speech recognition techniques and uses Dynamic Time Warping to match terms to domain concepts. Exploits context in the form of specialized dictionaries and mimics the process of transforming words via contraction rules. 5/25 WCRE’12, Kingston GenTestGenTest/Normalize [LBM10, LB11]:/Normalize [LBM10, LB11]: GenTest generates all possible splittings and evaluates a scoring function against each proposed splitting. Normalize uses a machine translation technique, the maximum coherence model. TIDIER,TIDIER, GenTestGenTest and Normalize are demandingand Normalize are demanding in terms of computation time.in terms of computation time.
- 6. TRIS in Essence TRIS assumes that developers compose identifiers:TRIS assumes that developers compose identifiers: Using terms and words reflecting domain concepts developer’s experience, knowledge. TRIS uses set of transformation rules:TRIS uses set of transformation rules: 6/25 WCRE’12, Kingston Drop vowels, drop prefix, drop suffix, etc. TRISTRIS treats the problem as an optimization problemtreats the problem as an optimization problem where the cost function is:where the cost function is: C(wOrig w) = α*Freq(wOrig) + C(type(wOrig w)) Freq(wOrig): frequency of wOrig in the source code C(type(wOrig w)): cost of the transformation type
- 7. TRIS in Essence TRIS applies a twoTRIS applies a two--phase strategy:phase strategy: 1) Building dictionary transformation1) Building dictionary transformation Computation of the frequency of dictionary words 7/25 WCRE’12, Kingston Construction of the set of transformations Construction of the arborescence 2) Identifier processing algorithm2) Identifier processing algorithm Construction of the auxiliary graph of the identifier Search for a shortest path in the auxiliary graph
- 8. TRIS in Essence TRIS phasesTRIS phases –– Example:Example: 8/25 WCRE’12, Kingston Dictionary Transformations Building Information for the Identifier callableint
- 9. TRIS in Essence 9/25 WCRE’12, Kingston Arborescence of Transformations for the Dictionary D Auxiliary Graph for the Identifier callableint
- 10. Case Study - Research Question Quality focus: Accuracy of TRIS with respect to oracles, and compared with state- of-the-art approaches: 1) Camel Case Splitter, 2) Samurai, 3) TIDIER, and 4) GenTest. 10/25 WCRE’12, Kingston TIDIER, and 4) GenTest. Research Question: What is the accuracy of the TRIS identifier splitting and expansion approach compared with alternative state-of-the art approaches?
- 11. Case Study – Analyzed Systems JHotDraw – Java 16 KLOC 155 files 2,348 identifiers (longer than 2 chars) 957 manually segmented identifiers Lynx – C 174 KLOC 11/25 WCRE’12, Kingston 174 KLOC 247 files 12,194 identifiers (longer than 2 chars) 3,085 manually segmented identifiers Lawrie et al. Data Set 186 programs 26 MLOC of C 15 MLOC of C++ 7 MLOC of Java 489 C/C++ sampled from 430 GNU projects
- 12. Case Study - Results 12/25 WCRE’12, Kingston Precision, recall and F-measure of TRIS, Camel Case, Samurai, and TIDIER on JHotDraw
- 13. Case Study - Results 13/25 WCRE’12, Kingston Precision, recall and F-measure of TRIS, Camel Case, Samurai, and TIDIER on Lynx
- 14. Case Study - Results Comparison among approaches: Results of Wilcoxon paired test and Cliff’s Delta effect size on JHotDraw Approach 1 Approach 2 Adj p-value Cliff's delta TRIS Camel Case 0.431 0.041 TRIS Samurai 0.894 0.001 TRIS TIDIER 0.024 0.043 14/25 WCRE’12, Kingston Comparison among approaches: Results of Wilcoxon paired test and Cliff’s Delta effect size on Lynx Approach 1 Approach 2 Adj p-value Cliff's delta TRIS Camel Case <0.001 0.743 TRIS Samurai <0.001 0.684 TRIS TIDIER <0.001 0.204 Cliff’s delta Interpretation: - small for 0.148 <= d <0.33 - medium for 0.33 <= d < 0.474 - large for d >= 0.474
- 15. Case Study - Results Metric Approach 1Q Median Mean 3Q σ Precision TIDIER 0.4000 0.6667 0.6368 1.0000 0.3681 TRIS 1.0000 1.0000 0.8933 1.0000 0.2471 Recall TIDIER 0.5000 0.6667 0.6496 1.0000 0.3654 TRIS 1.0000 1.0000 0.8720 1.0000 0.2606 15/25 WCRE’12, Kingston Precision, recall and F-measure of TRIS and TIDIER on the 489 C Sampled Identifiers TRIS 1.0000 1.0000 0.8720 1.0000 0.2606 F-measure TIDIER 0.4000 0.6667 0.6409 1.0000 0.3650 TRIS 1.0000 1.0000 0.8790 1.0000 0.2524
- 16. Case Study - Results Metric Approach 1Q Median Mean 3Q σ Precision TRIS 1.0000 1.0000 0.9763 1.0000 0.1184 Recall TRIS 1.0000 1.0000 0.9439 1.0000 0.1565 F-measure TRIS 1.0000 1.0000 0.9559 1.0000 0.1358 Precision, recall and F-measure of TRIS and TIDIER on the 16/25 WCRE’12, Kingston Approach Identifier Splitting Correctness Samurai 70% GenTest 82% TRIS 86% Correctness of the splitting provided using the data set from LAWRIE et al data set from LAWRIE et al
- 17. Conclusion TRIS maps the identifier splitting/expansion problem in a graph optimization problem to find the optimal path in an identifier acyclic weighted graph. TRIS was applied on Java, C, and C++ samples and compared to Camel Case, Samurai, TIDIER, and GenTest. 17/25 WCRE’12, Kingston Camel Case, Samurai, TIDIER, and GenTest. Results show that TRIS outperforms other approaches with medium to large effect size. TRIS is also efficient in terms of computation time: quadratic complexity in the length of the identifier.
- 18. Finally… Questions 18/25 WCRE’12, Kingston Thank you for your attention
- 19. References [LB11 ] D. Lawrie and D. Binkley, “Expanding identifiers to normalize source code vocabulary,” International Conference on Software Maintenance , 2011, pp. 113–122. [LBM10] D. Lawrie, D. Binkley, and C. Morrell, “Normalizing source code vocabulary,” Working Conference on Reverse Engineering, 2010, pp. 112–122. [GMA11] L. Guerrouj, D. P. Massimiliano, A. Giuliano, and Y.-G. Guéhéneuc, “Tidier: An identifier splitting approach using speech recognition techniques,” 19/25 WCRE’12, Kingston “Tidier: An identifier splitting approach using speech recognition techniques,” Journal of Software Maintenance and Evolution: Research and Practice, 2011. [MGD10] N. Madani, L. Guerrouj, M. Di Penta, Y. Guéhéneuc, and G. Antoniol. “Recognizing words from source code identifiers using speech recognition techniques”. 14th European Conference on Software Maintenance and Reengineering, March 2010. [ELK09] E. Enslen, E. Hill, L. Pollock, and K. Vijay-Shanker, “Mining source code to automatically split identifiers for software analysis”. Mining Software Repositories, International Workshop on, vol. 0, pp. 71-80, 2009.
- 20. References [MPF08] A. Marcus, D. Poshyvanyk, and R. Ferenc. “Using the conceptual cohesion of classes for fault prediction in object-oriented systems”. IEEE Transactions on Software Engineering, 34(2):287-300, 2008. LMFB07] Dawn Lawrie, Christopher Morrell, Henry Feild, and David Binkley. “Effective identifier names for comprehension and memory”. Innovations in Systems and Software Engineering, 3(4):303-318, 2007. [LFB06] D. Lawrie, H. Feild, and D. Binkley. “Syntactic identifier conciseness 20/25 WCRE’12, Kingston [LFB06] D. Lawrie, H. Feild, and D. Binkley. “Syntactic identifier conciseness and consistency”. 6th International Workshop on Source Code Analysis and Manipulation, pages139-148, Sept 27-29, 2006. [LMFB06] D. Lawrie, C. Morrell, H. Feild, and D. Binkley. “What’s in a name? a study of identifiers”. 14th International Conference on Program Comprehension, pages 3-12, Athens, Greece, 2006. [JH06] Z. Ming Jiang and Ahmed E. Hassan. “Examining the evolution of code comments in postgresql”. 2006 International Workshop on Mining Software Repositories, pages 179-180, 2006.
- 21. References [DP05] Florian Deissenbock and Markus Pizka. “Concise and consistent naming”. Proceedings of the International Workshop on Program Comprehension, May 2005. [Ian00] Ian Sommerville. Software Engineering. Addison-Wesley, sixth edition, 2000. 21/25 WCRE’12, Kingston [TGM96] Armstrong Takang, Penny A. Grubb, and Robert D. Macredie. “The effects of comments and identifier names on program comprehensibility: an experiential study”. Journal of Program Languages, 4(3):143–167, 1996. [HNey84] H. Ney, “The use of a one-stage dynamic programming algorithm for connected word recognition”. Acoustics, Speech and Signal Processing, IEEE Transactions on, vol. 32, no. 2, pp. 263-271, Apr 1984.