Icpc12b.ppt
0 likes544 views
The document summarizes a study on the impact of identifier style (camelCase vs underscore) and gender on program comprehension. The study involved 24 subjects (15 male and 9 female) who were shown source code with different identifier styles and tested on their accuracy and time in recalling identifiers as well as visual effort measured through eye tracking. The results showed no significant differences between males and females in terms of accuracy, time taken, or visual effort expended. However, females were found to expend more visual effort analyzing distractors before determining the right answer. The study concludes gender does not impact performance but finds males and females employ different strategies in comprehending source code.
Icpc12b.ppt
- 1. Women and Men – Different but Equal: On the Impact of Identifier Style on Source Code Reading Zohreh Sharafi, Zéphyrin Soh, Yann-Gaël Guéhéneuc, and Giuliano Antoniol Ptidej Team and Soccer Lab, École Polytechnique de Montréal, Montréal, Canada. 1
- 2. Introduction Documentation Program evolution is often solely based on source code. Comments Not available Outdated Program comprehension heavily depends on identifiers. Not available Not useful 2
- 3. Introduction What is the impact of identifiers on program comprehension? Identifier Style (Sharif 20101, Binkley 20092) Camel Case Underscore Identifier quality (Lawrie 20073) Length Structure 3 1. D. Binkley, M. Davis, D. Lawrie, and C. Morrell, “To camelcase or under score,” ICPC, 2009 2. B. Sharif and J. Maletic, “An eye tracking study on camelcase and under score identifier styles, ICPC, 2010 3. D. Lawrie, C. Morrell, H. Feild, and D. Binkley, “Effective identifier names for comprehension and memory,” Innovations in Systems and Software Engineering, 2007.
- 4. Identifier Style Investigate the Impact of Identifier Style “Camel Case vs. Underscore” Time Binkley, 20091 Time Higher accuracy for Camel Case No difference More time spent on Camel Case More time spent on Camel Case Not applicable Accuracy Sharif, 20102 Lower visual effort on Underscore Effort 4 1. D. Binkley, M. Davis, D. Lawrie, and C. Morrell, “To camelcase or under scoreI, CPC, 2009 2. B. Sharif and J. Maletic, “An eye tracking study on camelcase and under score identifier styles,” ICPC , 2010
- 5. Gender Preference Does gender affects identifier style preference? Camel Case? Underscore? Camel Case? Underscore? Does gender and identifier style affects program understanding? 5
- 6. Related Work Self-efficacy (1994)1 Lower self-efficacy Higher self-efficacy Selectivity Hypothesis (2001)2 Use comprehensive way Use simple heuristic Software Adaptation & Use (2003)3 No experience = No self-confidence Apply general knowledge for specific tasks Testing Strategies (2009)4 Use testing only for finding bugs 6 1. 2. 3. 4. Use testing for: 1. finding bugs 2. fixing bugs 3. evaluating their fix G. Torkzadeh and X. Koufteros, “Factorial validity of a computer self-efficacy scale and the impact of computer training1994. E. O’Donnell and E. Johnson, “The effects of auditor gender and task complexity on information processing efficiency,” 2001. K. Hartzel, “How self-efficacy and gender issues affect software adoption and use,” Communications of the ACM, 2003. V. Grigoreanu, J. Brundage, E. Bahna, M. Burnett, P. ElRif, and J. Snover, “Males and females script debugging strategies,” 2009.
- 7. Our Empirical Study Goal: Design and perform an experiment to investigate the impact of gender on identifier styles and identifier comprehension strategies Research question: Does the developers’ gender impact their effort, their required time, and as well as their ability to recall identifiers (accuracy) in source code reading? Memorability impacts the comprehension of identifiers1,2,3 7 Lacking memorability may impair: Identifiers recall Code readability Program comprehension 1. D. M. Jones, 2003, the New C Standard (Identifiers) An Economic and Cultural Commentary. 2. B. Sharif and J. Maletic, “An eye tracking study on camelcase and under score identifier styles, ICPC, 2010 3. D. Lawrie, C. Morrell, H. Feild, and D. Binkley, “Effective identifier names for comprehension and memory,” 2007
- 8. Experiment Design Independent variables 1. 2. Gender: male (M) or female (F) Identifier style: Camel Case (CC) or Underscore (US) Dependent variables 1. 2. 3. Accuracy Required time (Speed) Visual effort Mitigating variables 1. 2. 8 Study level Style preference of subjects
- 9. Experiment Design Subjects’ Demography (24 Subjects) Academic background Gender Ph.D. M.Sc. B.Sc. Male Female 11 10 3 15 9 Source code stimulus 9 Question stimulus
- 10. Experiment Design Eye-tracking system FaceLAB1: Video-based Two cameras An infrared sensor Non-intrusive No goggles, No wires No sensing device 10 1. http://www.seeingmachines.com/
- 11. Results and Analyses: Accuracy and Time Accuracy Time (min) Accuracy Men Women Time Men Women Camel Case 83% 80% Camel Case 5.25 6.6 Underscore 68% 85% Underscore 6.7 7.8 Table 1 - Accuracy Table 2 - Time Accuracy Time (min) Men 74% 5.94 Women 82% 7.18 Table 3 - Overall Time and Accuracy Accuracy No significant differences No significant differences Time 11
- 12. Results and Analyses: Visual Effort Visual effort: Calculated from eye-tracking data Calculated based on the amount of visual attention: Less attention Less effort Visual attention can trigger the mental processes Two types of eye gaze data: Fixation Saccade We use fixation to calculate effort as previous works 12
- 13. Results and Analyses: Visual Effort Metric - Source code Stimulus Convex hull: the smallest convex sets of fixations that contains all of a subject’s fixations1 Source Code Stimulus Smaller convex hull Close fixations Less effort 13 1. J. H. Goldberg and X. P. Kotval, “Computer interface evaluation using eye movements: methods and constructs,”, 1999.
- 14. Results and Analyses: Visual Effort Metrics - Question Stimulus Metrics to measure visual effort (Sharif 20101) Question Stimulus 1. B. Sharif and J. Maletic, “An eye tracking study on camelcase and under score identifier styles,” IEEE 18th International Conference on Program Comprehension , 2010
- 15. Result and Analysis: Visual Effort FC(Q) No significant differences FR(Correct) No significant differences Effort • Female FR(distracter) Female subjects spends more visual effort subjects put more visual effort on distracters • Female subjects analyze all distracters before making decision 15
- 16. Results and Analyses: Visual Effort 16 Heatmap
- 17. Conclusion Investigate the Impact of Identifier Style “Camel Case vs. Underscore” Binkley, 20091 Time Our Experiment (135 subjects) Accuracy Sharif, 20102 (15 subjects) (24 subjects) Higher accuracy for Camel Case More time spent on Camel Case Not applicable No difference No difference More time spent on Camel Case Lower visual effort on Underscore No difference No difference Effort 17 1. D. Binkley, M. Davis, D. Lawrie, and C. Morrell, “To camelcase or under score,” ICPC, 2009 2. B. Sharif and J. Maletic, “An eye tracking study on camelcase and under score identifier styles,” ICPC, 2010
- 18. Conclusion Gender does NOT impact: Accuracy • Spend Time more effort on alternative answers • Seem to: • Carefully weight all options • Rule out wrong answers • Statistically strong interaction between accuracy and effort on distracters Effort 18 Seem to quickly go for one choice • No correlation between effort and accuracy
- 19. Threats to Validity Internal validity: Random ordering of source code Comfortable environment External validity (generalization of the results): Students as subjects 19 Students are the next generation of software professionals1 24 subjects 9 female subjects Not large enough to be generalized A call for further studies 1. B. A. Kitchenham, S. L. Pfleeger, L. M. Pickard, P. W. Jones, D. C. Hoaglin, K. E. Emam, and J. Rosenberg, “Preliminary guidelines for empirical research in software engineering,” IEEE Trans. Softw. Eng., 2002
- 20. Last but not Least Gender does NOT impact: Accuracy Male and female subjects use different strategiess to select the correct answer Time Understanding systematic biases between men and women: 20 Design tools and methods better adapted to different developers Support different program understanding strategies