SlideShare a Scribd company logo

Investigating Code Review Practices in Defective Files

The University of Adelaide, profile picture
The University of Adelaide

The document investigates code review practices, particularly focusing on files with defects and the factors influencing the effectiveness of these reviews. It identifies that reviews of future-defective files tend to be less rigorous with lower participation and intensity compared to clean files, which can lead to higher risks of post-release defects. The findings emphasize the need for more thorough reviews, especially for risky files to mitigate future defects.

Presentations & Public Speaking
Related topics:
Software Engineering • Software Quality Insights
1 of 72
Download to read offline
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
Investigating Code Review Practices in Defective Files Patanamon (Pick) Thongtanunam Shane McIntosh Ahmed E. Hassan Hajimu Iida May 16-17, 2015. Firenze, Italy patanamon-t@is.naist.jp @pamon
Modern Code Review: A lightweight, tool-supported code review process Code Review Tool
Modern Code Review: A lightweight, tool-supported code review process Code Review Tool Code change
Modern Code Review: A lightweight, tool-supported code review process Code Review Tool Code change
Modern Code Review: A lightweight, tool-supported code review process Code Review Tool Code change Examine Code
Modern Code Review: A lightweight, tool-supported code review process Upstream VCS repositories Code Review Tool Code change Examine Code
Modern Code Review: A lightweight, tool-supported code review process Upstream VCS repositories Code change A lack of code review activity can increase the risk of post-release defects 
 [McIntosh et. al., MSR2014] My code is awesome! No needs for a review Code Review Tool Code change Examine Code
Modern Code Review: A lightweight, tool-supported code review process Upstream VCS repositories Code change A lack of code review activity can increase the risk of post-release defects 
 [McIntosh et. al., MSR2014] My code is awesome! No needs for a review Code Review Tool Code change How should reviewers do a code review to reduce the risk of having defects? Examine Code
What is the difference between code review practices of defective and clean files?
What is the difference between code review practices of defective and clean files? Review Practice A Defective i.e., files that have defects
Defect-free (Clean) i.e., files that do not have defects Review Practice B What is the difference between code review practices of defective and clean files? Review Practice A VS Defective i.e., files that have defects
Defect-free (Clean) i.e., files that do not have defects Review Practice B What is the difference between code review practices of defective and clean files? Review Practice A VS Defective i.e., files that have defects
We measure 3 dimensions of review activity metrics Review Intensity e.g., #Review Iterations, Discussion Length Review Participation e.g., #Reviewers, 
 Review Agreement Reviewing Time e.g., Review Length, Code Reading Speed
Defect-free (Clean) i.e., files that do not have defects Review Practice B What is the difference between code review practices of defective and clean files? Review Practice A VS Defective i.e., files that have defects
Defect-free (Clean) i.e., files that do not have defects Review Practice B What is the difference between code review practices of defective and clean files? Review Practice A VS Defective i.e., files that have defects
Defective i.e., files that have defects We investigate defective files along 2 perspectives
Defective i.e., files that have defects We investigate defective files along 2 perspectives
Risky Files Files that have historically been defective Past Defective i.e., files that have defects We investigate defective files along 2 perspectives
Risky Files Files that have historically been defective Past FutureDefective i.e., files that have defects Future-Defective Files Files that will eventually have defects We investigate defective files along 2 perspectives
Risky Files Files that have historically been defective Past FutureDefective i.e., files that have defectsFuture-Defective Files Files that will eventually have defects Conjecture: Reviews of Future- Defective will be • less intense, • with less team participation, • completed with a shorter time than reviews of clean files We investigate defective files along 2 perspectives
Future-Defective Files: Files that have post-release defects VCS Repositories
Future-Defective Files: Files that have post-release defects VCS Repositories Release date Bug-fixing commit
Future-Defective Files: Files that have post-release defects VCS Repositories Release date Bug-fixing commit Future-Defective
Future-Defective Files: Files that have post-release defects VCS Repositories Release date No bug-fixing commits Release date Bug-fixing commit Future-Defective
Future-Defective Files: Files that have post-release defects VCS Repositories Release date No bug-fixing commits Release date Bug-fixing commit Future-Defective Clean
Future-Defective Files: Files that have post-release defects Studied ReviewsVCS Repositories Release date No bug-fixing commits Release date Bug-fixing commit 6 months Future-Defective Clean
Future-Defective Files: Files that have post-release defects Studied ReviewsVCS Repositories Release date No bug-fixing commits Release date Bug-fixing commit 6 months 1,176 Files 3,470 Reviews 10,513 Files 2,727 Reviews 5.0.0 866 Files 2,849 Reviews 11,931 Files 2,690 Reviews 5.1.0 Future-Defective Clean
#Reviewers #Reviews of Clean files #Reviewers #Reviews of Future-Defective files VS Review Activity Analysis: Compare code review activity that has been applied to future-defective and clean files
#Reviewers #Reviews of Clean files #Reviewers #Reviews of Future-Defective files VS Using a statistical test to determine the difference between the distributions of code review activity Review Activity Analysis: Compare code review activity that has been applied to future-defective and clean files
#Reviewers #Reviews of Clean files #Reviewers #Reviews of Future-Defective files VS Using a statistical test to determine the difference between the distributions of code review activity Raw code review activity metric is normalized by patch size Review Activity Analysis: Compare code review activity that has been applied to future-defective and clean files
Findings Code review activity in the reviews of future-defective files Conjecture Results Review Intensity Review Participation Reviewing Time Less Intense Less Team Participation Completed with a shorter time
Findings Code review activity in the reviews of future-defective files Conjecture Less Intense Results Review Intensity Review Participation Reviewing Time Less Intense Less Team Participation Completed with a shorter time
Findings Code review activity in the reviews of future-defective files Conjecture Less Intense Less Team Participation Results Review Intensity Review Participation Reviewing Time Less Intense Less Team Participation Completed with a shorter time
Findings Code review activity in the reviews of future-defective files Conjecture Less Intense Less Team Participation Faster Code Reading Rate Results Review Intensity Review Participation Reviewing Time Less Intense Less Team Participation Completed with a shorter time
Defective i.e., files that have defects Past We investigate defective files along 2 perspectives Risky Files Files that have historically been defective FutureFuture-Defective Files Files that will eventually have defects Conjecture: Reviews of Future- Defective will be • less intense, • with less team participation, • completed with a shorter time than reviews of clean files
Defective i.e., files that have defects Past We investigate defective files along 2 perspectives Risky Files Files that have historically been defective FutureFuture-Defective Files Files that will eventually have defects Reviews of future-defective files tend to be less rigorous than reviews of clean files
Defective i.e., files that have defects Past We investigate defective files along 2 perspectives Risky Files Files that have historically been defective FutureFuture-Defective Files Files that will eventually have defects Reviews of future-defective files tend to be less rigorous than reviews of clean files
Defective i.e., files that have defects Past We investigate defective files along 2 perspectives Risky Files Files that have historically been defective Conjecture: Reviews of risky files should be • more intense, • with more team participation, • reviewed for a longer time to reduce the risk of having defects in the future FutureFuture-Defective Files Files that will eventually have defects Reviews of future-defective files tend to be less rigorous than reviews of clean files
Risky Files: Files that had post-release defects
 in prior release VCS Repositories
Risky Files: Files that had post-release defects
 in prior release VCS Repositories Release date Prior release date Bug-fixing commit
Risky Files: Files that had post-release defects
 in prior release VCS Repositories Release date Prior release date Bug-fixing commit Risky
Risky Files: Files that had post-release defects
 in prior release VCS Repositories No bug-fixing commits Release date Prior release date Release date Prior release date Bug-fixing commit Risky
Risky Files: Files that had post-release defects
 in prior release VCS Repositories No bug-fixing commits Release date Prior release date Release date Prior release date Bug-fixing commit Normal Risky
Risky Files: Files that had post-release defects
 in prior release Studied ReviewsVCS Repositories No bug-fixing commits Release date Prior release date Release date Prior release date Bug-fixing commit 6 months Normal Risky
Risky Files: Files that had post-release defects
 in prior release Studied ReviewsVCS Repositories No bug-fixing commits Release date Prior release date Release date Prior release date Bug-fixing commit 6 months 1,168 Files 2,671 Reviews 11,629 Files 2,868 Reviews 5.1.0 Normal Risky
Findings Code review activity in the reviews of risky files Conjecture Results Review Intensity Review Participation Reviewing Time More Intense More Team Participation Completed with a longer time
Findings Code review activity in the reviews of risky files Conjecture Less Intense Results Review Intensity Review Participation Reviewing Time More Intense More Team Participation Completed with a longer time
Findings Code review activity in the reviews of risky files Conjecture Less Intense Less Team Participation Results Review Intensity Review Participation Reviewing Time More Intense More Team Participation Completed with a longer time
Findings Code review activity in the reviews of risky files Conjecture Less Intense Less Team Participation Receive Slow Feedback & Faster Code Reading Rate Results Review Intensity Review Participation Reviewing Time More Intense More Team Participation Completed with a longer time
Defective i.e., files that have defects Past We investigate defective files along 2 perspectives FutureFuture-Defective Files Files that will eventually have defects Reviews of future-defective files tend to be less rigorous than reviews of clean files Risky Files Files that have historically been defective Conjecture: Reviews of risky files should be • more intense, • with more team participation, • reviewed for a longer time to reduce the risk of having defects in the future
Defective i.e., files that have defects Past We investigate defective files along 2 perspectives FutureFuture-Defective Files Files that will eventually have defects Reviews of future-defective files tend to be less rigorous than reviews of clean files Risky Files Files that have historically been defective Developers are not as careful when they review risky files.
Defective i.e., files that have defects Past We investigate defective files along 2 perspectives Future Reviews of future-defective files tend to be less rigorous than reviews of clean files Developers are not as careful when they review risky files. Future-Defective Files Files that will eventually have defects Risky Files Files that have historically been defective
Defective i.e., files that have defects Past We investigate defective files along 2 perspectives Future Reviews of future-defective files tend to be less rigorous than reviews of clean files Developers are not as careful when they review risky files. Future-Defective Files Files that will eventually have defects Risky Files Files that have historically been defective Will careless reviews of risky files lead to future defects?
Defective i.e., files that have defects Past We investigate defective files along 2 perspectives Future Reviews of future-defective files tend to be less rigorous than reviews of clean files Developers are not as careful when they review risky files. Future-Defective Files Files that will eventually have defects Risky Files Files that have historically been defective Will careless reviews of risky files lead to future defects? Investigating code review practice in risky & future- defective files
Risky & Future-Defective Files: Risky files that will eventually have defects VCS Repositories
Risky & Future-Defective Files: Risky files that will eventually have defects VCS Repositories Bug-fixing commit Release date Prior release date Bug-fixing commit
Risky & Future-Defective Files: Risky files that will eventually have defects VCS Repositories Bug-fixing commit Release date Prior release date Bug-fixing commit Risky & Future-Defective
Risky & Future-Defective Files: Risky files that will eventually have defects VCS Repositories Bug-fixing commit Release date Prior release date Bug-fixing commit No bug-fixing commits Release datePrior release date Bug-fixing commit Risky & Future-Defective
Risky & Future-Defective Files: Risky files that will eventually have defects VCS Repositories Bug-fixing commit Release date Prior release date Bug-fixing commit No bug-fixing commits Release datePrior release date Bug-fixing commit Risky & Clean Risky & Future-Defective
Risky & Future-Defective Files: Risky files that will eventually have defects Studied Reviews VCS Repositories Bug-fixing commit Release date Prior release date Bug-fixing commit No bug-fixing commits Release datePrior release date Bug-fixing commit 6 months Risky & Clean Risky & Future-Defective
Risky & Future-Defective Files: Risky files that will eventually have defects Studied Reviews VCS Repositories Bug-fixing commit Release date Prior release date Bug-fixing commit No bug-fixing commits Release datePrior release date Bug-fixing commit 6 months 206 Files 1,299 Reviews 962 Files 1,372 Reviews 5.1.0 Risky & Clean Risky & Future-Defective
Findings Code review activity in the reviews of risky & future-defective files Conjecture Less Intense Less Team Participation Receive Slow Feedback & Faster Code Reading Rate Results Review Intensity Review Participation Reviewing Time Less Intense Less Team Participation Completed with a shorter time
Defective i.e., files that have defects Past We investigate defective files along 2 perspectives Future Reviews of future-defective files tend to be less rigorous than reviews of clean files Developers are not as careful when they review risky files. Future-Defective Files Files that will eventually have defects Risky Files Files that have historically been defective Will careless reviews of risky files lead to future defects? Investigating code review practice in risky & future- defective files
Defective i.e., files that have defects Past We investigate defective files along 2 perspectives Future Reviews of future-defective files tend to be less rigorous than reviews of clean files Developers are not as careful when they review risky files. Future-Defective Files Files that will eventually have defects Risky Files Files that have historically been defective Reviews of files that are both risky & future defective are less rigorous than files that are risky but clean
Evolvability e.g., Fixing code comments,
 Decomposing complex function Functionality e.g., Fixing incorrect 
 program logic Traceability e.g., Updating commit message We compare concerns that are addressed during reviews of defective and clean files
Evolvability Functionality Traceability Proportion of reviews in future-defective files in Qt5.0.0 82% 40%40% Reviews of defective files often address evolvability concernsResults 10% higher than clean files 5% higher than clean files 10% lower than clean files We observe the similar results for the reviews of risky files and risky & future-defective files
Investigating Code Review Practices in Defective Files
Modern Code Review: A lightweight, tool-supported code review process Upstream VCS repositories Code change A lack of code review activity can increase the risk of post-release defects 
 [McIntosh et. al., MSR2014] My code is awesome! No needs for a review Code Review Tool Code change How should reviewers do a code review to reduce the risk of having defects? Examine Code
Modern Code Review: A lightweight, tool-supported code review process Upstream VCS repositories Code change A lack of code review activity can increase the risk of post-release defects 
 [McIntosh et. al., MSR2014] My code is awesome! No needs for a review Code Review Tool Code change How should reviewers do a code review to reduce the risk of having defects? Examine Code Defect-free (Clean) i.e., files that do not have defects Review Practice B What is the difference between code review practices of defective and clean files? Review Practice A VS Defective i.e., files that have defects
Modern Code Review: A lightweight, tool-supported code review process Upstream VCS repositories Code change A lack of code review activity can increase the risk of post-release defects 
 [McIntosh et. al., MSR2014] My code is awesome! No needs for a review Code Review Tool Code change How should reviewers do a code review to reduce the risk of having defects? Examine Code Defect-free (Clean) i.e., files that do not have defects Review Practice B What is the difference between code review practices of defective and clean files? Review Practice A VS Defective i.e., files that have defects We measure 3 dimensions of review activity metrics Review Intensity e.g., #Review Iterations, Discussion Length Review Participation e.g., #Reviewers, 
 Review Agreement Reviewing Time e.g., Review Length, Code Reading Speed
Modern Code Review: A lightweight, tool-supported code review process Upstream VCS repositories Code change A lack of code review activity can increase the risk of post-release defects 
 [McIntosh et. al., MSR2014] My code is awesome! No needs for a review Code Review Tool Code change How should reviewers do a code review to reduce the risk of having defects? Examine Code Defect-free (Clean) i.e., files that do not have defects Review Practice B What is the difference between code review practices of defective and clean files? Review Practice A VS Defective i.e., files that have defects We measure 3 dimensions of review activity metrics Review Intensity e.g., #Review Iterations, Discussion Length Review Participation e.g., #Reviewers, 
 Review Agreement Reviewing Time e.g., Review Length, Code Reading Speed Defective i.e., files that have defects Past We investigate defective files along 2 perspectives Future Reviews of future-defective files tend to be less rigorous than reviews of clean files Developers are not as careful when they review risky files. Future-Defective Files Files that will eventually have defects Risky Files Files that have historically been defective Reviews of files that are both risky & future defective are less rigorous than files that are risky but clean
Defect-free (Clean) i.e., files that do not have defects Review Practice B What is the difference between code review practices of defective and clean files? Review Practice A VS Defective i.e., files that have defects We measure 3 dimensions of review activity metrics Review Intensity e.g., #Review Iterations, Discussion Length Review Participation e.g., #Reviewers, 
 Review Agreement Reviewing Time e.g., Review Length, Code Reading Speed Defective i.e., files that have defects Past We investigate defective files along 2 perspectives Future Reviews of future-defective files tend to be less rigorous than reviews of clean files Developers are not as careful when they review risky files. Future-Defective Files Files that will eventually have defects Risky Files Files that have historically been defective Reviews of files that are both risky & future defective are less rigorous than files that are risky but cleanpatanamon-t@is.naist.jp @pamon Investigating Code Review Practices in Defective Files Modern Code Review: A lightweight, tool-supported code review process Upstream VCS repositories Code change A lack of code review activity can increase the risk of post-release defects 
 [McIntosh et. al., MSR2014] My code is awesome! No needs for a review Code Review Tool Code change How should reviewers do a code review to reduce the risk of having defects? Examine Code

More Related Content

PDF
Review Participation in Modern Code Review: An Empirical Study of the Android...
The University of Adelaide
 
PDF
Using HPC Resources to Exploit Big Data for Code Review Analytics
The University of Adelaide
 
PDF
Who Should Review My Code?
The University of Adelaide
 
PDF
Ph.D. Thesis Defense: Studying Reviewer Selection and Involvement in Modern ...
The University of Adelaide
 
PDF
Improving Code Review Effectiveness Through Reviewer Recommendations
The University of Adelaide
 
PDF
Revisiting Code Ownership and Its Relationship with Software Quality in the S...
The University of Adelaide
 
PDF
Partitioning Composite Code Changes to Facilitate Code Review (MSR2015)
Sung Kim
 
PPT
Is Text Search an Effective Approach for Fault Localization: A Practitioners ...
Debdoot Mukherjee
 
Review Participation in Modern Code Review: An Empirical Study of the Android...
The University of Adelaide
 
Using HPC Resources to Exploit Big Data for Code Review Analytics
The University of Adelaide
 
Who Should Review My Code?
The University of Adelaide
 
Ph.D. Thesis Defense: Studying Reviewer Selection and Involvement in Modern ...
The University of Adelaide
 
Improving Code Review Effectiveness Through Reviewer Recommendations
The University of Adelaide
 
Revisiting Code Ownership and Its Relationship with Software Quality in the S...
The University of Adelaide
 
Partitioning Composite Code Changes to Facilitate Code Review (MSR2015)
Sung Kim
 
Is Text Search an Effective Approach for Fault Localization: A Practitioners ...
Debdoot Mukherjee
 

What's hot (20)

PPTX
An Empirical Study of Adoption of Software Testing in Open Source Projects
Pavneet Singh Kochhar
 
PDF
Software Engineering Culture - Improve Code Quality
Dmytro Patserkovskyi
 
PDF
Icsm20.ppt
Yann-Gaël Guéhéneuc
 
PDF
Cser13.ppt
Ptidej Team
 
PDF
Msr17a.ppt
Yann-Gaël Guéhéneuc
 
PPTX
Revisiting Assert Use in GitHub Projects
Pavneet Singh Kochhar
 
PDF
Evaluating SZZ Implementations Through a Developer-informed Oracle (ICSE 2021)
Giovanni Rosa
 
PDF
Static Analysis of Your OSS Project with Coverity
Samsung Open Source Group
 
PDF
Euro python 2015 writing quality code
radek_j
 
PDF
Python - code quality and production monitoring
David Melamed
 
PPTX
How To Improve Quality With Static Code Analysis
Perforce
 
PPTX
Partitioning composite code changes to facilitate code review
Yida Tao
 
PDF
Understanding, measuring and improving code quality in JavaScript
Mark Daggett
 
PDF
[India Merge World Tour] Coverity
Perforce
 
PDF
On the Distribution of Test Smells in Open Source Android Applications: An Ex...
University of Hawai‘i at Mānoa
 
PDF
Proactive Empirical Assessment of New Language Feature Adoption via Automated...
Raffi Khatchadourian
 
PDF
IDEAL: An Open-Source Identifier Name Appraisal Tool
University of Hawai‘i at Mānoa
 
PDF
Early Detection of Collaboration Conflicts & Risks in Software Development
Roopesh Jhurani
 
PDF
Social Debt Analytics for Improving the Management of Software Evolution Tasks
Fabio Palomba
 
PDF
On to code review lessons learned at microsoft
Michaela Greiler
 
An Empirical Study of Adoption of Software Testing in Open Source Projects
Pavneet Singh Kochhar
 
Software Engineering Culture - Improve Code Quality
Dmytro Patserkovskyi
 
Icsm20.ppt
Yann-Gaël Guéhéneuc
 
Cser13.ppt
Ptidej Team
 
Msr17a.ppt
Yann-Gaël Guéhéneuc
 
Revisiting Assert Use in GitHub Projects
Pavneet Singh Kochhar
 
Evaluating SZZ Implementations Through a Developer-informed Oracle (ICSE 2021)
Giovanni Rosa
 
Static Analysis of Your OSS Project with Coverity
Samsung Open Source Group
 
Euro python 2015 writing quality code
radek_j
 
Python - code quality and production monitoring
David Melamed
 
How To Improve Quality With Static Code Analysis
Perforce
 
Partitioning composite code changes to facilitate code review
Yida Tao
 
Understanding, measuring and improving code quality in JavaScript
Mark Daggett
 
[India Merge World Tour] Coverity
Perforce
 
On the Distribution of Test Smells in Open Source Android Applications: An Ex...
University of Hawai‘i at Mānoa
 
Proactive Empirical Assessment of New Language Feature Adoption via Automated...
Raffi Khatchadourian
 
IDEAL: An Open-Source Identifier Name Appraisal Tool
University of Hawai‘i at Mānoa
 
Early Detection of Collaboration Conflicts & Risks in Software Development
Roopesh Jhurani
 
Social Debt Analytics for Improving the Management of Software Evolution Tasks
Fabio Palomba
 
On to code review lessons learned at microsoft
Michaela Greiler
 
Ad

Similar to Investigating Code Review Practices in Defective Files (20)

PDF
Getting Ahead of Delivery Issues with Deep SDLC Analysis by Donald Belcham
.NET Conf UY
 
PPTX
Code Reviews
phildenoncourt
 
PDF
Defect Prediction: Accomplishments and Future Challenges
Yasutaka Kamei
 
PDF
How To Handle Your Tech Debt Better - Sean Moir
Mike Harris
 
PDF
Reading Summary - Effective Software Defect Tracking + Pragmatic Unit Testing
Artemisa Yescas Engler
 
PDF
Software Defect Prevention via Continuous Inspection
Josh Gough
 
PPT
Code Review
Ravi Raj
 
PDF
Legacy Coderetreat @Budapest 2013 02 16
Adi Bolboaca
 
PPTX
Unit3 software review control software
Reetesh Gupta
 
PPT
Verifcation &validation
ssusere50573
 
PPT
Parasoft .TEST, Write better C# Code Using Data Flow Analysis
Engineering Software Lab
 
PPT
Software Inspection And Defect Management
Ajay K
 
PPS
Estimating test effort part 2 of 2
Ian McDonald
 
PPT
Introduction to Software Review
Philip Johnson
 
PDF
Ady beleanu automate-theprocessdelivery
Romania Testing
 
PDF
Rtc2014 automate the_process_deliver_quality_ady_beleanu
Ady Beleanu
 
PPTX
Complement Software Testing with Static Analysis
John Ruberto
 
PPTX
Code_Review_Presentation_v22222_LLM.pptx
SofienBoutaib
 
PDF
Code Review: How and When
Paul Gower
 
PDF
Code Reviews @ Quatico
Jan Wloka
 
Getting Ahead of Delivery Issues with Deep SDLC Analysis by Donald Belcham
.NET Conf UY
 
Code Reviews
phildenoncourt
 
Defect Prediction: Accomplishments and Future Challenges
Yasutaka Kamei
 
How To Handle Your Tech Debt Better - Sean Moir
Mike Harris
 
Reading Summary - Effective Software Defect Tracking + Pragmatic Unit Testing
Artemisa Yescas Engler
 
Software Defect Prevention via Continuous Inspection
Josh Gough
 
Code Review
Ravi Raj
 
Legacy Coderetreat @Budapest 2013 02 16
Adi Bolboaca
 
Unit3 software review control software
Reetesh Gupta
 
Verifcation &validation
ssusere50573
 
Parasoft .TEST, Write better C# Code Using Data Flow Analysis
Engineering Software Lab
 
Software Inspection And Defect Management
Ajay K
 
Estimating test effort part 2 of 2
Ian McDonald
 
Introduction to Software Review
Philip Johnson
 
Ady beleanu automate-theprocessdelivery
Romania Testing
 
Rtc2014 automate the_process_deliver_quality_ady_beleanu
Ady Beleanu
 
Complement Software Testing with Static Analysis
John Ruberto
 
Code_Review_Presentation_v22222_LLM.pptx
SofienBoutaib
 
Code Review: How and When
Paul Gower
 
Code Reviews @ Quatico
Jan Wloka
 
Ad

Recently uploaded (20)

PPT
First Aid Training Presentation Slides.ppt
WilliamMigoya
 
PPTX
Tour Presentation Educational Activity.pptx
PaulineAngeliqueBere
 
PPTX
An Unlikely Response 08 10 2025.pptx
FamilyWorshipCenterD
 
PDF
Parts of Speech Prepositions Presentation in Colorful Cute Style_20250724_230...
ShomailahRashid
 
PPTX
Emphasizing It's Not The End 08 06 2025.pptx
FamilyWorshipCenterD
 
PPTX
Effective_Handling_Information_Presentation.pptx
HarisKaimKhani
 
PPTX
Project and change Managment: short video sequences for IBA
safwanhossain11
 
PPTX
Non-Verbal-Communication .mh.pdf_110245_compressed.pptx
ansarihussainh
 
PPTX
2025-08-10 Joseph 02 (shared slides).pptx
Dale Wells
 
PPTX
PHIL.-ASTRONOMY-AND-NAVIGATION of ..pptx
ajp27480
 
PPTX
Relationship Management Presentation In Banking.pptx
ssuser1d0512
 
DOCX
"Project Management: Ultimate Guide to Tools, Techniques, and Strategies (2025)"
Myplanningtemplates
 
PPTX
BIOLOGY TISSUE PPT CLASS 9 PROJECT PUBLIC
iamsrinjandatta
 
PDF
Presentation1 [Autosaved].pdf diagnosiss
moibrahim20044
 
PPTX
INTERNATIONAL LABOUR ORAGNISATION PPT ON SOCIAL SCIENCE
AbinayaaAnbazhakan
 
PPTX
Primary and secondary sources, and history
valenciamarcchristia
 
PPTX
ART-APP-REPORT-FINctrwxsg f fuy L-na.pptx
ajp27480
 
PPTX
fundraisepro pitch deck elegant and modern
aishanmims
 
PPTX
worship songs, in any order, compilation
wilmalenetoledo
 
DOC
学位双硕士UTAS毕业证,墨尔本理工学院毕业证留学硕士毕业证
enmytc
 
First Aid Training Presentation Slides.ppt
WilliamMigoya
 
Tour Presentation Educational Activity.pptx
PaulineAngeliqueBere
 
An Unlikely Response 08 10 2025.pptx
FamilyWorshipCenterD
 
Parts of Speech Prepositions Presentation in Colorful Cute Style_20250724_230...
ShomailahRashid
 
Emphasizing It's Not The End 08 06 2025.pptx
FamilyWorshipCenterD
 
Effective_Handling_Information_Presentation.pptx
HarisKaimKhani
 
Project and change Managment: short video sequences for IBA
safwanhossain11
 
Non-Verbal-Communication .mh.pdf_110245_compressed.pptx
ansarihussainh
 
2025-08-10 Joseph 02 (shared slides).pptx
Dale Wells
 
PHIL.-ASTRONOMY-AND-NAVIGATION of ..pptx
ajp27480
 
Relationship Management Presentation In Banking.pptx
ssuser1d0512
 
"Project Management: Ultimate Guide to Tools, Techniques, and Strategies (2025)"
Myplanningtemplates
 
BIOLOGY TISSUE PPT CLASS 9 PROJECT PUBLIC
iamsrinjandatta
 
Presentation1 [Autosaved].pdf diagnosiss
moibrahim20044
 
INTERNATIONAL LABOUR ORAGNISATION PPT ON SOCIAL SCIENCE
AbinayaaAnbazhakan
 
Primary and secondary sources, and history
valenciamarcchristia
 
ART-APP-REPORT-FINctrwxsg f fuy L-na.pptx
ajp27480
 
fundraisepro pitch deck elegant and modern
aishanmims
 
worship songs, in any order, compilation
wilmalenetoledo
 
学位双硕士UTAS毕业证,墨尔本理工学院毕业证留学硕士毕业证
enmytc
 

Investigating Code Review Practices in Defective Files

  • 1. Investigating Code Review Practices in Defective Files Patanamon (Pick) Thongtanunam Shane McIntosh Ahmed E. Hassan Hajimu Iida May 16-17, 2015. Firenze, Italy patanamon-t@is.naist.jp @pamon
  • 2. Modern Code Review: A lightweight, tool-supported code review process Code Review Tool
  • 3. Modern Code Review: A lightweight, tool-supported code review process Code Review Tool Code change
  • 4. Modern Code Review: A lightweight, tool-supported code review process Code Review Tool Code change
  • 5. Modern Code Review: A lightweight, tool-supported code review process Code Review Tool Code change Examine Code
  • 6. Modern Code Review: A lightweight, tool-supported code review process Upstream VCS repositories Code Review Tool Code change Examine Code
  • 7. Modern Code Review: A lightweight, tool-supported code review process Upstream VCS repositories Code change A lack of code review activity can increase the risk of post-release defects 
 [McIntosh et. al., MSR2014] My code is awesome! No needs for a review Code Review Tool Code change Examine Code
  • 8. Modern Code Review: A lightweight, tool-supported code review process Upstream VCS repositories Code change A lack of code review activity can increase the risk of post-release defects 
 [McIntosh et. al., MSR2014] My code is awesome! No needs for a review Code Review Tool Code change How should reviewers do a code review to reduce the risk of having defects? Examine Code
  • 9. What is the difference between code review practices of defective and clean files?
  • 10. What is the difference between code review practices of defective and clean files? Review Practice A Defective i.e., files that have defects
  • 11. Defect-free (Clean) i.e., files that do not have defects Review Practice B What is the difference between code review practices of defective and clean files? Review Practice A VS Defective i.e., files that have defects
  • 12. Defect-free (Clean) i.e., files that do not have defects Review Practice B What is the difference between code review practices of defective and clean files? Review Practice A VS Defective i.e., files that have defects
  • 13. We measure 3 dimensions of review activity metrics Review Intensity e.g., #Review Iterations, Discussion Length Review Participation e.g., #Reviewers, 
 Review Agreement Reviewing Time e.g., Review Length, Code Reading Speed
  • 14. Defect-free (Clean) i.e., files that do not have defects Review Practice B What is the difference between code review practices of defective and clean files? Review Practice A VS Defective i.e., files that have defects
  • 15. Defect-free (Clean) i.e., files that do not have defects Review Practice B What is the difference between code review practices of defective and clean files? Review Practice A VS Defective i.e., files that have defects
  • 16. Defective i.e., files that have defects We investigate defective files along 2 perspectives
  • 17. Defective i.e., files that have defects We investigate defective files along 2 perspectives
  • 18. Risky Files Files that have historically been defective Past Defective i.e., files that have defects We investigate defective files along 2 perspectives
  • 19. Risky Files Files that have historically been defective Past FutureDefective i.e., files that have defects Future-Defective Files Files that will eventually have defects We investigate defective files along 2 perspectives
  • 20. Risky Files Files that have historically been defective Past FutureDefective i.e., files that have defectsFuture-Defective Files Files that will eventually have defects Conjecture: Reviews of Future- Defective will be • less intense, • with less team participation, • completed with a shorter time than reviews of clean files We investigate defective files along 2 perspectives
  • 21. Future-Defective Files: Files that have post-release defects VCS Repositories
  • 22. Future-Defective Files: Files that have post-release defects VCS Repositories Release date Bug-fixing commit
  • 23. Future-Defective Files: Files that have post-release defects VCS Repositories Release date Bug-fixing commit Future-Defective
  • 24. Future-Defective Files: Files that have post-release defects VCS Repositories Release date No bug-fixing commits Release date Bug-fixing commit Future-Defective
  • 25. Future-Defective Files: Files that have post-release defects VCS Repositories Release date No bug-fixing commits Release date Bug-fixing commit Future-Defective Clean
  • 26. Future-Defective Files: Files that have post-release defects Studied ReviewsVCS Repositories Release date No bug-fixing commits Release date Bug-fixing commit 6 months Future-Defective Clean
  • 27. Future-Defective Files: Files that have post-release defects Studied ReviewsVCS Repositories Release date No bug-fixing commits Release date Bug-fixing commit 6 months 1,176 Files 3,470 Reviews 10,513 Files 2,727 Reviews 5.0.0 866 Files 2,849 Reviews 11,931 Files 2,690 Reviews 5.1.0 Future-Defective Clean
  • 28. #Reviewers #Reviews of Clean files #Reviewers #Reviews of Future-Defective files VS Review Activity Analysis: Compare code review activity that has been applied to future-defective and clean files
  • 29. #Reviewers #Reviews of Clean files #Reviewers #Reviews of Future-Defective files VS Using a statistical test to determine the difference between the distributions of code review activity Review Activity Analysis: Compare code review activity that has been applied to future-defective and clean files
  • 30. #Reviewers #Reviews of Clean files #Reviewers #Reviews of Future-Defective files VS Using a statistical test to determine the difference between the distributions of code review activity Raw code review activity metric is normalized by patch size Review Activity Analysis: Compare code review activity that has been applied to future-defective and clean files
  • 31. Findings Code review activity in the reviews of future-defective files Conjecture Results Review Intensity Review Participation Reviewing Time Less Intense Less Team Participation Completed with a shorter time
  • 32. Findings Code review activity in the reviews of future-defective files Conjecture Less Intense Results Review Intensity Review Participation Reviewing Time Less Intense Less Team Participation Completed with a shorter time
  • 33. Findings Code review activity in the reviews of future-defective files Conjecture Less Intense Less Team Participation Results Review Intensity Review Participation Reviewing Time Less Intense Less Team Participation Completed with a shorter time
  • 34. Findings Code review activity in the reviews of future-defective files Conjecture Less Intense Less Team Participation Faster Code Reading Rate Results Review Intensity Review Participation Reviewing Time Less Intense Less Team Participation Completed with a shorter time
  • 35. Defective i.e., files that have defects Past We investigate defective files along 2 perspectives Risky Files Files that have historically been defective FutureFuture-Defective Files Files that will eventually have defects Conjecture: Reviews of Future- Defective will be • less intense, • with less team participation, • completed with a shorter time than reviews of clean files
  • 36. Defective i.e., files that have defects Past We investigate defective files along 2 perspectives Risky Files Files that have historically been defective FutureFuture-Defective Files Files that will eventually have defects Reviews of future-defective files tend to be less rigorous than reviews of clean files
  • 37. Defective i.e., files that have defects Past We investigate defective files along 2 perspectives Risky Files Files that have historically been defective FutureFuture-Defective Files Files that will eventually have defects Reviews of future-defective files tend to be less rigorous than reviews of clean files
  • 38. Defective i.e., files that have defects Past We investigate defective files along 2 perspectives Risky Files Files that have historically been defective Conjecture: Reviews of risky files should be • more intense, • with more team participation, • reviewed for a longer time to reduce the risk of having defects in the future FutureFuture-Defective Files Files that will eventually have defects Reviews of future-defective files tend to be less rigorous than reviews of clean files
  • 39. Risky Files: Files that had post-release defects
 in prior release VCS Repositories
  • 40. Risky Files: Files that had post-release defects
 in prior release VCS Repositories Release date Prior release date Bug-fixing commit
  • 41. Risky Files: Files that had post-release defects
 in prior release VCS Repositories Release date Prior release date Bug-fixing commit Risky
  • 42. Risky Files: Files that had post-release defects
 in prior release VCS Repositories No bug-fixing commits Release date Prior release date Release date Prior release date Bug-fixing commit Risky
  • 43. Risky Files: Files that had post-release defects
 in prior release VCS Repositories No bug-fixing commits Release date Prior release date Release date Prior release date Bug-fixing commit Normal Risky
  • 44. Risky Files: Files that had post-release defects
 in prior release Studied ReviewsVCS Repositories No bug-fixing commits Release date Prior release date Release date Prior release date Bug-fixing commit 6 months Normal Risky
  • 45. Risky Files: Files that had post-release defects
 in prior release Studied ReviewsVCS Repositories No bug-fixing commits Release date Prior release date Release date Prior release date Bug-fixing commit 6 months 1,168 Files 2,671 Reviews 11,629 Files 2,868 Reviews 5.1.0 Normal Risky
  • 46. Findings Code review activity in the reviews of risky files Conjecture Results Review Intensity Review Participation Reviewing Time More Intense More Team Participation Completed with a longer time
  • 47. Findings Code review activity in the reviews of risky files Conjecture Less Intense Results Review Intensity Review Participation Reviewing Time More Intense More Team Participation Completed with a longer time
  • 48. Findings Code review activity in the reviews of risky files Conjecture Less Intense Less Team Participation Results Review Intensity Review Participation Reviewing Time More Intense More Team Participation Completed with a longer time
  • 49. Findings Code review activity in the reviews of risky files Conjecture Less Intense Less Team Participation Receive Slow Feedback & Faster Code Reading Rate Results Review Intensity Review Participation Reviewing Time More Intense More Team Participation Completed with a longer time
  • 50. Defective i.e., files that have defects Past We investigate defective files along 2 perspectives FutureFuture-Defective Files Files that will eventually have defects Reviews of future-defective files tend to be less rigorous than reviews of clean files Risky Files Files that have historically been defective Conjecture: Reviews of risky files should be • more intense, • with more team participation, • reviewed for a longer time to reduce the risk of having defects in the future
  • 51. Defective i.e., files that have defects Past We investigate defective files along 2 perspectives FutureFuture-Defective Files Files that will eventually have defects Reviews of future-defective files tend to be less rigorous than reviews of clean files Risky Files Files that have historically been defective Developers are not as careful when they review risky files.
  • 52. Defective i.e., files that have defects Past We investigate defective files along 2 perspectives Future Reviews of future-defective files tend to be less rigorous than reviews of clean files Developers are not as careful when they review risky files. Future-Defective Files Files that will eventually have defects Risky Files Files that have historically been defective
  • 53. Defective i.e., files that have defects Past We investigate defective files along 2 perspectives Future Reviews of future-defective files tend to be less rigorous than reviews of clean files Developers are not as careful when they review risky files. Future-Defective Files Files that will eventually have defects Risky Files Files that have historically been defective Will careless reviews of risky files lead to future defects?
  • 54. Defective i.e., files that have defects Past We investigate defective files along 2 perspectives Future Reviews of future-defective files tend to be less rigorous than reviews of clean files Developers are not as careful when they review risky files. Future-Defective Files Files that will eventually have defects Risky Files Files that have historically been defective Will careless reviews of risky files lead to future defects? Investigating code review practice in risky & future- defective files
  • 55. Risky & Future-Defective Files: Risky files that will eventually have defects VCS Repositories
  • 56. Risky & Future-Defective Files: Risky files that will eventually have defects VCS Repositories Bug-fixing commit Release date Prior release date Bug-fixing commit
  • 57. Risky & Future-Defective Files: Risky files that will eventually have defects VCS Repositories Bug-fixing commit Release date Prior release date Bug-fixing commit Risky & Future-Defective
  • 58. Risky & Future-Defective Files: Risky files that will eventually have defects VCS Repositories Bug-fixing commit Release date Prior release date Bug-fixing commit No bug-fixing commits Release datePrior release date Bug-fixing commit Risky & Future-Defective
  • 59. Risky & Future-Defective Files: Risky files that will eventually have defects VCS Repositories Bug-fixing commit Release date Prior release date Bug-fixing commit No bug-fixing commits Release datePrior release date Bug-fixing commit Risky & Clean Risky & Future-Defective
  • 60. Risky & Future-Defective Files: Risky files that will eventually have defects Studied Reviews VCS Repositories Bug-fixing commit Release date Prior release date Bug-fixing commit No bug-fixing commits Release datePrior release date Bug-fixing commit 6 months Risky & Clean Risky & Future-Defective
  • 61. Risky & Future-Defective Files: Risky files that will eventually have defects Studied Reviews VCS Repositories Bug-fixing commit Release date Prior release date Bug-fixing commit No bug-fixing commits Release datePrior release date Bug-fixing commit 6 months 206 Files 1,299 Reviews 962 Files 1,372 Reviews 5.1.0 Risky & Clean Risky & Future-Defective
  • 62. Findings Code review activity in the reviews of risky & future-defective files Conjecture Less Intense Less Team Participation Receive Slow Feedback & Faster Code Reading Rate Results Review Intensity Review Participation Reviewing Time Less Intense Less Team Participation Completed with a shorter time
  • 63. Defective i.e., files that have defects Past We investigate defective files along 2 perspectives Future Reviews of future-defective files tend to be less rigorous than reviews of clean files Developers are not as careful when they review risky files. Future-Defective Files Files that will eventually have defects Risky Files Files that have historically been defective Will careless reviews of risky files lead to future defects? Investigating code review practice in risky & future- defective files
  • 64. Defective i.e., files that have defects Past We investigate defective files along 2 perspectives Future Reviews of future-defective files tend to be less rigorous than reviews of clean files Developers are not as careful when they review risky files. Future-Defective Files Files that will eventually have defects Risky Files Files that have historically been defective Reviews of files that are both risky & future defective are less rigorous than files that are risky but clean
  • 65. Evolvability e.g., Fixing code comments,
 Decomposing complex function Functionality e.g., Fixing incorrect 
 program logic Traceability e.g., Updating commit message We compare concerns that are addressed during reviews of defective and clean files
  • 66. Evolvability Functionality Traceability Proportion of reviews in future-defective files in Qt5.0.0 82% 40%40% Reviews of defective files often address evolvability concernsResults 10% higher than clean files 5% higher than clean files 10% lower than clean files We observe the similar results for the reviews of risky files and risky & future-defective files
  • 68. Modern Code Review: A lightweight, tool-supported code review process Upstream VCS repositories Code change A lack of code review activity can increase the risk of post-release defects 
 [McIntosh et. al., MSR2014] My code is awesome! No needs for a review Code Review Tool Code change How should reviewers do a code review to reduce the risk of having defects? Examine Code
  • 69. Modern Code Review: A lightweight, tool-supported code review process Upstream VCS repositories Code change A lack of code review activity can increase the risk of post-release defects 
 [McIntosh et. al., MSR2014] My code is awesome! No needs for a review Code Review Tool Code change How should reviewers do a code review to reduce the risk of having defects? Examine Code Defect-free (Clean) i.e., files that do not have defects Review Practice B What is the difference between code review practices of defective and clean files? Review Practice A VS Defective i.e., files that have defects
  • 70. Modern Code Review: A lightweight, tool-supported code review process Upstream VCS repositories Code change A lack of code review activity can increase the risk of post-release defects 
 [McIntosh et. al., MSR2014] My code is awesome! No needs for a review Code Review Tool Code change How should reviewers do a code review to reduce the risk of having defects? Examine Code Defect-free (Clean) i.e., files that do not have defects Review Practice B What is the difference between code review practices of defective and clean files? Review Practice A VS Defective i.e., files that have defects We measure 3 dimensions of review activity metrics Review Intensity e.g., #Review Iterations, Discussion Length Review Participation e.g., #Reviewers, 
 Review Agreement Reviewing Time e.g., Review Length, Code Reading Speed
  • 71. Modern Code Review: A lightweight, tool-supported code review process Upstream VCS repositories Code change A lack of code review activity can increase the risk of post-release defects 
 [McIntosh et. al., MSR2014] My code is awesome! No needs for a review Code Review Tool Code change How should reviewers do a code review to reduce the risk of having defects? Examine Code Defect-free (Clean) i.e., files that do not have defects Review Practice B What is the difference between code review practices of defective and clean files? Review Practice A VS Defective i.e., files that have defects We measure 3 dimensions of review activity metrics Review Intensity e.g., #Review Iterations, Discussion Length Review Participation e.g., #Reviewers, 
 Review Agreement Reviewing Time e.g., Review Length, Code Reading Speed Defective i.e., files that have defects Past We investigate defective files along 2 perspectives Future Reviews of future-defective files tend to be less rigorous than reviews of clean files Developers are not as careful when they review risky files. Future-Defective Files Files that will eventually have defects Risky Files Files that have historically been defective Reviews of files that are both risky & future defective are less rigorous than files that are risky but clean
  • 72. Defect-free (Clean) i.e., files that do not have defects Review Practice B What is the difference between code review practices of defective and clean files? Review Practice A VS Defective i.e., files that have defects We measure 3 dimensions of review activity metrics Review Intensity e.g., #Review Iterations, Discussion Length Review Participation e.g., #Reviewers, 
 Review Agreement Reviewing Time e.g., Review Length, Code Reading Speed Defective i.e., files that have defects Past We investigate defective files along 2 perspectives Future Reviews of future-defective files tend to be less rigorous than reviews of clean files Developers are not as careful when they review risky files. Future-Defective Files Files that will eventually have defects Risky Files Files that have historically been defective Reviews of files that are both risky & future defective are less rigorous than files that are risky but cleanpatanamon-t@is.naist.jp @pamon Investigating Code Review Practices in Defective Files Modern Code Review: A lightweight, tool-supported code review process Upstream VCS repositories Code change A lack of code review activity can increase the risk of post-release defects 
 [McIntosh et. al., MSR2014] My code is awesome! No needs for a review Code Review Tool Code change How should reviewers do a code review to reduce the risk of having defects? Examine Code