Ph.D. Dissertation - Studying the Impact of Developer Communication on the Quality and Evolution of a Software

T H E I M PA C T O F D E V E L O P E R C O M M U N I C AT I O N
O N T H E Q U A L I T Y A N D E V O L U T I O N O F A
S O F T WA R E S Y S T E M
Nicolas Bettenburg
May 20th, 2014
Queen’s University
School of Computing
A T H E S I S P R E S E N T E D T O T H E S C H O O L O F C O M P U T I N G
I N C O N F O R M I T Y W I T H T H E R E Q U I R E M E N T S
F O R T H E D E G R E E O F P H I L O S O P H I C A L D O C T O R

!2
OUTLINEPRESENTATION
B E G I N
I
Studying the Impact of Developer Communication on the Quality and Evolution of a Software System
I I
I V
E N D
MOTIVATION
LITERATURE 
REVIEW
ANALYSIS & 
RESULTS
I I I
TOOLS
V
CONCLUSIONS
i

I
INTRODUCTION
AND 
MOTIVATION
i

!4
MOTIVATIONRESEARCH

!4
MOTIVATIONRESEARCH
• Software Development is a highly social, collaborative process.
[Cataldo et al., CSCW 2006]

!4
MOTIVATIONRESEARCH
• Developers spend up to 50% of their time communicating.
[Perry et al., IEEE Software 1994]

!4
MOTIVATIONRESEARCH
• Developers spend up to 50% of their time communicating.
[Perry et al., IEEE Software 1994]
If communication plays such an integral part in developers’ work lives,  
does it affect software development? How?

!5
MOTIVATIONRESEARCH

!5
MOTIVATIONRESEARCH
How can we put
communication data
in a relationship to
software quality?

!5
MOTIVATIONRESEARCH
How can we put
communication data
software quality?
How can we put
communication data
software evolution?

!5
MOTIVATIONRESEARCH
Where do we find
communication
data? How can we
mine communication 
data ?
How can we put
communication data
software quality?
How can we put
communication data
software evolution?

!6
HYPOTHESISRESEARCH
“The communication between software developers plays a
key role in the quality and evolution of the software.”

!6
HYPOTHESISRESEARCH
Literature Review
What is already known
How that knowledge was gained

!6
HYPOTHESISRESEARCH
Literature Review
Tools & Techniques
Mining Communication Data
Preparation for use in Experiments

!6
HYPOTHESISRESEARCH
Literature Review
Tools & Techniques
Software Quality
Internal Developers Communication
Discussing Bugs in the Software

!6
HYPOTHESISRESEARCH
Literature Review
Tools & Techniques
Software Quality
Internal Developers Communication
Discussing Bugs in the Software
Software Evolution
Communication between Developers  
and External Contributors
Discussing Source Code Contributions

!8
REVIEWLITERATURE
Establish what is already known within the field of study about the relationships 
between socio-technical information about the software, the software development 
process, and the stakeholders, and software quality.
GOAL
• To determine what has already been investigated
• To gain a better understanding of the extraction of socio-technical information
• To identify challenges and open problems

!9
REVIEWLITERATURE
DIMENSIONS
Socio-Technical Metrics
Network Metrics
Technical Metrics
Social Metrics
Data Sources and Extraction
Methodology
Quality Metrics
Previous Results

!1 0Studying the Impact of Developer Communication on the Quality and Evolution of a Software System
REVIEWLITERATURE
MAJOR FINDINGS

REVIEWLITERATURE
MAJOR FINDINGS
• Social and Socio-Technical information can explain quality aspects
(software vulnerabilities) that technical information on its own cannot.

REVIEWLITERATURE
MAJOR FINDINGS
• Combinations of Social and Technical Information yield better
Models than using each source of information on its own.

REVIEWLITERATURE
MAJOR FINDINGS
• Socio-Technical Networks Together with Congruence Measures
provide empirical evidence that Conway’s Law does exist.
• Combinations of Social and Technical Information yield better
Models than using each source of information on its own.

!1 2
TOOLS AND TECHNIQUES
Off-The-Shelf Tools from Information Retrieval and Natural Language Processing 
are not readily usable for mining communication data found in software repositories.
REALIZATION

!1 2
REALIZATION
• Emails and Chat messages are not newspaper articles!

!1 2
REALIZATION
• Unstructured data = mixtures of Natural Language text and Technical Information

!1 2
REALIZATION
• Unstructured data = mixtures of Natural Language text and Technical Information
• Automated processing of extracted communication data

UNSTRUCTURED DATAMINING
From geek+@cmu.edu Wed Jan 21 08:11:26 1998
Date: Mon, 27 Jan 1997 12:50:44 -0500 (EST)
From: "Brian E. Gallew" <geek+@cmu.edu>
Subject: Re: [HACKERS] configure
!- ---559023410-851401618-854387445=:824
Content-Type: TEXT/PLAIN; CHARSET=US-ASCII
!> If you can grab a copy and run it on your machine, and send me
> the output, that would help alot.
!Here is a gzip'ed tar of the results.
!!=====================================================================
| Please do not shoot at the thermonuclear weapons! -- Deacon |
=====================================================================
| Finger geek@andrew.cmu.edu for my public key. |
=====================================================================
!- ---559023410-851401618-854387445=:824
Content-Type: APPLICATION/x-gzip
Content-Transfer-Encoding: BASE64
Content-Description: m88k-dg-dgux5.4R3.10.tar.gz
!H4sIAHDq7DICA+xba3vaSLLOV/MrepzsGHiQuNomOJ4MdnDMrC8csB17HQ8W
UgM9FpJWFxsmyX8/Vd0tIYGwye5kP+w5fp4EaHW9XV1dXbdu6bY1ZCNV1/Qx
ffWD/sql0k6tRl4RUq6IT1KWn4R/L5UI2ansVirVSrlWhpZqqVZ5RUqv/gN/
gedrLiGvHNvzRy71VvUbUYu6mvnqv+zvNbkYM48MmUkJfGrEG1PTJJ7uMscn
/ljzCdcND75TAvIJTN8j9pDoXHECl2ZeE5960OgGFrEt6Ac43szz6YR4NpLN
!

TECHNICAL INFORMATIONEXTRACTING
1
2
3

COMMUNICATION DATA TO CODELINKING
Description
Comment 1
Depends on:
Blocks:
Show dependency tree
Attachments
Add an attachment (proposed patch, testcase, etc.)
Note
You need to log in before you can comment on or make changes to this bug.
Markus Keller 2004-11-30 13:27:58 EST
I20041130-0800
Wrong compiler error when interface overrides two methods with same signature
but different thrown exceptions: The call to ij.m() is OK, but eclipse flags it
with "Unhandled exception type IOException".
public class Over {
void x() throws ZipException {
IandJ ij= new K();
ij.m(); //wrong compile error
}
void y() throws ZipException {
K k= new K();
k.m();
}
}
interface I { void m() throws IOException; }
interface J { void m() throws ZipException; }
interface IandJ extends I, J {} // swap I and J to make compile error disappear
class K implements IandJ { public void m() throws ZipException { } }
Kent Johnson 2004-12-01 14:14:02 EST
This is not a MethodVerifier problem.
This error is thrown from FlowContext.checkExceptionHandlers()
X.java
Y.java
Over.java
ZipExcepti
on.java
IOExceptio
n.java
I.java J.javaK.java

!1 6
SUMMARYTOOLS AND TECHNIQUES

!1 6
Data Mining
Tools and
Techniques
for mining
communication
data from
software
repositories

!1 6
Data Mining
Tools and
Techniques
for mining
communication
data from
software
repositories
Data Extraction
Tools and
Techniques
for extracting
technical
information
data from
unstructured data

!1 6
Data Mining
Tools and
Techniques
for mining
communication
data from
software
repositories
Data Extraction
Tools and
Techniques
for extracting
technical
information
data from
unstructured data
Linking
Different
conceptual
approaches
for linking
communication
data to source
code

!1 6
Data Mining
Tools and
Techniques
for mining
communication
data from
software
repositories
Data Extraction
Tools and
Techniques
for extracting
technical
information
data from
unstructured data
Linking
Different
conceptual
approaches
for linking
communication
data to source
code
MUD Workshop
Advancing the
state of the art of
mining
unstructured data
for software
engineering
research

!1 8
SOFTWARE QUALITY
The Relationships Between Communication
And
PART 1

!1 9
OVERVIEWEXPERIMENTAL SETUP
DISCUSSION
TOOLS
METRICS
MODELPREDICTIONS

!1 9
DISCUSSION
TOOLS
METRICS
MODELPREDICTIONS
VARIABLES

!1 9
DISCUSSION
TOOLS
METRICS
MODELPREDICTIONS
VARIABLES
UNDERSTANDING

!2 0
OVERVIEWSOCIO-TECHNICAL METRICS
Socio-Technical
DIMENSIONS

!2 0
Socio-Technical
DIMENSIONS
Discussion Contents
• Source Code
• Patches
• Stack Traces

!2 0
Socio-Technical
DIMENSIONS
Discussion Contents
• Source Code
• Patches
• Stack Traces
Communication Dynamics
• # Message
• Reply Time
• Length
• Subscribers

!2 0
Socio-Technical
DIMENSIONS
Discussion Contents
• Source Code
• Patches
• Stack Traces
Social Structures
• # Discussion Participants
• Role in Project
• Reputation
• SNA Centrality
• # Message
• Reply Time
• Length
• Subscribers

!2 0
Socio-Technical
DIMENSIONS
Discussion Contents
• Source Code
• Patches
• Stack Traces
Social Structures
• # Discussion Participants
• Role in Project
• Reputation
• SNA Centrality
• # Message
• Reply Time
• Length
• Subscribers
Workﬂow & Coordination
• (Re-)Assignments in 
Lifecycle Model

!2 1
FINDINGSMAIN

!2 1
FINDINGSMAIN
• We can use Statistical Models for Prediction AND Understanding

!2 1
FINDINGSMAIN
• Communication Metrics can explain Software Defects as well as traditional 
product and process metrics.

!2 1
FINDINGSMAIN
• Communication Metrics do NOT describe the same information as traditional 

!2 1
FINDINGSMAIN
• Communication Metrics do NOT describe the same information as traditional 
• Combinations of communication metrics with traditional metrics results 
in more powerful models.

!2 2
SOFTWARE EVOLUTION
The Relationships Between Communication
And
PART 2

!2 3
SETUPEXPERIMENTAL
MAILING
LIST (LINUX)
TOOLS
METRICS
MODELS
VARIABLES
GERRIT
(ANDROID)
MANUAL ANALYSIS
LITERATURE
& 
DOCUMENTATION
CONTRIBUTION 
MANAGEMENT 
MODEL

!2 4
FINDINGSMAIN

!2 4
FINDINGSMAIN
• External Contributors are available for communication only for a limited time.

!2 4
FINDINGSMAIN
• Communication problems can lead to wasting efforts in implementing 
undesired or already existing functionality.

!2 4
FINDINGSMAIN
• Communication problems can lead to wasting efforts in implementing 
undesired or already existing functionality.
• Delays in communicating review outcome are a key factor for abandoning 
desirable code contributions.

V
CONCLUSIONS
AND 
FUTURE WORK

!2 6
CONCLUSIONSSUMMARY

!2 6
CONCLUSIONSSUMMARY
• Software Development has a SOCIAL SIDE, too!

!2 6
CONCLUSIONSSUMMARY
• Socio-Technical Information is valuable and can help us gain a more 
holistic understanding of software quality.

!2 6
CONCLUSIONSSUMMARY
• Communication delays found to be a strong explainer for  
software defects.

!2 6
CONCLUSIONSSUMMARY
• Communication delays found to be a strong explainer for  
software defects.
• Communication delays found to be a key factor for abandoning  
code contributions.

!2 7
AHEADTHE ROAD

!2 7
AHEADTHE ROAD
• What other Socio-Technical Metrics exist?

!2 7
AHEADTHE ROAD
• How can we get them?

!2 7
AHEADTHE ROAD
• Measuring, Predicting, and Understanding not Enough!

!2 7
AHEADTHE ROAD
• How can we make social metrics actionable?

!2 7
AHEADTHE ROAD
• Looking at Repositories is not enough!

!2 7
AHEADTHE ROAD
• Looking at Repositories is not enough!
• Go to Google, use Google Glass to record and document “water cooler chat”.
• Go to Mozilla and IBM, corroborate findings with the actual developers.
• Use voice recognition software to mine face-to-face communication data.
PROJECTSFUN STUDENT

?B E G I N
I I I
I V
E N D
MOTIVATION
LITERATURE 
REVIEW
ANALYSIS & 
RESULTS
I I I
TOOLS
V
CONCLUSIONS
i
!

Ph.D. Dissertation - Studying the Impact of Developer Communication on the Quality and Evolution of a Software

More Related Content

Similar to Ph.D. Dissertation - Studying the Impact of Developer Communication on the Quality and Evolution of a Software (20)

More from Nicolas Bettenburg (20)

Recently uploaded (20)

Ph.D. Dissertation - Studying the Impact of Developer Communication on the Quality and Evolution of a Software