Vejovis: Suggesting Fixes for JavaScript Faults

Vejovis: Suggesting Fixes for
JavaScript Faults
Frolin S. Ocariza, Jr.
Karthik Pattabiraman, Ali Mesbah
University of British Columbia

Problem and Motivation
2
JavaScript in web applications
has plenty of reliability issues
JS faults are not trivially
Average of 4 JavaScript
faults in production
websites [ISSRE’11]
JS faults lead to major
functionality/fixed [ESEM’security
13]
issues [ESEM’13]

Problem and Motivation
3
JavaScript in web applications
has plenty of reliability issues,
these JavaScript faults matter
and these JavaScript faults are
non-trivial to fix

Faults in JavaScript Code
} Study of JS bug reports [ESEM’13]
} Key Insight: Most (65%) mistakes programmers make in
JS propagate to parameters of DOM API method calls
} DOM API methods: getElementById, getElementsByTagName,
jQuery’s $(), etc.
} We also found that such faults are the most impactful, and take
the longest to fix
DOM-RELATED FAULTS
4

DOM-Related Fault Example
var x = “yes”;!
var elem = document.getElementById(x);!
5
div
div
div
id = “yes”

var x = “no”;!
6
div
div
div
id = “yes”
MISTAKE!

var x = “no”;!
7
div
div
div
id = “yes”
MISTAKE!
ID parameter evaluates to “no”, which is
not in the DOM

Goal
8
Facilitate the process of fixing DOM-related
faults

Fault Model
} Suggest repairs for DOM-related faults
} Only one mistake made
9

Common Developer Fixes
} Study of 190 fixed bug reports from 12 web apps
elem = getElementById(param)
elem.innerHTML = “…”
10

elem = getElementById(new_param)
Ways Programmers Fix Faults
11
Modify the parameter
• Parameter Modification

if (elem)
• DOM Element Validation
12
Check if null

elem = querySelector(param)
Ways Programmers Fix Faults Modify the method
• Method Modification
13

• Method Modification
14
27.2%
25.7%
24.6%

Structure in DOM Method Parameters
getElementById(“no”)! ???!
15
WRONG RIGHT
Question: How do we know that we
should replace “no” with “yes”
Answer: We need to infer
programmer intent
- Very difficult to do in general,
but…
- We have the DOM!
div
div
div
id = “yes”

Structure in DOM Method Parameters
getElementById(“no”)! getElementById(“yes”)!
16
WRONG RIGHT
Question: How do we know that we
should replace “no” with “yes”
Answer: We need to infer
programmer intent
- Very difficult to do in general,
but…
- We have the DOM!
div
div
div
id = “yes”

CSS Selectors
17
div#sample > table tr.hello
Tag name “tr”
Class name “hello”
Is descendant
Tag name “div”
Is child
Tag name “table”
ID “sample”
Input to querySelector(), $(), etc. to retrieve
list of elements

Design
18
Direct DOM Access
Data Collector
(box a)
Web Application
URL
Symptom Analyzer
(box b)
Treatment
Suggester
(box c)
Supplementary
Information
Symptoms
Data
Possible Sicknesses
List of Workaround
Suggestions
Figure 3: High-level block diagram of our design.

Running Example
1 firstTag = “div”;!
2 prefix = “pain-”;!
3 suffix = “elem”;!
4 level1 = firstTag + “#” + prefix + suffix;!
5 level2 = “span.cls”;!
6 e = $(level1 + “ “ + level2);!
7 e[0].innerHTML = “new content”;!
19
Access
DOM
element
using CSS
selector

Running Example
6 e = $(level1 + “ “ + level2);!
20
Lines to set
up the
CSS selector
passed to $()

Running Example
6 e = $(level1 + “ “ + level2);!
21
Constructed selector: div#pain-elem span.cls

Running Example
6 e = $(level1 + “ “ + level2);!
22
div
Id = “main-elem”
div
Id = “wrapper”
span
class=“cls”
span
class=“cls”

Running Example
6 e = $(level1 + “ “ + level2);!
23
div
div
Id = “wrapper”
span
class=“cls”
span
class=“cls”
Would return
empty set!

Main Idea
} Parameter Analysis: What portion of the parameter do
we replace?
} Context Analysis: How do we perform the replacement
in the code?
where a call to aDOMAPI meth-od
added, removed, or modified in
modification refers to changing
originally called, not the param-eter
getElementsByClassName, the
is called instead). This cat-egory
24
fixes.
significantly large portions of the Java-
restructured to implement the
10.5% of the fixes.
12% of the fixes.
the most prominent cate-gories
DOM Element Validation,
the fixes. Therefore, we fo-cus
Although we do not consider
repair approach, our algo-rithm
class of errors, at the cost of
7).
describe how programmers modify
We discuss our findings for
– Parameter Modification,
Direct DOM Access
Data Collector
(box a)
Web Application
URL
Symptom Analyzer
(box b)
Treatment
Suggester
(box c)
Supplementary
Information
Symptoms
Data
Possible Sicknesses
List of Workaround
Suggestions
Summary of Findings. Our study shows that the most prominent
fix categories are Parameter Modification and DOM Element Vali-dation.
Our analysis also shows the prevalence of string value mod-ifications

Parameter Analysis: Dividing
Components
25
Method/Property Modification, where a call to aDOMAPI meth-od
(or property) is either added, removed, or modified in
the JavaScript code. Here, modification refers to changing
the method (or property) originally called, not the param-eter
(e.g., instead of calling getElementsByClassName, the
method getElementsByTagName is called instead). This cat-egory
makes up 24.6% of the fixes.
Major Refactoring, where significantly large portions of the Java-
Invalid selector: Script code div#are modified pain-and restructured to implement the
fix. This category makes up 10.5% elem of the fixes.
span.cls
Other/Uncategorized, which make up 12% of the fixes.
As seen in the above fix categories, the most prominent cate-gories
are Parameter Modification and DOM Element Validation,
which make up over half (52.9%) of the fixes. Therefore, we fo-cus
on these categories in our work. Although we do not consider
Divide into components
Method/Property Modifications in our repair approach, our algo-rithm
can be adapted to include this class of errors, at the cost of
increasing its complexity (see Section 7).
Application of Fixes. We next describe how programmers modify
the JavaScript code to apply the fixes. We discuss our findings for
the three most prominent fix categories – Parameter Modification,
DOM Element Validation, and Method/Property Modification.
Parameter Modification: We found that 67.3% of fixes belong-ing
Direct DOM Access
div | # | pain-elem | | span | . | cls
to the Parameter Modification fix category involve the modifi-cation
of string values. The vast majority (around 70%) of these
string value modifications were direct modifications of string liter-als
in the JavaScript code. However, we also found cases where
the string value modification was applied by adding a call to string
modification methods such as replace().
We also analyzed the DOM methods/properties whose parame-ters
are affected by the modified values. For string value modifica-tions,
the methods/properties involved in multiple bug report fixes
are getElementById(), $() and jQuery(); together, fixes involv-ing
these methods comprise 51.4% of all string value modifications.
For non-string value modifications, fixes involved modification of
the numerical values assigned to elements’ style properties, partic-ularly
their alignment and scroll position.
DOM Element Validation: 75.5% of fixes belonging to this cat-egory
are applied by simply wrapping the code using the pertinent
DOM element within an if statement that performs the necessary
validation (so that the code only executes if the check passes).
Data Collector
(box a)
Web Application
URL
Symptom Analyzer
(box b)
Treatment
Suggester
(box c)
Supplementary
Information
Symptoms
Data
Possible Sicknesses
List of Workaround
Suggestions
Summary of Findings. Our study shows that the most fix categories are Parameter Modification and DOM Element Our analysis also shows the prevalence of string value and null/undefined checks when applying fixes. most parameter modifications are for values eventually in DOM methods that retrieve elements from the DOM, the $(), jQuery() and getElementById() methods. results motivate our fault model choice in Section 4 as well choice of possible sickness classes in Section 5.2.
4. FAULT MODEL
In this work, we focus on DOM API methods that retrieve from the DOM using CSS selectors, IDs, tag names, names, as we found that these were the common sources made by programmers (Section 3). These DOM API include getElementById(), getElementsByTagName(), querySelector(), and querySelect-orAll().
We also support DOM API wrapper methods made by commonly used JavaScript libraries including those (e.g., $() and jQuery()); Prototype (e.g., $$() and tinyMCE (e.g., get()), among others. For simplicity, we to all these DOM API methods as the direct DOM access.
We further focus on code-terminating DOM-related faults,

Components
26
Invalid selector: div#pain-elem span.cls
Divide into components
Direct DOM Access
div | # | pain-elem | | span | . | cls
tag has-id id has-descendant
tag has-class
class
Script code are modified and restructured to implement the
fix. This category makes up 10.5% of the fixes.
Data Collector
(box a)
Web Application
URL
Symptom Analyzer
(box b)
Treatment
Suggester
(box c)
Supplementary
Information
Symptoms
Data
Possible Sicknesses
List of Workaround
Suggestions
4. FAULT MODEL

Components
27
span.cls
Subdivide each Method/Property component Modifications in our repair approach, according our algo-rithm
to
dynamic backward can be adapted to slice
include this class of errors, at the cost of
Direct DOM Access
div | # | pain- | elem | | span | . | cls
Line 2 Line 3
Data Collector
(box a)
Web Application
URL
Symptom Analyzer
(box b)
Treatment
Suggester
(box c)
Supplementary
Information
Symptoms
Data
Possible Sicknesses
List of Workaround
Suggestions
4. FAULT MODEL

Components
6 e = $(level1 + “ “ + level2);!
28
Backward slice
of “pain-elem”
div
div
Id = “wrapper”
span
class=“cls”
span
class=“cls”

Components
29
span.cls
Subdivide each Method/Property component Modifications in our repair approach, according our algo-rithm
to
dynamic backward can be adapted to slice
include this class of errors, at the cost of
Direct DOM Access
Data Collector
(box a)
Web Application
URL
Symptom Analyzer
(box b)
Treatment
Suggester
(box c)
Supplementary
Information
Symptoms
Data
Possible Sicknesses
List of Workaround
Suggestions
4. FAULT MODEL

Parameter Analysis: Finding
Valid Selectors
30
span.cls
div
div
Id = “wrapper”
the three most prominent fix categories – span
Parameter Modification,
DOM Element Validation, and Method/class=“Property Modification.
Parameter Modification: We found that 67.3% cls”
of fixes belong-ing
span
class=“cls”
Direct DOM Access
Construct VALID selectors are affected by the modified values. For string value modifica-tions,
the methods/properties from involved in current multiple bug report fixes
DOM that are
“sufficiently close” are getElementById(), $() and jQuery(); together, these to methods the comprise erroneous 51.4% of all string value one
fixes involv-ing
modifications.
Data Collector
(box a)
Web Application
URL
Symptom Analyzer
(box b)
Treatment
Suggester
(box c)
Supplementary
Information
Symptoms
Data
Possible Sicknesses
List of Workaround
Suggestions
4. FAULT MODEL

Parameter Analysis: Finding
Valid Selectors
31
span.cls
List of valid selectors:
div#main-elem span.cls
div#wrapper span.cls
div
div
Id = “wrapper”
the three most prominent fix categories – span
Parameter Modification,
DOM Element Validation, and Method/class=“Property Modification.
Parameter Modification: We found that 67.3% cls”
of fixes belong-ing
span
class=“cls”
Direct DOM Access
Construct VALID selectors are affected by the modified values. For string value modifica-tions,
the methods/properties from involved in current multiple bug report fixes
DOM that are
“sufficiently close” are getElementById(), $() and jQuery(); together, these to methods the comprise erroneous 51.4% of all string value one
fixes involv-ing
modifications.
Data Collector
(box a)
Web Application
URL
Symptom Analyzer
(box b)
Treatment
Suggester
(box c)
Supplementary
Information
Symptoms
Data
Possible Sicknesses
List of Workaround
Suggestions
4. FAULT MODEL

Parameter Analysis: Inferring Possible
Replacements [Example]
32

33
Assumed
incorrect

34
div | # | | elem | | span | . | cls
Use as pattern
div#main-elem span.cls

35
div | # | | elem | | span | . | cls
div#main-elem span.cls – MATCHES PATTERN!

Context Analysis
Direct DOM Access
6 e = $(level1 + “ “ + level2);!
36
Invalid selector: div#the string value modification was applied by adding a call to string
modification pain-methods such elem as replace().
span.cls
Replacement selector: are affected by the div#modified values. For string value modifica-tions,
the methods/properties involved main-in multiple bug report elem fixes
span.cls
Other modifications include (1) adding a check before the DOM
element is used so that the method returns if the check fails; (2)
adding a check before the DOM element is used such that the value
of the DOM element or its property is updated if the check fails;
(3) encapsulating the code using the DOM element in an if-else
statement so that a backup value can be used in case the check
fails; and finally (4) encapsulating the code in a try-catch state-ment.
The most prevalent checks are null/undefined checks, i.e.,
the code has been modified to check if the DOM element is null or
Data Collector
(box a)
Web Application
URL
Symptom Analyzer
(box b)
Treatment
Suggester
(box c)
Supplementary
Information
Symptoms
Data
Possible Sicknesses
List of Workaround
Suggestions
fix categories are Parameter Modification and DOM Element Our analysis also shows the prevalence of string value and null/undefined checks when applying fixes. most parameter modifications are for values eventually in DOM methods that retrieve elements from the DOM, particu-larly
the $(), jQuery() and getElementById() methods. results motivate our fault model choice in Section 4 as well as choice of possible sickness classes in Section 5.2.
4. FAULT MODEL
In this work, we focus on DOM API methods that retrieve from the DOM using CSS selectors, IDs, tag names, or names, as we found that these were the common sources of made by programmers (Section 3). These DOM API methods
include getElementById(), getElementsByTagName(), getEl-ementsByClassName(),
querySelector(), and querySelect-orAll().
We also support DOM API wrapper methods made by commonly used JavaScript libraries including those in (e.g., $() and jQuery()); Prototype (e.g., $$() and $()); tinyMCE (e.g., get()), among others. For simplicity, we will to all these DOM API methods as the direct DOM access.
We further focus on code-terminating DOM-related faults, means theDOMAPI method returns null, undefined, or an set of elements, eventually leading to a null or an undefined excep-tion
(thereby terminating JavaScript execution). However, our can also be extended to apply to output-related DOM-related
faults, i.e., those that lead to incorrect output manifested on DOM. Such faults would require the programmer to manually the directDOMaccess. In contrast, with code-terminating faults, the direct DOM access can be determined automat-ically
using the AUTOFLOX tool proposed in our prior work

Context Analysis
Direct DOM Access
2 prefix = “main-”;!
6 e = $(level1 + “ “ + level2);!
37
String literal
replaced
Replacement selector: div#main-elem span.cls
Data Collector
(box a)
Web Application
URL
Symptom Analyzer
(box b)
Treatment
Suggester
(box c)
Supplementary
Information
Symptoms
Data
Possible Sicknesses
List of Workaround
Suggestions
4. FAULT MODEL

Context Analysis
Direct DOM Access
2 prefix = “main-”;!
6 e = $(level1 + “ “ + level2);!
38
String literal
replaced
Replacement selector: div#main-elem span.cls
Message:
REPLACE STRING LITERAL are applied by simply “wrapping pain-” the code using the in pertinent
line 2 with string
literal “main-”
Data Collector
(box a)
Web Application
URL
Symptom Analyzer
(box b)
Treatment
Suggester
(box c)
Supplementary
Information
Symptoms
Data
Possible Sicknesses
List of Workaround
Suggestions
4. FAULT MODEL

Context Analysis: Non-”Replace”
Messages
} Loops – “replace” may be unsafe
} String value doesn’t originate from string literal
} Analyze the context!
39
MESSAGE TYPES
REPLACE
REPLACE AT ITERATION
OFF BY ONE AT BEGINNING
OFF BY ONE AT END
MODIFY UPPER BOUND
EXCLUDE ITERATION
ENSURE

Implementation
} Vejovis
http://ece.ubc.ca/~frolino/projects/vejovis
} Data collection: Rhino and Crawljax
} Pattern matching: Hampi
40

Usage Model
41
VEJOVIS
INPUTS
OUTPUT
URL AUTOFLOX
DOM METHOD LOCATION
LIST OF ACTIONABLE REPAIR MESSAGES

Evaluation: Research Questions
RQ1: What is the accuracy of Vejovis in suggesting a correct
repair?
RQ2: How quickly can Vejovis determine possible
replacements? What is its performance overhead?
42

RQ1: Accuracy of Vejovis
Subjects JS Code Size
(KB)
Drupal 213
Ember.js 745
Joomla 434
jQuery 94
Moodle 352
MooTools 101
Prototype 164
Roundcube 729
TYPO3 2252
WikiMedia 160
WordPress 197
43
• 22 bug reports (2 per app,
and randomly chosen)
• Replicated bug and ran
with Vejovis
• Recall and Precision
RECALL: 100% if correct fix
appears; 0% otherwise
PRECISION: Measure of
extraneous suggestions

RQ1: Recall
Subject Bug Report
#1
Bug Report
#2
Drupal ✔ ✔
Ember.js ✔ ✔
Joomla ✔ ✔
jQuery ✔ ✗
Moodle ✔ ✔
MooTools ✔ ✔
Prototype ✔ ✔
Roundcube ✔ ✗
TYPO3 ✔ ✔
WikiMedia ✔ ✔
WordPress ✔ ✔
44
Overall
Recall: 91%

RQ1: Precision
Subject Bug Report
#1
Bug Report
#2
Drupal 3% 25%
Ember.js 50% 33%
Joomla 1% 1%
jQuery 1% 0%
Moodle 3% 3%
MooTools 50% 50%
Prototype 17% 50%
Roundcube 1% 0%
TYPO3 1% 100%
WikiMedia 4% 1%
WordPress 3% 1%
45
Avg. Precision: 2%
49 suggestions per
bug on average!
Improvements
1. Edit distance
bound
2. Ranked
suggestions

Alternative: Ranking
Subject Bug Report
46
#1
Bug Report
#2
Drupal 31 / 40 1 / 4
Ember.js 1 / 2 1 / 3
Joomla 1 / 88 1 / 88
jQuery 2 / 108 -
Moodle 2 / 37 1 / 37
MooTools 2 / 2 1 / 2
Prototype 1 / 6 1 / 2
Roundcube 4 / 79 -
TYPO3 1 / 187 1 / 1
WikiMedia 6 / 24 1 / 71
WordPress 13 / 30 1 / 170
#1 Ranking in 13 out
of 20 bugs
Conservative
ranking
Ranking seems to be
beneficial

RQ2: Performance
} Takes average of 44 seconds to find correct fix
} Worst case: 91.1 seconds (Joomla)
47

Threats to Validity
} External: Evaluated on 11 web apps
} Internal: Took bugs from earlier empirical study
48

Conclusion
} Vejovis: replacement suggestor for DOM-related faults
} Project Link: http://ece.ubc.ca/~frolino/projects/vejovis
} Evaluated on 22 real-world bugs
} Good recall – 91%
} Correct fix ranked #1 in 13/20 cases
} Average 44 s to complete
49

Vejovis: Suggesting Fixes for JavaScript Faults

More Related Content

What's hot (13)

Viewers also liked (14)

Similar to Vejovis: Suggesting Fixes for JavaScript Faults (20)

Recently uploaded (20)

Vejovis: Suggesting Fixes for JavaScript Faults