SlideShare a Scribd company logo

Griffin: Grouping Suspicious Memory-Access Patterns to Improve Understanding of Concurrency Bugs

Download as PPTX, PDF
Sangmin Park, profile picture
Sangmin Park

Griffin is a technique that aims to improve the understanding of concurrency bugs by grouping suspicious memory access patterns from failing tests. It first performs fault localization to generate ranked lists of memory access patterns, then clusters related tests together based on similarity of patterns. Finally, it reconstructs bugs by clustering patterns based on call stack similarity and identifying suspicious methods and a bug graph.

Software
1 of 45
Download to read offline
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
Background Limitations Griffin Empirical Study Conclusion Sangmin Park, Mary Jean Harrold, Richard Vuduc Georgia Institute of Technology Griffin: Grouping Suspicious Memory-Access Patterns to Improve Understanding of Concurrency Bugs
Background Limitations Griffin Empirical Study Conclusion Difficult to Debug and Fix Time to Debug Concurrency Bugs* Hours (28%) Months (9%) Days (63%) * P. Godefroid and N. Nagappan. Concurrency at Microsoft: An exploratory survey. (EC)2, 2008. 2
Background Limitations Griffin Empirical Study Conclusion Difficult to Debug and Fix * Z. Yin et al. How do fixes become bugs? — a comprehensive characteristic study on incorrect fixes in commercial and open source operating systems, ESEC/FSE 2011 Incorrect Fix of Concurrency Bugs* Correct (61%) Incorrect (39%) 3
Background Limitations Griffin Empirical Study Conclusion Existing Techniques • Automatic fault-localization • Suspicious pair of interaction [Jin 10] • Memory-interaction list [Lucia 11] • Memory-access patterns [Park 10, Park12] • Semi-automated fix • Atomicity violation fix [Jin 11, Liu 12] • Order/atomicity violation fix [Jin 12] Limitations • Low-level memory accesses • Too much spurious information Limitation • Require developer input 4
Background Limitations Griffin Empirical Study Conclusion Overview • Background • Limitations • Our technique: Griffin • Empirical Study • Conclusion 5
Background Limitations Griffin Empirical Study Conclusion Concurrency Bugs • Order violation A pair of memory accesses  unintended program behavior • Atomicity violation Code region should be atomic but is not  unintended program behavior 6* Lu et al. Learning from Mistakes---A comprehensive study on concurrency bugs. ASPLOS 2008.
Background Limitations Griffin Empirical Study Conclusion Concurrency Bugs: Atomicity Violation * Example from Java Collection Library (Vector). b.size b.array k k a.size a.array Thread 2 b.addElement(c) k k 7 Thread 1 a.initObject(b) b.copyElements(a) a = new Data(b) + c c
Background Limitations Griffin Empirical Study Conclusion Concurrency Bugs: Atomicity Violation * Example from Java Collection Library (Vector). b.size b.array k k a.size a.array Thread 2 b.addElement(c) k k 8 Thread 1 a.initObject(b) b.copyElements(a) a = new Data(b) + c cRb.size Rb.size Wb.size Rb.array Rb.size Wb.size Wb.array
Background Limitations Griffin Empirical Study Conclusion Variables Type Memory Access Patterns Single Order R1,S(x) W2,S(x) W1,S(x) R2,S(x) W1,S(x) W2,S(x) Single- Variable Atomicity R1,S1(x) W2,S2(x) R1,S3(x) W1,S1(x) W2,S2(x) R1,S3(x) W1,S1(x) R2,S2(x) W1,S3(x) W1,S1(x) R2,S2(x) W1,S3(x) W1,S1(x) W2,S2(x) W1,S3(x) Multiple Multi- Variable Atomicity W1,S1(x) W2,S2(x) W2,S3(y) W1,S4(y) W1,S1(x) W2,S2(y) W2,S3(x) W1,S4(y) W1,S1(x) W2,S2(y) W1,S3(y) W2,S4(x) W1,S1(x) R2,S2(x) R2,S3(y) W1,S4(y) W1,S1(x) R2,S2(y) R2,S3(x) W1,S4(y) R1,S1(x) W2,S2(x) W2,S3(y) R1,S4(y) R1,S1(x) W2,S2(y) W2,S3(x) R1,S4(y) R1,S1(x) W2,S2(y) R1,S3(y) W2,S4(x) W1,S1(x) R2,S2(x) W1,S3(y) R2,S4(y) Problematic Memory-Access Patterns Patterns identified by Vaziri, Tip, Dolby. POPL 2006. 9 Fault-localization techniques Record suspicious memory-access patterns and report them in a ranked list (e.g., [Jin 10, Lucia 11, Park 10, Park 12])
Background Limitations Griffin Empirical Study Conclusion L1. Context Information main Data getSize copyElements …. Dynamic Calling Context main addElement addSize addArray …. 10* Example from Java Collection Library (Vector). Thread 2Thread 1 b.addElement(c) a.initObject(b) b.copyElements(a) a = new Data(b) initObject Thread 1 Thread 2 a.size = b.getSize() b.addSize(c.array) b.addArray(c.array) Problems Existing techniques • report only low-level memory accesses • lose context information
Background Limitations Griffin Empirical Study Conclusion Thread 2Thread 1 b.addElement(c) a.initObject(b) b.copyElements(a) a = new Data(b) L2. Multiple Bugs Rb.array Rb.size Wb.size Wb.array Sample Report … 1) 2) 3) 4) RWR – size RWWR – size/array mem-order 3 mem-order 4 Problem Existing techniques • do not handle multiple concurrency bugs 11* Example from Java Collection Library (Vector).
Background Limitations Griffin Empirical Study Conclusion L3. False-positive Patterns Sample Report … 1) 2) 3) 4) RWR – size RWWR – size/array mem-order 3 mem-order 4 12* Example from Java Collection Library (Vector). Thread 2Thread 1 b.addElement(c) a.initObject(b) b.copyElements(a) a = new Data(b) Rb.array Rb.size Wb.size Wb.array Problem Existing techniques • do not handle false-positive memory accesses
Background Limitations Griffin Empirical Study Conclusion Our Technique: Griffin 13 1. Fault Localization 2. Test Clustering 3. Bug Reconstruction Test Case Ranked Lists Clustered Lists Bug Graph Patterns Methods Bug Graph Patterns Methods Program
Background Limitations Griffin Empirical Study Conclusion Step 1: Fault Localization * Park, Vuduc, Harrold [ICST 2012] 14 Method [Unicorn, ICST 2012]: 1. Collect pairs of memory accesses in multiple tests 2. Combine pairs to patterns offline 3. Rank patterns by associating patterns with failures Program Test Case Ranked Lists Clustered Lists Bug Graph Patterns Methods Bug Graph Patterns Methods
Background Limitations Griffin Empirical Study Conclusion Step 1: Fault Localization 15 Thread 2Thread 1 b.addElement(c) a.initObject(b) b.copyElements(a) a = new Data(b) Generate ranked list of patterns for each failing test t1 RWR 271-851-681 RWWR 271-851-852-682 RW 271-851 RWR 250-353-252 t2 RWR 271-801-681 RWWR 271-801-802-682 RW 271-801 RWR 222-453-224 t3 t4 RWR 271-851-681 RWWR 271-851-852-682 RWR 250-354-253 RW 271-851 RWR 271-801-681 RWR 222-454-225 RW 271-801 RWWR 271-801-802-682
Background Limitations Griffin Empirical Study Conclusion Step 2: Test Clustering Method [Fault-localization-based clustering]: 1. Create initial clusters for each failing test with p patterns 2. Merge if similarity (Jaccard) above threshold th Until no more clusters can be merged * Jones, Bowring, Harrold [ISSTA 2007] 16 Program Test Case Ranked Lists Clustered Lists Bug Graph Patterns Methods Bug Graph Patterns Methods
Background Limitations Griffin Empirical Study Conclusion t1 t2 t3 t4 RWR 271-851-681 RWWR 271-851-852-682 RW 271-851 RWR 250-353-252 RWR 271-851-681 RWWR 271-851-852-682 RWR 250-354-253 RW 271-851 RWR 271-801-681 RWWR 271-801-802-682 RW 271-801 RWR 222-453-224 RWR 271-801-681 RWR 222-454-225 RW 271-801 RWWR 271-801-802-682 Step 2: Test Clustering Cluster by similarity of top patterns p =4 and th = 0.6 17
Background Limitations Griffin Empirical Study Conclusion RWR 271-851-681 RWWR 271-851-852-682 RW 271-851 RWR 250-353-252 t1 t2 t3 t4 RWR 271-851-681 RWWR 271-851-852-682 RWR 250-354-253 RW 271-851 RWR 271-801-681 RWWR 271-801-802-682 RW 271-801 RWR 222-453-224 RWR 271-801-681 RWR 222-454-225 RW 271-801 RWWR 271-801-802-682 Step 2: Test Clustering 3/5 or 0.6≥th Cluster by similarity of top patterns p =4 and th = 0.6 18
Background Limitations Griffin Empirical Study Conclusion t1 t2 t3 t4 RWR 271-851-681 RWWR 271-851-852-682 RW 271-851 RWR 250-353-252 RWR 271-851-681 RWWR 271-851-852-682 RWR 250-354-253 RW 271-851 RWR 271-801-681 RWWR 271-801-802-682 RW 271-801 RWR 222-453-224 RWR 271-801-681 RWR 222-454-225 RW 271-801 RWWR 271-801-802-682 Step 2: Test Clustering Cluster by similarity of top patterns p =4 and th = 0.6 19
Background Limitations Griffin Empirical Study Conclusion t1 t2 t3 t4 RWR 271-851-681 RWWR 271-851-852-682 RW 271-851 RWR 250-353-252 RWR 271-851-681 RWWR 271-851-852-682 RWR 250-354-253 RW 271-851 RWR 271-801-681 RWWR 271-801-802-682 RW 271-801 RWR 222-453-224 RWR 271-801-681 RWR 222-454-225 RW 271-801 RWWR 271-801-802-682 Step 2: Test Clustering 3/5 or 0.6≥th Cluster by similarity of top patterns p =4 and th = 0.6 20
Background Limitations Griffin Empirical Study Conclusion t1, t3 t2, t4 Two clusters of failing executions Step 2: Test Clustering Cluster by similarity of top patterns p =4 and th = 0.6 21
Background Limitations Griffin Empirical Study Conclusion Step 3: Bug Reconstruction * See the paper for detailed clustering policy 22 Method: 1. Perform call-stack-based clustering to group true/false positive patterns (Agglomerative clustering like Step 2) 2. Identify suspicious methods, bug graph Program Test Case Ranked Lists Clustered Lists Bug Graph Patterns Methods Bug Graph Patterns Methods
Background Limitations Griffin Empirical Study Conclusion Step 3: Bug Reconstruction {t1,t3} Cluster patterns based on call-stack similarity RWR 271-851-681 RWWR 271-851-852-682 RW 271-851 RWR 250-353-252 RWR 271-851-681 RWWR 271-851-852-682 RWR 250-354-253 RW 271-851 23
Background Limitations Griffin Empirical Study Conclusion Step 3: Bug Reconstruction {t1,t3} Cluster patterns based on call-stack similarity RWR 271-851-681 RWWR 271-851-852-682 RW 271-851 RWR 250-353-252 RWR 271-851-681 RWWR 271-851-852-682 RWR 250-354-253 RW 271-851 RWR 271-851-681 RWWR 271-851-852-682 RW 271-851 RWR 250-353-252 RWR 250-354-253 24
Background Limitations Griffin Empirical Study Conclusion Step 3: Bug Reconstruction Cluster patterns based on call-stack similarity RWR 271-851-681 RWWR 271-851-852-682 RW 271-851 RWR 250-353-252 RWR 250-354-253 Initial Clusters RWR 271-851-681 RWR 250-354-253 RW 271-851 RWWR 271-851-852-682 RWR 250-353-252 25
Background Limitations Griffin Empirical Study Conclusion 120 main() 150 Data (Data c) 270 int getSize() 130 void run() 850 void addAll(Data c) 120 main() 151 Data (Data b) 680 void copyArray(a) Step 3: Bug Reconstruction Cluster patterns based on call-stack similarity Initial Clusters RWR 271-851-681 RWWR 271-851-852-682 26
Background Limitations Griffin Empirical Study Conclusion 120 main() 150 Data (Data c) 270 int getSize() 130 void run() 850 void addAll(Data c) 120 main() 151 Data (Data b) 680 void copyArray(a) Step 3: Bug Reconstruction Cluster patterns based on call-stack similarity Initial Clusters RWR 271-851-681 RWWR 271-851-852-682 120 main() 150 Data (Data c) 270 int getSize() 130 void run() 850 void addAll(Data c) 120 main() 151 Data (Data b) 680 void copyArray(a) 130 void run() 850 void addAll(Data c) 27 Common call stacks are same for both clusters  merge * See the paper for detailed clustering policy
Background Limitations Griffin Empirical Study Conclusion Step 3: Bug Reconstruction Cluster patterns based on call-stack similarity Initial Clusters RWR 271-851-681 RWR 250-354-253 RW 271-851 RWWR 271-851-852-682 RWR 250-353-252 271-851 part of 271-851-681  merge 28
Background Limitations Griffin Empirical Study Conclusion Step 3: Bug Reconstruction Cluster patterns based on call-stack similarity Initial Clusters RWR 271-851-681 RW 271-851 RWWR 271-851-852-682 120 main() 150 Data (Data c) 270 int getSize() 130 void run() 850 void addAll(Data c) 120 main() 151 Data (Data b) 680 void copyArray(a) 29
Background Limitations Griffin Empirical Study Conclusion Step 3: Bug Reconstruction Cluster patterns based on call-stack similarity Initial Clusters RWR 271-851-681 RW 271-851 RWWR 271-851-852-682 120 main() 150 Data (Data c) 270 int getSize() 130 void run() 850 void addAll(Data c) 120 main() 151 Data (Data b) 680 void copyArray(a) Thread 2Thread 1 30
Background Limitations Griffin Empirical Study Conclusion Step 3: Bug Reconstruction Identify suspicious methods Initial Clusters RWR 271-851-681 RW 271-851 RWWR 271-851-852-682 120 main() 150 Data (Data c) 270 int getSize() 130 void run() 850 void addAll(Data c) 120 main() 151 Data (Data b) 680 void copyArray(a) Thread 2Thread 1 31 suspicious method: the method at the top in the common call stack.
Background Limitations Griffin Empirical Study Conclusion Step 3: Bug Reconstruction Thread 1 Thread 2 120 main() 152 Data (Data b) 680 void copyArray( a) 681 a.size = b.size; 682 a.array = b.array; 120 main() 150 Data (Data c) 270 int getSize() 271 return size; 130 void run() 850 void addAll(Data c) 851 b.size += c.size; 852 b.array += c.array; R W R R W Present bug graph to developer 32
Background Limitations Griffin Empirical Study Conclusion Empirical Studies Studies 1. Evaluate effectiveness of finding multiple faults 2. Evaluate effectiveness of explaining the bug 3. Evaluate efficiency of the technique (See paper) Empirical Setup • Implemented in Java (Soot) and C++ (Pin) • Evaluated on a set of subjects 33
Background Limitations Griffin Empirical Study Conclusion Evaluation: Subjects Language Program KLOC Num. Bugs Bug Type Java TreeSet-1 7.5 5 Atomicity TreeSet-2 7.5 3 Atomicity StringBuffer-1 1.4 4 Atomicity StringBuffer-2 1.4 1 Atomicity Vector-1 9.5 4 Atomicity Vector-2 9.5 2 Atomicity C++ Mysql-169 331 1 Atomicity Mysql-791 372 1 Atomicity NSPR-165586 125 1 Atomicity PBZip2 2 1 Order Transmission 90 1 Order 34
Background Limitations Griffin Empirical Study Conclusion Study 1: Handling Multiple Bugs Goal To investigate how well Griffin clusters failing executions responsible for the same bug Method • Ran Step 2 of algorithm; p= 30, th= 0.8 • Computed F-measure* values to evaluate effectiveness of clustering algorithm 35 * F-measure is a standard method to evaluate clustering. See “M. Steinbach et al. A comparison of document clustering techniques. In Wksp, Text Mining, 2000.“
Background Limitations Griffin Empirical Study Conclusion Study 1: Handling Multiple Bugs Program # Patterns # Bugs # Output Clusters F-measure TreeSet-1 714 5 7 0.88 TreeSet-2 656 3 4 0.91 StringBuffer-1 12 4 4 1.00 StringBuffer-2 3 1 1 1.00 Vector-1 18 4 4 1.00 Vector-2 10 2 2 1.00 Mysql-169 21834 1 1 1.00 Mysql-791 71694 1 2 0.94 NSPR-165586 1479 1 2 0.86 PBZip2 427 1 2 0.96 Transmission 226 1 1 1.00 36 • Most F-measures close to 1.00; indicates effectiveness clustering • Manual inspection when F- measures < 1.00 indicates that if th is a lesser value, clustering is more effective  may need to adjust parameters
Background Limitations Griffin Empirical Study Conclusion Study 2: Reconstructing Bug Context Goal To investigate how well Griffin reconstructs bug context Method • Ran Step 3 of algorithm • Investigated the results 37
Background Limitations Griffin Empirical Study Conclusion Study 2: Reconstructing Bug Context Program # Bugs # Output Clusters # False Positives Suspicious Method contains bug Call stack size TreeSet-1 5 5 0 Y 6 TreeSet-2 3 3 0 Y 6 StringBuffer-1 4 4 0 Y 1 StringBuffer-2 1 1 0 Y 1 Vector-1 4 4 0 Y 1 Vector-2 2 2 0 Y 1 Mysql-169 1 2 1 Y 9 Mysql-791 1 1 0 Y 1 NSPR-165586 1 1 0 Y 4 PBZip2 1 1 0 Y 0 Transmission 1 1 0 Y 7 38
Background Limitations Griffin Empirical Study Conclusion Study 2: Reconstructing Bug Context 39 Program # Bugs # Output Clusters # False Positives Suspicious Method contains bug Call stack size TreeSet-1 5 5 0 Y 6 TreeSet-2 3 3 0 Y 6 StringBuffer-1 4 4 0 Y 1 StringBuffer-2 1 1 0 Y 1 Vector-1 4 4 0 Y 1 Vector-2 2 2 0 Y 1 Mysql-169 1 2 1 Y 9 Mysql-791 1 1 0 Y 1 NSPR-165586 1 1 0 Y 4 PBZip2 1 1 0 Y 0 Transmission 1 1 0 Y 7 1 Technique successfully outputs clusters of patterns with false positives
Background Limitations Griffin Empirical Study Conclusion Study 2: Reconstructing Bug Context 40 Program # Bugs # Output Clusters # False Positives Suspicious Method contains bug Call stack size TreeSet-1 5 5 0 Y 6 TreeSet-2 3 3 0 Y 6 StringBuffer-1 4 4 0 Y 1 StringBuffer-2 1 1 0 Y 1 Vector-1 4 4 0 Y 1 Vector-2 2 2 0 Y 1 Mysql-169 1 2 1 Y 9 Mysql-791 1 1 0 Y 1 NSPR-165586 1 1 0 Y 4 PBZip2 1 1 0 Y 0 Transmission 1 1 0 Y 7 2 Technique successfully locates the bug in the suspicious method
Background Limitations Griffin Empirical Study Conclusion Study 2: Reconstructing Bug Context 41 Program # Bugs # Output Clusters # False Positives Suspicious Method contains bug Call stack size TreeSet-1 5 5 0 Y 6 TreeSet-2 3 3 0 Y 6 StringBuffer-1 4 4 0 Y 1 StringBuffer-2 1 1 0 Y 1 Vector-1 4 4 0 Y 1 Vector-2 2 2 0 Y 1 Mysql-169 1 2 1 Y 9 Mysql-791 1 1 0 Y 1 NSPR-165586 1 1 0 Y 4 PBZip2 1 1 0 Y 0 Transmission 1 1 0 Y 7 3 Call-stack sizes greater than 0 in all but one case  difficult to infer method with bug
Background Limitations Griffin Empirical Study Conclusion Future Work • Perform user studies to determine the usefulness of the technique to developers 42 • Perform more studies that involve multiple bugs • Perform studies to give more guidance to select clustering parameters
Background Limitations Griffin Empirical Study Conclusion Contributions • Fault explanation technique that provides • Information about multiple bugs • Patterns of true- and false-positive • Visualization in a Bug Graph • Empirical results that indicate the effectiveness of fault explanation • Effective in grouping concurrency bugs • Effective in explaining concurrency bugs • See www.cc.gatech.edu/~sangminp/issta2013 43 QUESTIONS?
Background Limitations Griffin Empirical Study Conclusion 44
Background Limitations Griffin Empirical Study Conclusion Challenges 45 Engineering issues… Large context size Efficient information gathering Expensive manual inspection Large number of patterns

More Related Content

PDF
Effective Fault-Localization Techniques for Concurrent Software
Sangmin Park
 
PPTX
Automated Program Repair Keynote talk
Abhik Roychoudhury
 
PPTX
STAR: Stack Trace based Automatic Crash Reproduction
Sung Kim
 
PDF
SherLog: Error Diagnosis by Connecting Clues from Run-time Logs
Dacong (Tony) Yan
 
PPTX
Binary Analysis - Luxembourg
Abhik Roychoudhury
 
PPTX
Dagstuhl2021
Abhik Roychoudhury
 
PDF
Automated Vulnerability Testing Using Machine Learning and Metaheuristic Search
Lionel Briand
 
PPTX
LSRepair: Live Search of Fix Ingredients for Automated Program Repair
Dongsun Kim
 
Effective Fault-Localization Techniques for Concurrent Software
Sangmin Park
 
Automated Program Repair Keynote talk
Abhik Roychoudhury
 
STAR: Stack Trace based Automatic Crash Reproduction
Sung Kim
 
SherLog: Error Diagnosis by Connecting Clues from Run-time Logs
Dacong (Tony) Yan
 
Binary Analysis - Luxembourg
Abhik Roychoudhury
 
Dagstuhl2021
Abhik Roychoudhury
 
Automated Vulnerability Testing Using Machine Learning and Metaheuristic Search
Lionel Briand
 
LSRepair: Live Search of Fix Ingredients for Automated Program Repair
Dongsun Kim
 

What's hot (20)

PPTX
APSEC2020 Keynote
Abhik Roychoudhury
 
PPTX
You Cannot Fix What You Cannot Find! --- An Investigation of Fault Localizati...
Dongsun Kim
 
PDF
Automated Repair of Feature Interaction Failures in Automated Driving Systems
Lionel Briand
 
PPTX
Mobilesoft 2017 Keynote
Abhik Roychoudhury
 
PPTX
Heterogeneous Defect Prediction (

ESEC/FSE 2015)
Sung Kim
 
PPT
Dissertation Defense
Sung Kim
 
PPTX
Automated Program Repair, Distinguished lecture at MPI-SWS
Abhik Roychoudhury
 
PDF
Log-Based Slicing for System-Level Test Cases
Lionel Briand
 
PDF
Impact of Tool Support in Patch Construction
Dongsun Kim
 
PPTX
Isorc18 keynote
Abhik Roychoudhury
 
PPTX
TBar: Revisiting Template-based Automated Program Repair
Dongsun Kim
 
PDF
IPA Fall Days 2019
Annibale Panichella
 
PDF
Search-driven String Constraint Solving for Vulnerability Detection
Lionel Briand
 
PPTX
NUS PhD e-open day 2020
Abhik Roychoudhury
 
PDF
Test Case Prioritization for Acceptance Testing of Cyber Physical Systems
Lionel Briand
 
PDF
A Search-based Testing Approach for XML Injection Vulnerabilities in Web Appl...
Lionel Briand
 
PPTX
A Closer Look at Real-World Patches
Dongsun Kim
 
PDF
Testing of Cyber-Physical Systems: Diversity-driven Strategies
Lionel Briand
 
PDF
Bench4BL: Reproducibility Study on the Performance of IR-Based Bug Localization
Dongsun Kim
 
PDF
Mining Fix Patterns for FindBugs Violations
Dongsun Kim
 
APSEC2020 Keynote
Abhik Roychoudhury
 
You Cannot Fix What You Cannot Find! --- An Investigation of Fault Localizati...
Dongsun Kim
 
Automated Repair of Feature Interaction Failures in Automated Driving Systems
Lionel Briand
 
Mobilesoft 2017 Keynote
Abhik Roychoudhury
 
Heterogeneous Defect Prediction (

ESEC/FSE 2015)
Sung Kim
 
Dissertation Defense
Sung Kim
 
Automated Program Repair, Distinguished lecture at MPI-SWS
Abhik Roychoudhury
 
Log-Based Slicing for System-Level Test Cases
Lionel Briand
 
Impact of Tool Support in Patch Construction
Dongsun Kim
 
Isorc18 keynote
Abhik Roychoudhury
 
TBar: Revisiting Template-based Automated Program Repair
Dongsun Kim
 
IPA Fall Days 2019
Annibale Panichella
 
Search-driven String Constraint Solving for Vulnerability Detection
Lionel Briand
 
NUS PhD e-open day 2020
Abhik Roychoudhury
 
Test Case Prioritization for Acceptance Testing of Cyber Physical Systems
Lionel Briand
 
A Search-based Testing Approach for XML Injection Vulnerabilities in Web Appl...
Lionel Briand
 
A Closer Look at Real-World Patches
Dongsun Kim
 
Testing of Cyber-Physical Systems: Diversity-driven Strategies
Lionel Briand
 
Bench4BL: Reproducibility Study on the Performance of IR-Based Bug Localization
Dongsun Kim
 
Mining Fix Patterns for FindBugs Violations
Dongsun Kim
 
Ad

Viewers also liked (14)

PPTX
CarFast: Achieving Higher Statement Coverage Faster
Sangmin Park
 
PPTX
Falcon: Fault Localization in Concurrent Programs
Sangmin Park
 
PDF
The power of datomic
Konrad Szydlo
 
PPTX
Testing Concurrent Programs to Achieve High Synchronization Coverage
Sangmin Park
 
PPTX
Hitchhiker Trees - Strangeloop 2016
David Greenberg
 
PDF
PyCon APAC 2016 Keynote
Wes McKinney
 
PDF
Apache Arrow and Python: The latest
Wes McKinney
 
PDF
Huohua: A Distributed Time Series Analysis Framework For Spark
Jen Aman
 
PDF
Python Data Ecosystem: Thoughts on Building for the Future
Wes McKinney
 
PDF
Python Data Wrangling: Preparing for the Future
Wes McKinney
 
PPTX
Raising the Tides: Open Source Analytics for Data Science
Wes McKinney
 
PDF
Improving Python and Spark (PySpark) Performance and Interoperability
Wes McKinney
 
PPT
Datomic
jperkelens
 
PDF
Mesos: The Operating System for your Datacenter
David Greenberg
 
CarFast: Achieving Higher Statement Coverage Faster
Sangmin Park
 
Falcon: Fault Localization in Concurrent Programs
Sangmin Park
 
The power of datomic
Konrad Szydlo
 
Testing Concurrent Programs to Achieve High Synchronization Coverage
Sangmin Park
 
Hitchhiker Trees - Strangeloop 2016
David Greenberg
 
PyCon APAC 2016 Keynote
Wes McKinney
 
Apache Arrow and Python: The latest
Wes McKinney
 
Huohua: A Distributed Time Series Analysis Framework For Spark
Jen Aman
 
Python Data Ecosystem: Thoughts on Building for the Future
Wes McKinney
 
Python Data Wrangling: Preparing for the Future
Wes McKinney
 
Raising the Tides: Open Source Analytics for Data Science
Wes McKinney
 
Improving Python and Spark (PySpark) Performance and Interoperability
Wes McKinney
 
Datomic
jperkelens
 
Mesos: The Operating System for your Datacenter
David Greenberg
 
Ad

Similar to Griffin: Grouping Suspicious Memory-Access Patterns to Improve Understanding of Concurrency Bugs (20)

PDF
Multimodal Residual Learning for Visual Question-Answering
NAVER D2
 
PPTX
Masters Thesis Defense Presentation
Vitor Hirota Makiyama
 
PPTX
Large-Scale Geographically Weighted Regression on Spark
Viet-Trung TRAN
 
PPTX
Second Order Heuristics in ACGP
hauschildm
 
PDF
FAST Approaches to Scalable Similarity-based Test Case Prioritization
brenoafmiranda
 
DOCX
Sample Project Report okokokokokokokokok
SamraKanwal9
 
PDF
Backpropagation - Elisa Sayrol - UPC Barcelona 2018
Universitat Politècnica de Catalunya
 
PDF
Controlled dropout: a different dropout for improving training speed on deep ...
Byung Soo Ko
 
PDF
教師なし画像特徴表現学習の動向 {Un, Self} supervised representation learning (CVPR 2018 完全読破...
cvpaper. challenge
 
PDF
Black-box Behavioral Model Inference for Autopilot Software Systems
Mohammad Jafar Mashhadi
 
PPTX
ARIMA
Venkata Reddy Konasani
 
PDF
2014-mo444-practical-assignment-04-paulo_faria
Paulo Faria
 
PDF
Introduction of Feature Hashing
Wush Wu
 
PPSX
White Box testing by Pankaj Thakur, NITTTR Chandigarh
Pankaj Thakur
 
PDF
Statistics & Decision Science for Agile - A Guided Tour
Sanjaya K Saxena
 
PDF
Using AI Planning to Automate the Performance Analysis of Simulators
Roland Ewald
 
PPTX
An optimal and progressive algorithm for skyline queries slide
WooSung Choi
 
PDF
Data driven model optimization [autosaved]
Russell Jarvis
 
PDF
Machine learning for functional connectomes
Gael Varoquaux
 
PPT
FUNCTION OF RIVAL SIMILARITY IN A COGNITIVE DATA ANALYSIS

Maxim Kazantsev
 
Multimodal Residual Learning for Visual Question-Answering
NAVER D2
 
Masters Thesis Defense Presentation
Vitor Hirota Makiyama
 
Large-Scale Geographically Weighted Regression on Spark
Viet-Trung TRAN
 
Second Order Heuristics in ACGP
hauschildm
 
FAST Approaches to Scalable Similarity-based Test Case Prioritization
brenoafmiranda
 
Sample Project Report okokokokokokokokok
SamraKanwal9
 
Backpropagation - Elisa Sayrol - UPC Barcelona 2018
Universitat Politècnica de Catalunya
 
Controlled dropout: a different dropout for improving training speed on deep ...
Byung Soo Ko
 
教師なし画像特徴表現学習の動向 {Un, Self} supervised representation learning (CVPR 2018 完全読破...
cvpaper. challenge
 
Black-box Behavioral Model Inference for Autopilot Software Systems
Mohammad Jafar Mashhadi
 
ARIMA
Venkata Reddy Konasani
 
2014-mo444-practical-assignment-04-paulo_faria
Paulo Faria
 
Introduction of Feature Hashing
Wush Wu
 
White Box testing by Pankaj Thakur, NITTTR Chandigarh
Pankaj Thakur
 
Statistics & Decision Science for Agile - A Guided Tour
Sanjaya K Saxena
 
Using AI Planning to Automate the Performance Analysis of Simulators
Roland Ewald
 
An optimal and progressive algorithm for skyline queries slide
WooSung Choi
 
Data driven model optimization [autosaved]
Russell Jarvis
 
Machine learning for functional connectomes
Gael Varoquaux
 
FUNCTION OF RIVAL SIMILARITY IN A COGNITIVE DATA ANALYSIS

Maxim Kazantsev
 

Recently uploaded (20)

PPTX
ISO 45001 Occupational Health and Safety Management System
EHA Soft Solutions
 
PDF
Digital Strategies for Manufacturing Companies
Virendra Rai, PMP
 
PDF
Adobe Illustrator 28.6 Crack My Vision of Vector Design
ghrom2211g
 
PDF
How to Choose the Right IT Partner for Your Business in Malaysia
Asian Business Services & Technologies
 
PDF
SAP S4 Hana Brochure 3 (PTS SYSTEMS AND SOLUTIONS)
pts464036
 
PDF
Softaken Excel to vCard Converter Software.pdf
markwillsonmw004
 
PPTX
Online Work Permit System for Fast Permit Processing
SHEQ Network Limited
 
PDF
Claude Code: Everyone is a 10x Developer - A Comprehensive AI-Powered CLI Tool
William Chong
 
PDF
Navsoft: AI-Powered Business Solutions & Custom Software Development
Navsoft
 
PDF
How Creative Agencies Leverage Project Management Software.pdf
Orangescrum
 
PDF
top salesforce developer skills in 2025.pdf
CloudMetic
 
PDF
Understanding Forklifts - TECH EHS Solution
TECH EHS Solution
 
PPT
Introduction Database Management System for Course Database
ReinertYosua
 
PDF
Audit Checklist Design Aligning with ISO, IATF, and Industry Standards — Omne...
rohnjim07
 
PPTX
Operating system designcfffgfgggggggvggggggggg
rmskumara09
 
PPTX
Lecture 3: Operating Systems Introduction to Computer Hardware Systems
hci7se
 
PPTX
Introduction to Artificial Intelligence
lohedi9363
 
PDF
Internet Downloader Manager (IDM) Crack 6.42 Build 41
ghrom2211g
 
PDF
Internet Downloader Manager (IDM) Crack 6.42 Build 42 Updates Latest 2025
itskinga12
 
PDF
T3DD25 TYPO3 Content Blocks - Deep Dive by André Kraus
Andre Kraus
 
ISO 45001 Occupational Health and Safety Management System
EHA Soft Solutions
 
Digital Strategies for Manufacturing Companies
Virendra Rai, PMP
 
Adobe Illustrator 28.6 Crack My Vision of Vector Design
ghrom2211g
 
How to Choose the Right IT Partner for Your Business in Malaysia
Asian Business Services & Technologies
 
SAP S4 Hana Brochure 3 (PTS SYSTEMS AND SOLUTIONS)
pts464036
 
Softaken Excel to vCard Converter Software.pdf
markwillsonmw004
 
Online Work Permit System for Fast Permit Processing
SHEQ Network Limited
 
Claude Code: Everyone is a 10x Developer - A Comprehensive AI-Powered CLI Tool
William Chong
 
Navsoft: AI-Powered Business Solutions & Custom Software Development
Navsoft
 
How Creative Agencies Leverage Project Management Software.pdf
Orangescrum
 
top salesforce developer skills in 2025.pdf
CloudMetic
 
Understanding Forklifts - TECH EHS Solution
TECH EHS Solution
 
Introduction Database Management System for Course Database
ReinertYosua
 
Audit Checklist Design Aligning with ISO, IATF, and Industry Standards — Omne...
rohnjim07
 
Operating system designcfffgfgggggggvggggggggg
rmskumara09
 
Lecture 3: Operating Systems Introduction to Computer Hardware Systems
hci7se
 
Introduction to Artificial Intelligence
lohedi9363
 
Internet Downloader Manager (IDM) Crack 6.42 Build 41
ghrom2211g
 
Internet Downloader Manager (IDM) Crack 6.42 Build 42 Updates Latest 2025
itskinga12
 
T3DD25 TYPO3 Content Blocks - Deep Dive by André Kraus
Andre Kraus
 

Griffin: Grouping Suspicious Memory-Access Patterns to Improve Understanding of Concurrency Bugs

  • 1. Background Limitations Griffin Empirical Study Conclusion Sangmin Park, Mary Jean Harrold, Richard Vuduc Georgia Institute of Technology Griffin: Grouping Suspicious Memory-Access Patterns to Improve Understanding of Concurrency Bugs
  • 2. Background Limitations Griffin Empirical Study Conclusion Difficult to Debug and Fix Time to Debug Concurrency Bugs* Hours (28%) Months (9%) Days (63%) * P. Godefroid and N. Nagappan. Concurrency at Microsoft: An exploratory survey. (EC)2, 2008. 2
  • 3. Background Limitations Griffin Empirical Study Conclusion Difficult to Debug and Fix * Z. Yin et al. How do fixes become bugs? — a comprehensive characteristic study on incorrect fixes in commercial and open source operating systems, ESEC/FSE 2011 Incorrect Fix of Concurrency Bugs* Correct (61%) Incorrect (39%) 3
  • 4. Background Limitations Griffin Empirical Study Conclusion Existing Techniques • Automatic fault-localization • Suspicious pair of interaction [Jin 10] • Memory-interaction list [Lucia 11] • Memory-access patterns [Park 10, Park12] • Semi-automated fix • Atomicity violation fix [Jin 11, Liu 12] • Order/atomicity violation fix [Jin 12] Limitations • Low-level memory accesses • Too much spurious information Limitation • Require developer input 4
  • 5. Background Limitations Griffin Empirical Study Conclusion Overview • Background • Limitations • Our technique: Griffin • Empirical Study • Conclusion 5
  • 6. Background Limitations Griffin Empirical Study Conclusion Concurrency Bugs • Order violation A pair of memory accesses  unintended program behavior • Atomicity violation Code region should be atomic but is not  unintended program behavior 6* Lu et al. Learning from Mistakes---A comprehensive study on concurrency bugs. ASPLOS 2008.
  • 7. Background Limitations Griffin Empirical Study Conclusion Concurrency Bugs: Atomicity Violation * Example from Java Collection Library (Vector). b.size b.array k k a.size a.array Thread 2 b.addElement(c) k k 7 Thread 1 a.initObject(b) b.copyElements(a) a = new Data(b) + c c
  • 8. Background Limitations Griffin Empirical Study Conclusion Concurrency Bugs: Atomicity Violation * Example from Java Collection Library (Vector). b.size b.array k k a.size a.array Thread 2 b.addElement(c) k k 8 Thread 1 a.initObject(b) b.copyElements(a) a = new Data(b) + c cRb.size Rb.size Wb.size Rb.array Rb.size Wb.size Wb.array
  • 9. Background Limitations Griffin Empirical Study Conclusion Variables Type Memory Access Patterns Single Order R1,S(x) W2,S(x) W1,S(x) R2,S(x) W1,S(x) W2,S(x) Single- Variable Atomicity R1,S1(x) W2,S2(x) R1,S3(x) W1,S1(x) W2,S2(x) R1,S3(x) W1,S1(x) R2,S2(x) W1,S3(x) W1,S1(x) R2,S2(x) W1,S3(x) W1,S1(x) W2,S2(x) W1,S3(x) Multiple Multi- Variable Atomicity W1,S1(x) W2,S2(x) W2,S3(y) W1,S4(y) W1,S1(x) W2,S2(y) W2,S3(x) W1,S4(y) W1,S1(x) W2,S2(y) W1,S3(y) W2,S4(x) W1,S1(x) R2,S2(x) R2,S3(y) W1,S4(y) W1,S1(x) R2,S2(y) R2,S3(x) W1,S4(y) R1,S1(x) W2,S2(x) W2,S3(y) R1,S4(y) R1,S1(x) W2,S2(y) W2,S3(x) R1,S4(y) R1,S1(x) W2,S2(y) R1,S3(y) W2,S4(x) W1,S1(x) R2,S2(x) W1,S3(y) R2,S4(y) Problematic Memory-Access Patterns Patterns identified by Vaziri, Tip, Dolby. POPL 2006. 9 Fault-localization techniques Record suspicious memory-access patterns and report them in a ranked list (e.g., [Jin 10, Lucia 11, Park 10, Park 12])
  • 10. Background Limitations Griffin Empirical Study Conclusion L1. Context Information main Data getSize copyElements …. Dynamic Calling Context main addElement addSize addArray …. 10* Example from Java Collection Library (Vector). Thread 2Thread 1 b.addElement(c) a.initObject(b) b.copyElements(a) a = new Data(b) initObject Thread 1 Thread 2 a.size = b.getSize() b.addSize(c.array) b.addArray(c.array) Problems Existing techniques • report only low-level memory accesses • lose context information
  • 11. Background Limitations Griffin Empirical Study Conclusion Thread 2Thread 1 b.addElement(c) a.initObject(b) b.copyElements(a) a = new Data(b) L2. Multiple Bugs Rb.array Rb.size Wb.size Wb.array Sample Report … 1) 2) 3) 4) RWR – size RWWR – size/array mem-order 3 mem-order 4 Problem Existing techniques • do not handle multiple concurrency bugs 11* Example from Java Collection Library (Vector).
  • 12. Background Limitations Griffin Empirical Study Conclusion L3. False-positive Patterns Sample Report … 1) 2) 3) 4) RWR – size RWWR – size/array mem-order 3 mem-order 4 12* Example from Java Collection Library (Vector). Thread 2Thread 1 b.addElement(c) a.initObject(b) b.copyElements(a) a = new Data(b) Rb.array Rb.size Wb.size Wb.array Problem Existing techniques • do not handle false-positive memory accesses
  • 13. Background Limitations Griffin Empirical Study Conclusion Our Technique: Griffin 13 1. Fault Localization 2. Test Clustering 3. Bug Reconstruction Test Case Ranked Lists Clustered Lists Bug Graph Patterns Methods Bug Graph Patterns Methods Program
  • 14. Background Limitations Griffin Empirical Study Conclusion Step 1: Fault Localization * Park, Vuduc, Harrold [ICST 2012] 14 Method [Unicorn, ICST 2012]: 1. Collect pairs of memory accesses in multiple tests 2. Combine pairs to patterns offline 3. Rank patterns by associating patterns with failures Program Test Case Ranked Lists Clustered Lists Bug Graph Patterns Methods Bug Graph Patterns Methods
  • 15. Background Limitations Griffin Empirical Study Conclusion Step 1: Fault Localization 15 Thread 2Thread 1 b.addElement(c) a.initObject(b) b.copyElements(a) a = new Data(b) Generate ranked list of patterns for each failing test t1 RWR 271-851-681 RWWR 271-851-852-682 RW 271-851 RWR 250-353-252 t2 RWR 271-801-681 RWWR 271-801-802-682 RW 271-801 RWR 222-453-224 t3 t4 RWR 271-851-681 RWWR 271-851-852-682 RWR 250-354-253 RW 271-851 RWR 271-801-681 RWR 222-454-225 RW 271-801 RWWR 271-801-802-682
  • 16. Background Limitations Griffin Empirical Study Conclusion Step 2: Test Clustering Method [Fault-localization-based clustering]: 1. Create initial clusters for each failing test with p patterns 2. Merge if similarity (Jaccard) above threshold th Until no more clusters can be merged * Jones, Bowring, Harrold [ISSTA 2007] 16 Program Test Case Ranked Lists Clustered Lists Bug Graph Patterns Methods Bug Graph Patterns Methods
  • 17. Background Limitations Griffin Empirical Study Conclusion t1 t2 t3 t4 RWR 271-851-681 RWWR 271-851-852-682 RW 271-851 RWR 250-353-252 RWR 271-851-681 RWWR 271-851-852-682 RWR 250-354-253 RW 271-851 RWR 271-801-681 RWWR 271-801-802-682 RW 271-801 RWR 222-453-224 RWR 271-801-681 RWR 222-454-225 RW 271-801 RWWR 271-801-802-682 Step 2: Test Clustering Cluster by similarity of top patterns p =4 and th = 0.6 17
  • 18. Background Limitations Griffin Empirical Study Conclusion RWR 271-851-681 RWWR 271-851-852-682 RW 271-851 RWR 250-353-252 t1 t2 t3 t4 RWR 271-851-681 RWWR 271-851-852-682 RWR 250-354-253 RW 271-851 RWR 271-801-681 RWWR 271-801-802-682 RW 271-801 RWR 222-453-224 RWR 271-801-681 RWR 222-454-225 RW 271-801 RWWR 271-801-802-682 Step 2: Test Clustering 3/5 or 0.6≥th Cluster by similarity of top patterns p =4 and th = 0.6 18
  • 19. Background Limitations Griffin Empirical Study Conclusion t1 t2 t3 t4 RWR 271-851-681 RWWR 271-851-852-682 RW 271-851 RWR 250-353-252 RWR 271-851-681 RWWR 271-851-852-682 RWR 250-354-253 RW 271-851 RWR 271-801-681 RWWR 271-801-802-682 RW 271-801 RWR 222-453-224 RWR 271-801-681 RWR 222-454-225 RW 271-801 RWWR 271-801-802-682 Step 2: Test Clustering Cluster by similarity of top patterns p =4 and th = 0.6 19
  • 20. Background Limitations Griffin Empirical Study Conclusion t1 t2 t3 t4 RWR 271-851-681 RWWR 271-851-852-682 RW 271-851 RWR 250-353-252 RWR 271-851-681 RWWR 271-851-852-682 RWR 250-354-253 RW 271-851 RWR 271-801-681 RWWR 271-801-802-682 RW 271-801 RWR 222-453-224 RWR 271-801-681 RWR 222-454-225 RW 271-801 RWWR 271-801-802-682 Step 2: Test Clustering 3/5 or 0.6≥th Cluster by similarity of top patterns p =4 and th = 0.6 20
  • 21. Background Limitations Griffin Empirical Study Conclusion t1, t3 t2, t4 Two clusters of failing executions Step 2: Test Clustering Cluster by similarity of top patterns p =4 and th = 0.6 21
  • 22. Background Limitations Griffin Empirical Study Conclusion Step 3: Bug Reconstruction * See the paper for detailed clustering policy 22 Method: 1. Perform call-stack-based clustering to group true/false positive patterns (Agglomerative clustering like Step 2) 2. Identify suspicious methods, bug graph Program Test Case Ranked Lists Clustered Lists Bug Graph Patterns Methods Bug Graph Patterns Methods
  • 23. Background Limitations Griffin Empirical Study Conclusion Step 3: Bug Reconstruction {t1,t3} Cluster patterns based on call-stack similarity RWR 271-851-681 RWWR 271-851-852-682 RW 271-851 RWR 250-353-252 RWR 271-851-681 RWWR 271-851-852-682 RWR 250-354-253 RW 271-851 23
  • 24. Background Limitations Griffin Empirical Study Conclusion Step 3: Bug Reconstruction {t1,t3} Cluster patterns based on call-stack similarity RWR 271-851-681 RWWR 271-851-852-682 RW 271-851 RWR 250-353-252 RWR 271-851-681 RWWR 271-851-852-682 RWR 250-354-253 RW 271-851 RWR 271-851-681 RWWR 271-851-852-682 RW 271-851 RWR 250-353-252 RWR 250-354-253 24
  • 25. Background Limitations Griffin Empirical Study Conclusion Step 3: Bug Reconstruction Cluster patterns based on call-stack similarity RWR 271-851-681 RWWR 271-851-852-682 RW 271-851 RWR 250-353-252 RWR 250-354-253 Initial Clusters RWR 271-851-681 RWR 250-354-253 RW 271-851 RWWR 271-851-852-682 RWR 250-353-252 25
  • 26. Background Limitations Griffin Empirical Study Conclusion 120 main() 150 Data (Data c) 270 int getSize() 130 void run() 850 void addAll(Data c) 120 main() 151 Data (Data b) 680 void copyArray(a) Step 3: Bug Reconstruction Cluster patterns based on call-stack similarity Initial Clusters RWR 271-851-681 RWWR 271-851-852-682 26
  • 27. Background Limitations Griffin Empirical Study Conclusion 120 main() 150 Data (Data c) 270 int getSize() 130 void run() 850 void addAll(Data c) 120 main() 151 Data (Data b) 680 void copyArray(a) Step 3: Bug Reconstruction Cluster patterns based on call-stack similarity Initial Clusters RWR 271-851-681 RWWR 271-851-852-682 120 main() 150 Data (Data c) 270 int getSize() 130 void run() 850 void addAll(Data c) 120 main() 151 Data (Data b) 680 void copyArray(a) 130 void run() 850 void addAll(Data c) 27 Common call stacks are same for both clusters  merge * See the paper for detailed clustering policy
  • 28. Background Limitations Griffin Empirical Study Conclusion Step 3: Bug Reconstruction Cluster patterns based on call-stack similarity Initial Clusters RWR 271-851-681 RWR 250-354-253 RW 271-851 RWWR 271-851-852-682 RWR 250-353-252 271-851 part of 271-851-681  merge 28
  • 29. Background Limitations Griffin Empirical Study Conclusion Step 3: Bug Reconstruction Cluster patterns based on call-stack similarity Initial Clusters RWR 271-851-681 RW 271-851 RWWR 271-851-852-682 120 main() 150 Data (Data c) 270 int getSize() 130 void run() 850 void addAll(Data c) 120 main() 151 Data (Data b) 680 void copyArray(a) 29
  • 30. Background Limitations Griffin Empirical Study Conclusion Step 3: Bug Reconstruction Cluster patterns based on call-stack similarity Initial Clusters RWR 271-851-681 RW 271-851 RWWR 271-851-852-682 120 main() 150 Data (Data c) 270 int getSize() 130 void run() 850 void addAll(Data c) 120 main() 151 Data (Data b) 680 void copyArray(a) Thread 2Thread 1 30
  • 31. Background Limitations Griffin Empirical Study Conclusion Step 3: Bug Reconstruction Identify suspicious methods Initial Clusters RWR 271-851-681 RW 271-851 RWWR 271-851-852-682 120 main() 150 Data (Data c) 270 int getSize() 130 void run() 850 void addAll(Data c) 120 main() 151 Data (Data b) 680 void copyArray(a) Thread 2Thread 1 31 suspicious method: the method at the top in the common call stack.
  • 32. Background Limitations Griffin Empirical Study Conclusion Step 3: Bug Reconstruction Thread 1 Thread 2 120 main() 152 Data (Data b) 680 void copyArray( a) 681 a.size = b.size; 682 a.array = b.array; 120 main() 150 Data (Data c) 270 int getSize() 271 return size; 130 void run() 850 void addAll(Data c) 851 b.size += c.size; 852 b.array += c.array; R W R R W Present bug graph to developer 32
  • 33. Background Limitations Griffin Empirical Study Conclusion Empirical Studies Studies 1. Evaluate effectiveness of finding multiple faults 2. Evaluate effectiveness of explaining the bug 3. Evaluate efficiency of the technique (See paper) Empirical Setup • Implemented in Java (Soot) and C++ (Pin) • Evaluated on a set of subjects 33
  • 34. Background Limitations Griffin Empirical Study Conclusion Evaluation: Subjects Language Program KLOC Num. Bugs Bug Type Java TreeSet-1 7.5 5 Atomicity TreeSet-2 7.5 3 Atomicity StringBuffer-1 1.4 4 Atomicity StringBuffer-2 1.4 1 Atomicity Vector-1 9.5 4 Atomicity Vector-2 9.5 2 Atomicity C++ Mysql-169 331 1 Atomicity Mysql-791 372 1 Atomicity NSPR-165586 125 1 Atomicity PBZip2 2 1 Order Transmission 90 1 Order 34
  • 35. Background Limitations Griffin Empirical Study Conclusion Study 1: Handling Multiple Bugs Goal To investigate how well Griffin clusters failing executions responsible for the same bug Method • Ran Step 2 of algorithm; p= 30, th= 0.8 • Computed F-measure* values to evaluate effectiveness of clustering algorithm 35 * F-measure is a standard method to evaluate clustering. See “M. Steinbach et al. A comparison of document clustering techniques. In Wksp, Text Mining, 2000.“
  • 36. Background Limitations Griffin Empirical Study Conclusion Study 1: Handling Multiple Bugs Program # Patterns # Bugs # Output Clusters F-measure TreeSet-1 714 5 7 0.88 TreeSet-2 656 3 4 0.91 StringBuffer-1 12 4 4 1.00 StringBuffer-2 3 1 1 1.00 Vector-1 18 4 4 1.00 Vector-2 10 2 2 1.00 Mysql-169 21834 1 1 1.00 Mysql-791 71694 1 2 0.94 NSPR-165586 1479 1 2 0.86 PBZip2 427 1 2 0.96 Transmission 226 1 1 1.00 36 • Most F-measures close to 1.00; indicates effectiveness clustering • Manual inspection when F- measures < 1.00 indicates that if th is a lesser value, clustering is more effective  may need to adjust parameters
  • 37. Background Limitations Griffin Empirical Study Conclusion Study 2: Reconstructing Bug Context Goal To investigate how well Griffin reconstructs bug context Method • Ran Step 3 of algorithm • Investigated the results 37
  • 38. Background Limitations Griffin Empirical Study Conclusion Study 2: Reconstructing Bug Context Program # Bugs # Output Clusters # False Positives Suspicious Method contains bug Call stack size TreeSet-1 5 5 0 Y 6 TreeSet-2 3 3 0 Y 6 StringBuffer-1 4 4 0 Y 1 StringBuffer-2 1 1 0 Y 1 Vector-1 4 4 0 Y 1 Vector-2 2 2 0 Y 1 Mysql-169 1 2 1 Y 9 Mysql-791 1 1 0 Y 1 NSPR-165586 1 1 0 Y 4 PBZip2 1 1 0 Y 0 Transmission 1 1 0 Y 7 38
  • 39. Background Limitations Griffin Empirical Study Conclusion Study 2: Reconstructing Bug Context 39 Program # Bugs # Output Clusters # False Positives Suspicious Method contains bug Call stack size TreeSet-1 5 5 0 Y 6 TreeSet-2 3 3 0 Y 6 StringBuffer-1 4 4 0 Y 1 StringBuffer-2 1 1 0 Y 1 Vector-1 4 4 0 Y 1 Vector-2 2 2 0 Y 1 Mysql-169 1 2 1 Y 9 Mysql-791 1 1 0 Y 1 NSPR-165586 1 1 0 Y 4 PBZip2 1 1 0 Y 0 Transmission 1 1 0 Y 7 1 Technique successfully outputs clusters of patterns with false positives
  • 40. Background Limitations Griffin Empirical Study Conclusion Study 2: Reconstructing Bug Context 40 Program # Bugs # Output Clusters # False Positives Suspicious Method contains bug Call stack size TreeSet-1 5 5 0 Y 6 TreeSet-2 3 3 0 Y 6 StringBuffer-1 4 4 0 Y 1 StringBuffer-2 1 1 0 Y 1 Vector-1 4 4 0 Y 1 Vector-2 2 2 0 Y 1 Mysql-169 1 2 1 Y 9 Mysql-791 1 1 0 Y 1 NSPR-165586 1 1 0 Y 4 PBZip2 1 1 0 Y 0 Transmission 1 1 0 Y 7 2 Technique successfully locates the bug in the suspicious method
  • 41. Background Limitations Griffin Empirical Study Conclusion Study 2: Reconstructing Bug Context 41 Program # Bugs # Output Clusters # False Positives Suspicious Method contains bug Call stack size TreeSet-1 5 5 0 Y 6 TreeSet-2 3 3 0 Y 6 StringBuffer-1 4 4 0 Y 1 StringBuffer-2 1 1 0 Y 1 Vector-1 4 4 0 Y 1 Vector-2 2 2 0 Y 1 Mysql-169 1 2 1 Y 9 Mysql-791 1 1 0 Y 1 NSPR-165586 1 1 0 Y 4 PBZip2 1 1 0 Y 0 Transmission 1 1 0 Y 7 3 Call-stack sizes greater than 0 in all but one case  difficult to infer method with bug
  • 42. Background Limitations Griffin Empirical Study Conclusion Future Work • Perform user studies to determine the usefulness of the technique to developers 42 • Perform more studies that involve multiple bugs • Perform studies to give more guidance to select clustering parameters
  • 43. Background Limitations Griffin Empirical Study Conclusion Contributions • Fault explanation technique that provides • Information about multiple bugs • Patterns of true- and false-positive • Visualization in a Bug Graph • Empirical results that indicate the effectiveness of fault explanation • Effective in grouping concurrency bugs • Effective in explaining concurrency bugs • See www.cc.gatech.edu/~sangminp/issta2013 43 QUESTIONS?
  • 44. Background Limitations Griffin Empirical Study Conclusion 44
  • 45. Background Limitations Griffin Empirical Study Conclusion Challenges 45 Engineering issues… Large context size Efficient information gathering Expensive manual inspection Large number of patterns

Editor's Notes

  • #5: [RV] In the top box, it should say, “Limitations”, since you list more than one. Also, when you mention the limitations, be sure to tell the audience that you will give an example later in the talk.
  • #10: [RV] When you present this slide, emphasize that a major result of *prior work* from POPL 2006 identified this table of interleaved read/write access patterns was enough to identify atomic and order violations, including those that involve multiple variables. Also say that you will momentarily show an example. Particular patterns of interleaved reads and writes on program variables -- table is enough to explain bugs…
  • #11: [RV] The “problem” here remains too abstract. I think you need to show it by using animation to *replace* the code executed by Threads 1 and 2 with the *actual* code as a developer would see it (namely, the methods corresponding to the leaves). Otherwise, someone will not be convinced that there is a real problem here, since the example as you show it is easy to understand.