An Execution-Semantic and Content-and-Context-Based Code-Clone Detection and Analysis
1 like926 views
The document details a method for code-clone detection and analysis that utilizes execution-semantic and content-and-context characteristics. The proposed approach involves analyzing execution traces to identify inter-procedural clones, with specific algorithms for extracting and filtering content. Preliminary experiments demonstrated the method's application on functional and dynamically typed programming languages, revealing its sensitivities and areas for refinement.
An Execution-Semantic and Content-and-Context-Based Code-Clone Detection and Analysis
- 1. An {Execution-Semantic, Content-and-Context}-Based Code-Clone {Detection,Analysis} Toshihiro Kamiya Future University Hakodate kamiya@fun.ac.jp Toshihiro Kamiya: An Execution-Semantic and Content-and-Context-Based Code-Clone Detection and Analysis, Proceedings of the 9th IEEE International Workshop on Software Clones (IWSC'15), pp. 1-7 (2015).
- 2. TOC ● Problem/Motivation ● Outline of proposed method ● Example ● Algorithm of clone detection ● Visualization ● Implementation ● Preliminary experiment
- 3. The problems / Motivation ● In functional PLs, developers can define their own control structure. – Analyzing only pre-defined control statements is no longer sufficient to represent code pattern. – E.g., if (C) A; else B; ⇔ myIf(C, lambdaA, lambdaB); → inter-procedural analysis ● Dynamic dispatching makes inter-procedural analysis difficult. – Esp. in functional + OO + dynamically typed PLs (no explicit type declaration → hard to analyze dispatches in a static way)
- 4. Idea Detect clones from an execution trace ! ● Dispatches and control structures have been expanded (resolved). ● Detected clones are inter-procedural, type 3 clones.
- 5. Outline of proposed method ● Execution trace → Call tree → Contents and Context (for each node) ● main() os.listdir() print_extensions _w_for_stmt() print_extensions _w_map_func() os.path. splitext() print str.join()get_extensions() print map() lambda() at line 8 os.path. splitext() contents context Clone detection Clone analysis Contents Context
- 6. Example code
- 7. These two functions are... A helper function
- 8. ...a semantic clone. The same functionality: finds extensions of given files and prints them out
- 9. Shared items
- 10. and differences Distinct loops. for vs map All shared items are contained in a function. Shared items are spread into functions.
- 11. Detection steps Input: a call tree (← execution trace ← target program) 1. Extracts contents and context of each node 2. Identifies sets of contents-sharing nodes 3. Removes redundant nodes (filtering with contexts)
- 12. Input … call __main__//<module> runpy//_run_code 69 : load_const __main__//<module> 0 load_const __main__//<module> 12 load_const __main__//<module> 21 load_const __main__//<module> 30 load_const __main__//<module> 39 call __main__//main __main__//<module> 63 : call __main__//print_extensions_w_for_stmt __main__//main 24 : <list> call posixpath//splitext __main__//print_extensions_w_for_stmt 25 : 'about.txt' call genericpath//_splitext posixpath//splitext 18 : 'about.txt' '/' None '.' load_const genericpath//_splitext 0 return genericpath//_splitext 139 : * 'about' '.txt' return posixpath//splitext 21 : * 'about' '.txt' call pygoat.hook/Out/write __main__//print_extensions_w_for_stmt 32 : '.txt' return pygoat.hook/Out/write 15 call pygoat.hook/Out/write __main__//print_extensions_w_for_stmt 33 : 'n' return pygoat.hook/Out/write 15 call posixpath//splitext __main__//print_extensions_w_for_stmt 25 : 'pygoat.data' call genericpath//_splitext posixpath//splitext 18 : 'pygoat.data' '/' None '.' load_const genericpath//_splitext 0 return genericpath//_splitext 139 : * 'pygoat' '.data' return posixpath//splitext 21 : * 'pygoat' '.data' call pygoat.hook/Out/write __main__//print_extensions_w_for_stmt 32 : '.data' return pygoat.hook/Out/write 15 call pygoat.hook/Out/write __main__//print_extensions_w_for_stmt 33 : 'n' return pygoat.hook/Out/write 15 call posixpath//splitext __main__//print_extensions_w_for_stmt 25 : 'greeting.md' call genericpath//_splitext posixpath//splitext 18 : 'greeting.md' '/' None '.' load_const genericpath//_splitext 0 return genericpath//_splitext 139 : * 'greeting' '.md' return posixpath//splitext 21 : * 'greeting' '.md' call pygoat.hook/Out/write __main__//print_extensions_w_for_stmt 32 : '.md' return pygoat.hook/Out/write 15 call pygoat.hook/Out/write __main__//print_extensions_w_for_stmt 33 Program Execution trace main() os.listdir() print_extensions _w_for_stmt() print_extensions _w_map_func() os.path. splitext() print str.join()get_extensions() print map() lambda() at line 8 os.path. splitext() Call tree Input: a call tree (← execution trace ← target program) 1. Extracts contents and context of each node 2. Identifies sets of contents-sharing nodes 3. Removes redundant nodes (filtering with contexts)
- 13. Step 1. 1. Extracts contents and context of each node main() os.listdir() print_extensions _w_for_stmt() print_extensions _w_map_func() os.path. splitext() print str.join()get_extensions() print map() lambda() at line 8 os.path. splitext() main() get_extensions(), map(), lambda() at line 8, os.listdir(), os.path.split(), print, print_extensions_w_for_stmt(), print_extensions_w_map_func(), str.join() print_extensions_w_for_stmt() main() os.path.split() print print_extensions_w_map_func() main() get_extensions(), map(), lambda() at line 8, os.path.split(), print, str.join() Input: a call tree (← execution trace ← target program) 1. Extracts contents and context of each node 2. Identifies sets of contents-sharing nodes 3. Removes redundant nodes (filtering with contexts)
- 14. Step 2. 2. Identifies sets of contents-sharing nodes main() get_extensions(), map(), lambda() at line 8, os.listdir(), os.path.split(), print, print_extensions_w_for_stmt(), print_extensions_w_map_func(), str.join() print_extensions_w_for_stmt() main() os.path.split() print print_extensions_w_map_func() main() get_extensions(), map(), lambda() at line 8, os.path.split(), print, str.join() Input: a call tree (← execution trace ← target program) 1. Extracts contents and context of each node 2. Identifies sets of contents-sharing nodes 3. Removes redundant nodes (filtering with contexts)
- 15. Step 3. 3. Removes redundant nodes (filtering with contexts) main() get_extensions(), map(), lambda() at line 8, os.listdir(), os.path.split(), print, print_extensions_w_for_stmt(), print_extensions_w_map_func(), str.join() print_extensions_w_for_stmt() main() os.path.split() print print_extensions_w_map_func() main() get_extensions(), map(), lambda() at line 8, os.path.split(), print, str.join() Included by all of other nodes in the set ⇒ redundant Input: a call tree (← execution trace ← target program) 1. Extracts contents and context of each node 2. Identifies sets of contents-sharing nodes 3. Removes redundant nodes (filtering with contexts)
- 16. Detection result A clone class: { print_extensions_w_map_func(), print_extensions_w_for_stmt() } Shared items: { os.path.split(), print } print_extensions_w_for_stmt() main() os.path.split() print print_extensions_w_map_func() main() get_extensions(), map(), lambda() at line 8, os.path.split(), print, str.join()
- 17. Detection result A clone class: { print_extensions_w_map_func(), print_extensions_w_for_stmt() } Shared items: { os.path.split(), print } dagified (merged) by label (DAG = directed acyclic graph) Context Contents main() print_extensions _w_for_stmt() print_extensions _w_map_func() get_extensions()print map() lambda() at line 8 os.path. splitext()
- 18. Content-and-context analysis for triaging ● Clone class (a), shared items (b), distinct contents (or gap) (c) ● The distinct contents (c) shared the same set of (sub-)contents (d) → (c) is another clone class. ● If (c) is merged before (a), (c) will not be a gap of (a) anymore. (a) (b) (c) (d) Detected from markdown2's code (described later)
- 19. Tool prototype Target program Inputs / Test cases Execution (Python interpreter) Execution trace Debugging / profiling APIs Execution trace extraction String balloon generation String balloons Frequent item-set mining (Apriori) Similar sets of contents Redundant context removal Code clones Step 1 Step 2 Step 3 Detection Visualization Metrics calculation Analysis ● Input: Python source code ● Uses a frequent item-set mining algorithm / implementation – Apriori (www.borgelt.net/apriori.html) ● Heuristics / optimizations – Max. depth of contents from a target node (default 5) – Max. number of content items of a candidate node (default 25) ● Filters out the nodes with large contents, i.e., nodes near to the root of call tree – Removal of basic, primitive functions – ... Content-and-context clone on call graph
- 20. Preliminary experiment for each of the parameter(“Max. number of content items of a candidate node”) values: 10, 15, …, 30. Target product Collection of exe. seq. # function calls # unique labels markdown2 Running 144 unit tests 227K 1128 wxPython Invoking a sample program “pySketch” 483K 1058
- 21. Results
- 22. Results Exponential to number of contents Too “peaky” for practical use
- 23. Summary ● A code-clone detection from a dynamic info, execution trace – Aiming to apply functional/dynamically typed PLs ● Context-and-content analysis for triage ● Algorithm, implementation, heuristics ● Preliminary experiment – Targets: markdown2 and wxPython – Peaky, sensitive to a parameter Max. number of content items of a candidate node → Needs refinements Omitted, refer the paper: ● Threats to validity ● Future plan (a) (b) (c) (d)