![The enormous amount of code produced, the
lack of manpower and expertise, make
Security Audits difficult to perform
A software audit is
"An independent examination of a software product, software process, or set of
software processes to assess compliance with specifications, standards, contractual
agreements, or other criteria"[1]
[1] IEEE Std. 1028-1997, IEEE Standard for Software Reviews, clause 3.2](https://image.slidesharecdn.com/vccfinder-180627203255/85/VCCFinder-Finding-Potential-Vulnerabilities-in-Open-Source-Projects-to-Assist-Code-Audits-2-320.jpg)
VCCFinder: Finding Potential Vulnerabilities in Open-Source Projects to Assist Code Audits
- 1. VCCFinder FINDING POTENTIAL VULNERABILITIES IN OPEN-SOURCE PROJECTS TO ASSIST CODE AUDITS Presented by DALLA PALMA STEFANO University of Molise Software System Security Henning Perl et al. Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security. ACM, 2015
- 2. The enormous amount of code produced, the lack of manpower and expertise, make Security Audits difficult to perform A software audit is "An independent examination of a software product, software process, or set of software processes to assess compliance with specifications, standards, contractual agreements, or other criteria"[1] [1] IEEE Std. 1028-1997, IEEE Standard for Software Reviews, clause 3.2
- 3. is a code analysis tool for finding potentially dangerous code in code repositories. It combines code-metric analysis with metadata gathered from code repositories to help code review teams to prioritise their work create a classification engine to predict which commits are more likely to be vulnerable It uses a machine-learning approach to extract and combine relevant features VCCFinder
- 4. Who wrote the code and how it was commited Code repositories contain a wealth of metadata which can be highly relevant to the code quality, e.g. you can see whether a committer is new to the project or if she is one of the core contributors
- 7. METHODOLOGY
- 8. Get commit known to fix a CVE Select all CVEs containing a link to a commit of one of the 66 projects fixing a vulnerability as part of the "proof" 1
- 9. Get commit known to fix a CVE Create a crawler that searches commit messages of the 66 projects for mentions of CVE Ids 2
- 10. Heuristics to map CVEs fixing commits to VCCs (Vulnerability-Contributing Commits) Ignore changes in documentation
- 11. Heuristics to map CVEs fixing commits to VCCs (Vulnerability-Contributing Commits) Ignore changes in documentation For each deletion, blame the line that was deleted
- 12. git blame <file_name> Show what revision and author last modified each line of a file
- 14. Heuristics to map CVEs fixing commits to VCCs (Vulnerability-Contributing Commits) Ignore changes in documentation For each deletion, blame the line that was deleted For every continuous block of code inserted in the fixing commit, blame the lines before and after the block
- 15. Heuristics to map CVEs fixing commits to VCCs (Vulnerability-Contributing Commits) Ignore changes in documentation For each deletion, blame the line that was deleted For every continuous block of code inserted in the fixing commit, blame the lines before and after the block Mark the commit vulnerable that was blamed most. If two commits were blamed for the same amount of lines, blame both
- 16. Features extraction and analysis New commiters are more likely to introduce security bugs than frequent contributors Longer commits may be more suspicious than shorter ones Code that has been iterated over frequently, possibly by many different authors, is more suspicious than code that did't change often Hyphoteses
- 17. Features extraction and analysis How many commits the author has made in the project in percent Contributors # commits of the author # commits
- 18. Features extraction and analysis The number of continuous block of changes in a diff. This number assesses how fragmented the commit is. Number of Hunks Lot of changes all over the project vs One big change in one function
- 19. git diff <commit> <commit> <path> Show changes between commits, commit and working tree, etc.
- 21. Features extraction and analysis All changes made by a commit as text represented as a bag of words Patches For each patch, count the number of occurrences of each c/c++ keywords break char goto if int sizeof static struct return etc. Patches keywords
- 22. Vulnerable commits Unclassified commits Is each feature distributed indipendently or dependently from whether the commit contained a bug or not?
- 23. Null hypothesis Each feature is distributed independently from whether the commit contained a bug or not Alternative hypothesis Each feature is distributed differently in each set, i.e. vulnerable commits and unclassified commits
- 24. The effect size measures the percentage of pairs that support the hypothesis. For example, for the feature additions, the vulnerable commit contains more additions than the unclassified commits in 62% of the cases
- 25. Security bugs are not commonly introduced by code edits or refactoring New code is a more likely entry points for vulnerabilities
- 26. For keywords like if, int, struct, the vulnerable commit contains more ifs, ints, structs than the unclassified commits in around 70% of the cases
- 28. Generality Scalability Explainability Need for a classifier capable of jointly analyzing both numerical code metrics and structured metadata Need for a very efficient algorithm to analyze large code repositories and huge number of features Need for a human comprehensible explanation as to why a commit is flagged The construction of a learning-based classifier poses several challenges that need to be addressed to make the approach useful in practice
- 29. Generalized bag-of-words models Consider a generic set of token S. This set contain textual words from commit messages as well as keywords, identifiers and other tokens from the code of a commit. These tokens have been obtained by splitting the commit message and its code using spaces and newlines. Define the mapping φ from a commit to a vector space as 𝛟 ∶ 𝑿 → 𝐑|𝑺| , 𝛟: 𝒙 → 𝒃 𝒙, 𝒔 𝒔∈𝑺 where 𝑋 is the set of all commits, and 𝑥 ∈ 𝑋 an individual commit to be embedded in the vector space 𝑏 𝑥, 𝑠 = ቊ 1 𝑖𝑓 𝑡𝑜𝑘𝑒𝑛 𝑠 𝑖𝑠 𝑐𝑜𝑛𝑡𝑎𝑖𝑛𝑒𝑑 𝑖𝑛 𝑥 0 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
- 30. Example Lets consider a fictitious commit x, where a patch has been written by a user who did not contribute to a project before. The committed patch is written in C and contains a call to an API function which is associated with a buffer write operation. The corresponding vector representation of the commit x looks as follow … 𝟏 0 … 𝟏 0 … 𝜙 𝑥 → AUTHOR_CONTRIBUTION: 0.0 AUTHOR_CONTRIBUTION: 10.0 . . . buf_write_func(); some_other_func(); . . .
- 31. We can use the hyperplane vector w for explaining the decisions of our classifier By calculating the inner product between 𝝋 𝒙 and the vector w, we obtain a score which describes the distance from x to the hyperplane; that is, how likely the commit introduces a vulnerability 𝑓 𝑥 = 𝜑 𝑥 , 𝑤 = 𝑠 𝜖 𝑆 𝑤𝑠 𝑏(𝑥, 𝑠) Linear SVM
- 33. How is the dataset splitted between training data and test data? Testing set contains all commits data up from 2011 to 2014 Training set contains all commits data up until 31st of December 2010
- 34. Detection performance of VCCFinder using different feature sets Combining different features is BENEFICIAL for finding VCCs
- 35. FlawFinder is a static source code scanner that scans C/C++ source code for calls to typical vulnerable library functions Typical error types found: • Calls to library functions creating buffer overflow vulnerabilities (gets, strcpy, sprintf, ...) • Calls to library functions potentially vulnerable to string formatting attacks (sprintf, printf, ...) • Potential race conditions in file handling When given a source file, FlawFinder returns lines with suspected vulnerabilities
- 36. Comparison of the tools in three different cases: setting the same recall, same number of false positive, and same precision
- 37. 99% The percentage of VCCFinder false positive rate improvement with respect to Flawfinder 90% VCCFinder finds almost 90% of all VCCs compared to Flawfinder’s 24%
- 38. VCCFinder outperforms FlawFinder by huge margin in terms of precision and recall
- 39. CASE STUDY
- 40. CVE-2013-0862 A commit in FFmpeg introduces multiple integer overflow in the process_frame_obj function in libavcodec/sanm.c before 1.1.2 that allow remote attackers to have an unspecified impact via crafted image dimensions in LucasArts Smush video data, which triggers an out-of-bounds array access The SVM detected that the author contributed little to the project before as well as that the commit inserted a large chunk of code at once. 1
- 41. CVE-2012-2119 A commit includes a buffer overflow in the macvtap device driver in the Linux Kernel before 3.4.5, when running in certain configurations, allows privileged KVM guest users to cause a denial of service via a long description with a long vector length The SVM detected the commit because of the edited file’s high code churn, and because the author made few contributions to the Kernel 2
- 42. LIMITATIONS
- 43. Is VCCFinder able to detect VCCs on projects that have not received any CVE
- 44. How many real vulnerabilities there are in the annotated database
- 45. VCCFinder FINDING POTENTIAL VULNERABILITIES IN OPEN-SOURCE PROJECTS TO ASSIST CODE AUDITS DALLA PALMA STEFANO University of Molise Software System Security QUESTIONS?