Comparing PVS-Studio with other code analyzers
- 1. Comparing PVS-Studio with other code analyzers Author: Evgeniy Ryzhkov Date: 14.10.2009 Abstract The article discusses comparison of PVS-Studio code analyzer with other solutions in this sphere. As PVS- Studio contains modules for diagnosing 64-bit and parallel errors, it is compared to various types of tools. The approach to comparison Programmers-users and developers of static code analyzers know how difficult it is to compare tools of this type. For besides different classes of diagnosed errors, it is often very difficult to formalize diagnosis of errors of even one type to "yes/no" estimate. However, such a comparison may be needed to make it possible to select from several tools a solution fit for some particular task best. In this article, we compare PVS-Studio code analyzer developed by OOO "Program Verification Systems" with similar solutions. The article is intended for those users of the tool who want to have a basis for choosing a code analyzer for their tasks. What is compared to what? The comparison is based on other articles by our authors where it is described more thoroughly what had been compared and in what way. The readers who would like to make out the comparison methods and principles may see additional materials. We will single out two types of errors: • "64-bit errors" - errors occurring when migrating from 32-bit systems to 64-bit ones or when developing new 64-bit applications; • "parallel errors" - parallel programming errors occurring when using OpenMP technology incorrectly. In the category "64-bit errors", the following four solutions participate: PVS-Studio (Viva64), PC-Lint, C++-test, /Wp64 switch of Visual C++ or diagnostics emitted by Visual C++ x64 compiler. The choice is explained by the fact that only these solutions have special support of diagnosing 64-bit errors described in the documentation. In the category "parallel errors", two solutions are compared: PVS-Studio (VivaMP) and Intel Parallel Studio (Parallel Lint). These two tools allow diagnosing parallel errors when using OpenMP technology on Windows-systems. All the tools listed perform diagnosis of errors in C/C++ code.
- 2. Comparison of the tools was performed in the following way. For each group of errors (64-bit and parallel errors) we created a base of patterns of errors described in the documentation for the tools being compared. The tools to be compared were tested on the ability to detect errors of each type. The results are given in the articles providing detailed comparisons of the tools. Proceeding from the results, we gave a total estimate for each tool. This was done in the following way. If an error had been diagnosed fully, the tool got 1 point; if partly, 0.5; if the error had not been diagnosed at all, 0 points. Then a total estimate was calculated in per cent: 100% - all the error patterns had been detected; 0% - no error patterns had been detected. Diagnosis of 64-bit errors Figure 1 - Results of comparing the tools for 64-bit error diagnosis Here is an article providing a description of the base of error patterns and a detailed description of comparison: "Comparison of analyzers' diagnostic possibilities at checking 64-bit code". Let's explain the comparison results. It is not that some code analyzer is better and some other is worse. But when diagnosing a particular type of errors ("64-bit errors") the analyzers show different results and, fortunately, we can make our choice objectively. We should specially mention the compiler switch /Wp64 in Visual Studio environment. Although this switch is not a code analysis tool unlike other solutions mentioned here, it participated in the comparison as well. Yet, its abilities of diagnosing 64-bit errors are very limited and this explains its low
- 3. total estimate. To learn more about it see the article "64 bits, Wp64, Visual Studio 2008, Viva64 and all the rest...". Diagnosis of parallel errors Figure 2 - Results of comparing the tools for OpenMP parallel error diagnosis Here is an article with a description of the base of error patterns and a detailed description of comparison:"An unsuccessful attempt to compare PVS-Studio (VivaMP) and Intel C/C++ ("Parallel Lint")". This kind of comparison is much more incorrect than comparison in the sphere of 64-bit error diagnosis. The point is that the number of error patterns diagnosed in Intel Parallel Studio is rather large. But documentation on these patterns is too poor. Ideally, the base of patterns should be enlarged with the errors diagnosed by Intel Parallel Studio but we cannot do this yet. That is why Intel Parallel Studio got few points. This situation is described more thoroughly in the article with the detailed description of comparison. Comparison results This article's point was not to try to show weak points of other developers' tools. We just tried to show those tasks at which PVS-Studio is good. There are two such tasks: • diagnosis of 64-bit errors; • diagnosis of parallel OpenMP errors. What these two tasks is concerned, our tool surpasses other similar solutions. For other tasks, it is reasonable to use more suitable tools. References 1. Andrey Karpov. Comparison of analyzers' diagnostic possibilities at checking 64-bit code. http://www.viva64.com/art-1-2-914146540.html
- 4. 2. Andrey Karpov. 64 bits, Wp64, Visual Studio 2008, Viva64 and all the rest... http://www.viva64.com/art-1-2-621693540.html 3. Andrey Karpov. An unsuccessful attempt to compare PVS-Studio (VivaMP) and Intel C/C++ ("Parallel Lint"). http://www.viva64.com/art-3-2-1186541094.html