Evaluation of Static Analyzers for Weakness in C/C++ Programs using Juliet and STONESOUP Test Suites

초록

In this paper, we compared four analyzers Clang, CppCheck, Compass, and a commercial one from a domestic startup using the NIST’s Juliet test suit and STONESOUP that is introduced recently. Tools showed detection efficacy in the order of Clang, CppCheck, the domestic one, and Compass under Juliet tests; and Clang, the domestic one, Compass, and CppCheck under STONESOUP tests. We expect it would be desirable to utilize symbolic execution for vulnerability analysis in the future. On the other hand, the results of tool evaluation also testifies that Juliet and STONESOUP as a benchmark for static analysis tools can reveal differences among tools. Finally, each analyzer has different CWEs that it can detect all given test programs. This result can be used for selection of proper tools with respect to specific CWEs.

키워드