How Virtual Compilation Transforms Code Analysis

Transcription

1 How Virtual Compilation Transforms Code Analysis 2009 Checkmarx. All intellectual property rights in this publication are owned by Checkmarx Ltd. and are protected by United States copyright laws, other applicable copyright laws and international treaty provisions. Checkmarx Ltd. retains all rights not expressly granted. For further information, contact Checkmarx or your local distributor or reseller.

2 2 Executive Summary Secure software development has become a priority for all organizations whether they build their own software or outsource. And code analysis is becoming the de facto choice to introduce secure development as well as measure inherent software risk. Many assume that code analysis requires code compilation as a prerequisite. Today, all major static code analyzers are built on this assumption and only scan post compilation requiring buildable code. The reliance on compilation has major and negative implications for all stake holders: developers, auditors, CISOs, as well as the organizations that hope to build a secure development lifecycle (SDLC). Historically, static code analysis required a complete and buildable project to run against, which made the logical place to do the analysis at the build server and in-line with the entire build process. The buildable requirement also forced the execution of the scan nearer the end of the development process, making security repairs to code more expensive and greatly reducing any benefits. There is evidence that compilation-based code analysis tools negatively impact risk mitigation efforts. As Gartner analyst Neil MacDonald observed, we ve talked with a number of clients that purchased a [static analysis] tool which later becomes expensive shelfware or where the project was halted after delivering mixed results. 1 Mr. MacDonald correctly singles out poor security process as an obstacle but there are serious technical factors that contribute to the shelfware problem. A key, overlooked bottleneck comes from the compiler based approach. Getting the code into a state where it can be compiled and linked is not an easy task. How does the need for compilation negatively impact the stakeholders who rely on code analysis? Developers: With compiler-based approaches, vulnerability scanning is limited to unit testing or even later weeks after the code is written. This necessitates iterative coding turnarounds that are inefficient and not effective at bringing a culture of building in application security. Auditors: Forced to rely on testing tools dependent on compilation, auditors lose the flexibility to make spot checks on suspect code early in the development process. Problems of duplicating the developer s environment and code integration further challenge the auditor s efficiency. These difficulties can result in vulnerabilities not being revealed in the test environment or just missed due to the inefficiencies of the process itself. CISOs: CISOs, who bear the responsibility of mitigating risks in the enterprise, often face developer resistance when introducing security code analyzers and frequently are challenged to receive risk assessments from auditors on a timely basis. 1

3 3 Checkmarx s Virtual Compiler eliminates these problems by removing the dependency on compilation and linking for software testing. It transforms code, whether freshly written or old legacy applications, into a form that contains structure and application flow properties. Testing is not dependent on having all modules complete, duplicating the development environment or creating a final build-test harness. Instead, scanning can take place early, and often, as the code is developed. Once scanning is complete, all code and flow properties are stored in a data base that can be interrogated for vulnerabilities. Inspecting applications can be completed without lengthy setups and configurations since virtual compilation is compiler and operating system independent. How does the Virtual Compiler benefit the key stakeholders in the software development process? Developers: The Virtual Compiler enables developers to test code anywhere, anytime, while avoiding problems of compiler and operating system compatibility. Developers can test uncompiled and unlinked code, their independent modules or any other application subsets in a true developer desktop deployment that reinforces good security awareness and practices as the code is written. Auditors: Auditors can test code earlier in the SDLC. Further, auditors can easily conduct spot checks without worrying about duplicating development environments. CISOs: CISOs will be able to monitor and reinforce secure coding practices as the code is written, giving them a better understanding of potential exposure to vulnerabilities earlier in the SDLC. Finally, and most importantly, the Virtual Compiler is not only accepted but welcomed by developers, auditors and CISOs avoiding a common obstacle in building an efficient and effective SDLC ensuring that applications get tested thoroughly and effectively, thus saving time and costs.

4 4 The Need In order to scan compiled code, the code has to successfully compile, without syntax errors or linkage issues. In complex applications, achieving a full build often requires long efforts and coordination between multiple stake holders. Often, such holistic builds take place in later stages of project development as system testing looms. The problems with compiled code are not finished once the code successfully compiles. The binaries from code compiled in compiler A and operating system B differ from code compiled with compiler X for operating system Y. To support this large number of combinations, code analyzers must adapt to all possible permutations of compilers and operating systems. This presents even bigger problems when modern, agile and iterative techniques are used that require testing to be done inline. These techniques assume that whatever gets checked into the build system is solid, secure and plays well with all the other code in the build presupposing the use of static security analysis by all developers and testers involved in the project. To avoid the dependencies, problems and complexities of numerous compiler and operating systems, and enable the scanning of incomplete code that otherwise could not be compiled; Checkmarx eliminated the need for compilation and invented the Virtual Compiler. The Virtual Compiler: What is it? The Virtual Compiler reads any source code and transforms it to a common language form that can then be scanned thoroughly for vulnerabilities. It can take non-compiled code or any project subset and virtually compiles it by compensating for syntactical errors and stubbing the missing linking parts. Moreover, it is based on published standards that define the exact context and behavior of a computer language. Using this approach it can use the source code itself for analysis and bypasses formal compilation and linkage procedures making it compiler and platform agnostic and avoids any compatibility issues. It enables easy correlation to the code for remediation as well as easy additions of languages and dialects creating a true language agnostic platform. The Virtual Compiler takes the concept of the Java Virtual Machine innovation a step further. Whereas in Java the language is agnostic to operational environment considerations, the Virtual Compiler is agnostic to the language intricacies altogether. It treats all languages and dialects alike bringing them to a common language form. The commonality that was once achieved at the binary level has been successfully transformed to the source level. Significantly, all sources do not have to be alike or even complete: the Virtual Compiler forgives the developer on compile and linkage errors. Furthermore, the code is enhanced to bypass pitfalls presented by standard compilers improving analysis accuracy. While scanning incomplete code early does not find all problems that could show up later, it has been proven to find a significant portion, and has the advantage of being used in the early stages of the development cycle where efficiencies are best achieved.

5 5 How does it work? The Virtual Compiler takes any source code and transforms it to a unified form that can then be scanned for vulnerabilities. Following is a diagram of the Virtual Compiler: The Virtual Compiler works in the following steps: 1. Language Adaptor This first step analyzes the source code based on published standards used by all the compilers in the market. 2. Syntax Compensator Checkmarx then identifies syntactical errors and isolates the nearby unresolved portion of the program while enabling the complete portions to proceed. 3. Linkage Resolver Checkmarx identifies missing and unresolved links and stubs the missing links enabling the detection throughout the resolved flow. 4. Code Enhancer Compilation is proprietary and optimized for runtime thus creating during the resolving process pitfalls avoided by Checkmarx code enhancer: o Add missing Control Flow elements

6 6 o Distinct between ambiguous data elements o Avoid misrepresentations created by Compiler optimization o Resolve Run-time virtual function calls 5. Common Language Form The language is virtualized into a common form containing structure and data flow properties. 6. Exhaustive flow scanner Finally, scan complexity and accuracy are correlated to the depth of application graphs. Checkmarx s patent pending algorithm, implemented by an Exhaustive Flow Scanner, enables the scanning for flaws of all paths within a flow graph, avoiding shortcuts taken by other code analyzers. Consequently, the EFS approach has the added benefit of pinpoint accuracy. The Checkmarx code analyzer is the only product today with virtually zero false positives. Whereas other products can feel like shock therapy, due to long configurations and high false positives, Checkmarx users experience faster time to adoption and a low usage overhead. Once scanning the source is complete, all code and flow properties are stored in a data base and can be interrogated by an open query language for vulnerabilities. The out-of-the-box queries coupled with customization for corporate standards and business logic ensure full detection throughout the vulnerability spectrum. The impact of Virtual Compilation The key risks with compiler-based approaches are: first, builds may fail often and key security vulnerabilities are not reported and second, static analysis will not be deployed at the desktop early in development. The Virtual Compiler gives proper solution to all stake holders who impact security during the development process. Developers: The ability to scan unbuilt code pushes static analysis even further back into the development life cycle when it is most useful. The biggest impact of virtual compilation is desktop usability by developers and auditors. The promise of code analysis was the reduction of errors at the cheapest phase of development. In addition due to code enhancements and Exhaustive Flow Scanning the user gets much more accurate results. Auditors: Virtual compilation means auditors are ready to conduct an inspection at any time on any code base. Auditors should have the ability to quickly get into code level reviews but then also review high level trends. Without a dependency upon compiler-based approaches, auditors are not hampered by issues of compiler or platform compatibility all they need is the source. And faster audits means reviewing more code in less time. Finally, auditors rarely have access to the code for a complete project which is not an obstacle with virtual compilation.

7 7 CISOs: Static analyzers raise major concerns around developer adoption and productivity. The Virtual Compiler means code analysis will be used more broadly to give CISOs a faster and accurate view of their "risk factory" and enable them to put in place effective controls to contain it. Conclusion: the Virtual Compiler delivers a solid ROI with a significantly reduced TCO Virtual compilation provides the best way for organizations to introduce secure development while systematically eliminating software risk. Virtual compilation streamlines the workflow of key stakeholders in the software development process, increasing their effectiveness in finding problems and reducing the need for costly professional services. The Virtual Compiler enables developers and auditors to scan code anywhere, anytime. For CISOs, it means that securing applications in the enterprise is finally practical and achievable. Checkmarx Virtual Compiler delivers: Strong ROI: The Virtual Compiler enables problems to be discovered earlier in the SDLC with improved accuracy compared to solutions deployed later during formal testing reducing the cost to find and fix defects. Low TCO that facilitates quick, frequent code scanning: The Virtual Compiler is platform independent, enabling quick setup in any environment all you need is source code. It does not matter if the developer uses Linux, Windows, Apple or Solaris as the operating system. The complexities and system overhead of compiling and building applications are avoided. The faster and more convenient the testing methodology, the more likely that it will be used often and thoroughly, ensuring that your code will be solid and secure. Contact Us For more information about Checkmarx, or any of our products, please contact us or visit our Web site at. For immediate information, contact our staff at: info@checkmarx.com