Detecting Critical Defects on the Developer s Desktop Seth Hallem CEO Coverity, Inc. Copyright Coverity, Inc. 2006. All Rights Reserved. This publication, in whole or in part, may not be reproduced, stored in a computerized, or other retrieval system or transmitted in any form, or by any means whatsoever without the prior written permission of Coverity, Inc.
Significant Challenge: High Quality Software Code is increasingly complex The cost of Failure is high Software bugs are costly Code is increasing in size and complexity A single defect or security vulnerability can have an enormous impact on the customer Bugs delay development efforts and impact new feature development MLOC Exponential LOC growth in typical GM car 0.1 1 10 100 1970 1990 2000 2010 Source: Tony Scott CIO, GM Application-level security attacks on the rise # applicationlevel attacks Source: Gartner 80% increase 2005 2008 Developers spend significant time testing & fixing bugs Product time on projects 24% Time on canceled projects 15% Source: Caper Jones Testing, Repairs 61% 2
Software Complexity is Rising By 2010, cars will have 100 million lines of code Exponential LOC growth in typical GM car 100 10 MLOC 0.1 1 1970 1990 2000 2010 Source: Tony Scott CIO, GM 3
Rising Cost The cost of inadequate software testing is rising In the United States: The annual cost to software developers is over 22 billion dollars The annual cost to end-users is over 35 billion dollars Annual Software Testing Cost to US Economy (Millions of Dollars) 45,000 40,000 35,000 30,000 25,000 20,000 15,000 10,000 5,000 0 Development Cost End-user Cost NIST Planning Report 02-3. May, 2002. 4
The Promise of Static Analysis Tools Software Development Process Design Code Integrate QA Release Static Analysis BENEFITS Bugs Detects problems early in SDLC Security Vulnerabilities No test cases required Points to specific LOC Systematic 5
Traditional Challenges in Static Analysis Software Development Process Design Code Integration QA Release Static Analysis TRADITIONAL FAILURES Warnings False Positives High Cost Of Ownership Poor Results Hard to integrate Significant configuration & tuning Does not scale Partial code path coverage Shallow analysis Uninteresting results Rife with False Positives 6
Coverity: Breakthrough Technology Breakthrough Research At Stanford University Computer Systems Lab Analysis Depth Analysis Accuracy Scalability 100% of all code paths Interprocedural analysis 20% false positive rate Millions of lines of code 7
Coverity: Core Technologies Build C/C++ Source Code Byte Code Java Source Code C/C++ Virtual Build Java Parser Quality Prevent Checkers Security Interprocedural Dataflow Analysis Statistical Analysis Concurrency Analysis Platform False path pruning 100% of all paths Incremental analysis Extend Custom Checks Defect Manager Developer Dashboard Management Reporting Open Standard Interfaces Uses innovative source code analysis algorithms originating from compiler research Performs a whole program analysis Integrates easily into the software development process Integrated database application enables complete workflow and reporting 8
Coverity: Core Features What defects can it find? Security Vulnerabilities System and Process Crashes Infinite Loops Performance Degradations Denial of Service Privilege Escalation How does it work? Do not run the code Zero test cases Runs at compile time Data, Memory and File Corruption Unpredictable Behavior Concurrency issues 9
Coverity: Market Leader Accuracy Finds the most valuable flaws in your software Integration Minimal impact on the development process False Positives Likelihood of use Avoids reporting costly noise Built for developers to use and appreciate 10
Sample of Coverity Customers 11
Coverity History 158 Customers Number 98 53 43 35 Employees 19 7 4 4 1 1999-2003 2003 2004 2005 2006 2007 Stanford Checker Finds 2000+ bugs In Linux 1.0 release C analysis C++ analysis released DHS Vulnerability Initiative Contract Awarded Java analysis introduced 12
Customer Success: Wall Street Journal Many companies, including RIM, are teaching programmers to write safer code and test their security as software is built, not afterward. 13
Coverity Success: Wall Street Journal Many companies, including RIM, are teaching programmers to write safer code Now, Mr. Little uses Coverity every and test their security as software is built, not night afterward. to scan the WSJ code 05/04/06 turned in by engineers. The tool sends Mr. Little an email listing red flags. WSJ 05/04/06 14
Coverity Success: Quality improvement is top priority designated by executive management Complex requirements for development tools: Had to fit into the existing infrastructure Had to fit into the Capability Maturity Model (CMM) According to WindRiver s s Director of Engineering: We compared and evaluated a number of programming and error detection tools and Coverity was superior. 15
Coverity Success: Ease of integration was critical integration with Coverity Prevent is seamless and the usage is straightforward. We went from trial to purchase in 3 weeks. Coverity s impact: Immediate We found several important defects. It does validate the purchase of the tool. Ongoing Development productivity up 30% Time to market cut by 20% 16