CUT COSTS, NOT PROJECTS Understanding and Managing Software Development Costs A WEBINAR for State of Washington Agencies Critical Logic, Inc. July 9 2009 Starting at 3pm, Pacific Daylight Time
Critical Logic Highly advanced technology for integrating requirements validation with software testing, assuring ultra-high reliability software at substantially reduced cost
What This Series Is About DIS Budget 2010: 10,733,000 DIS Budget 2008: 17,690,000 When the budget is cut, IT reduces its capability The enterprise loses, IT loses How can IT cut costs while retaining capability Understand the difference Make the right decisions Improve the ROI The enterprise wins, IT wins
How This Webinar Is Organized The source of costs in software development. Costs vs Projects Cost analysis for IT. How to measure and analyze costs Attacking the major IT cost driver: rework Cost strategies. How cost analysis drives tool, staffing and project decision making
Agency vs IT Agency challenges in austere budget period: Deliver more value to the state and its citizens Determine what contributes to the costs of delivery Reduce those cost drivers, improve the margin Traditional IT reaction to revenue (budget) reductions: Reduce deliveries (eliminate or scale back projects) Reduce delivery capability (staff) Ignore cost drivers
Meet demand while reducing cost DEMAND
Reduce demand and cost together INFORMATION TECHNOLOGY PROJECTS X X Perceived value to the enterprise
The Importance Of Cost Analysis To Business 1. Break down the cost to deliver a product or service into individual elements 2. Understanding the factors which most influence those costs: the Cost Drivers. 3. Modify business processes to optimize the cost drivers to deliver products at lowest possible cost.
Cost Drivers In Business Activity Cost Pool Cost Driver Annual Count of Clients Cost Screen client application 10 minutes/ application $15/hr 350 applications per week 875 Certify applicant is qualified 1 hrs per application $24/hr 350 applications. Typical rejection rate is 50 per batch of 350 8,400 Generate the license Time to Setup $250 1 hour/batch. 250 Generate the license # licenses @ $15/license 300 licenses per batch 4,500 Total per Batch 14,025 Process oriented Quantitative Consistent
Is It Different for Government Enterprises? Mandates, revenue generation, public scrutiny are profit measures for public agencies Resource ceilings may make IT solutions the only option Delivering solutions on time and on budget remains the key management priority. It is still a business: product vs cost
Cost Drivers In Software Development Numerous attempts to quantify software cost drivers Boehm, COCOMO II Capers Jones Estimating Models All are project oriented Product: complexity, DB size, architecture, language Personnel: experience, skill level, allocation Project: tools, schedule, practices Useful for estimating, not helpful in reducing costs
Need A Different Viewpoint Every software project is unique, but Software development processes and activities are not unique Writing requirements Specifying design (interfaces, programs, databases) Constructing programs Testing Reducing the cost of a process will lower the cost of all projects using that process
Focus On Cost Of Process SW development process cost drivers Identify the processes Determine what drives the cost to perform the process Find ways to reduce that cost Typical processes Specify business requirement Designing screens or interfaces Coding software components Testing software
Cost Of Process 1. What does it cost to accomplish a typical unit of work Design a screen Code a module Test a transaction 2. Cost drivers Skills of staff Nature and complexity of applications Organizational factors 3. Focus on improving the cost of accomplishing typical processes
Process Cost Elements Every process has two required cost components Cost to perform the work Cost to validate that the results are correct Unfortunately, there is a third possible cost: doing the work again because it was done incorrectly the first time. Task (e.g coding) Work Effort QA Rework Essential Cost Avoidable Cost
What Do IT Processes Really Cost? The average programmer spends only 47 days a year coding new functionality. 80% of coding bugs are due to errors in specification of requirements. 50% or more of project costs are rework. Software development processes are much less efficient than we think. Processes are repeated because the results are incomplete or wrong.
Process And Project The project plan assumes processes are allocated like this: Requirements Solution Design Construction Testing Design Coding Requirement Design Coding Testing In reality software development processes are employed like this: Requirement Requirement Requirement Requirement Design Design Design Design Coding Coding Coding Testing Cost overruns, schedule overruns, quality problems
Why Process Costs Expand Processes get repeated because the work was incomplete the first time The process does not have clear and verifiable deliverables Wrong or inadequate tools to perform the process work Processes and their deliverables do not integrate well Understand, measure and improve each process, project costs will come down. Leverage!
Summary 1. The business objective is to reduce cost, not deliverables. IT tends to reduce deliverables to avoid cost. 2. Traditional IT efficiency measures are project oriented; not effective in managing for more deliverables at less cost 3. Cost is incurred in the processes used to build software, not the project per se 4. Projects go over budget because the processes have to be repeated or are performed inefficiently 5. Understand and manage SW development process cost drivers for better success
Quick Process Assessment Methodology We typically don t track and measure processes We do track resources in a project As a first-cut assessment of process cost drivers: 1. Determine the processes in which each team member participates 2. Track the participation of team members in project phases 3. Compare to the expected participation of team members in project phases 4. Unexpected participation indicates extended or repeated processes
Process And Project Requirements Solution Design Construction Testing GUI Designer 40 hrs Programmer 75 hrs Programmer 10 hrs SME 200 hrs Sys Analyst 100 hrs Programmer 125 hrs Tester 200 hrs Poor requirements processes? SME 70 hrs SME 30 hrs SME 30 hrs Design rework?? GUI Designer 20 hrs Sys Analyst 100 hrs GUI Designer 40 hrs Sys Analyst 100 hrs SME 200 hrs Sys Analyst 100 hrs Programmer 75 hrs Programmer 125 hrs Programmer 100 hrs Tester 200 hrs Cost overruns, schedule overruns, quality problems
Quick-analysis Template One time setup List standard processes in SW development List the actors (skills or titles) Allocate actors to processes For a project or set of projects, determine which processes take place in which project phase and the estimated hours Template allocates the hours to the actors Compare to the actual hours. Determine which actors/processes are the cost drivers
Process Cost Drivers Was the amount of work underestimated (scope or complexity)? Were the deliverables from the process incomplete or in error? Was there a black swan? Is the process itself poorly defined or too ad hoc?
Processes: Inefficient or Ineffective Inefficient processes: requires more resources but results are error-free. Process does not have to be revisited. Ineffective processes: process completes but quality of deliverable is insufficient. Process work has to be revisited. This is REWORK! Key rework indicators: Defect escapes Skill allocations Project time allocation
Easy Targets For Process Improvements Technologies that Help verify functional specifications Automatically design test cases Quickly automate test execution
DEFECT ESCAPES Other 65-70% of Defects originate in Functional Specs 85% of REWORK cost (after 1st system test) is fixing these defects Coding Design Reqmts *Capers Jones, Applied Software Measurement 2nd Ed.
Skills Allocation Requirements Solution Design Construction Testing GUI Designer 40 hrs Programmer 75 hrs Programmer 10 hrs SME 200 hrs Sys Analyst 100 hrs Programmer 125 hrs Tester 200 hrs Poor requirements processes? SME 70 hrs SME 30 hrs SME 30 hrs Design rework?? GUI Designer 20 hrs Sys Analyst 100 hrs GUI Designer 40 hrs Sys Analyst 100 hrs SME 200 hrs Sys Analyst 100 hrs Programmer 75 hrs Programmer 125 hrs Programmer 100 hrs Tester 200 hrs Cost overruns, schedule overruns, quality problems
Project Time Allocation Software Development System Test Implementation Production Fixes Rework and Re-test And support Project Start DEVELOPMENT REWORK Project End Development all effort through first run of system tests Rework all effort after first system test until project is terminated
Measuring Time Allocation Project Start Project Start Software Development Software Development System Test System Test Implementation Production Fixes Rework and Re-test And support Rework and Re-test Fixes & support Implementation Project End PQM 1.8-2.0 Project End PQM 2.1-3.0 High test coverage Project Start Software Development Early defect detection System Test Implementation Rework and Retest Fixes & support Project End PQM 3.1 5.0
Identifying Sources Of Rework Most rework results from incomplete or ambiguous definition of required system behavior Can test cases be derived from specifications? Root cause analysis of test defects PQM < 2.0 Rework is largest avoidable cost Hard dollars can be returned to new projects
Strategy For Process Improvement New tools or training are only valuable to the extent that they improve processes Primary process improvements start with reducing rework Measure the ROI in terms of rework reduction across multiple projects.
Where Can We Invest? Automating test execution Requirements management Business process analysis tools (including UML) Requirements based testing Remember, 80% of rework is due to poor specification of expected system behavior
Automated Test Execution Automated test execution tools allow existing test cases to be run using automation rather than a tester. Only impacts test execution Does not improve test case design coverage Makes testing more efficient, not necessarily more effective No impact on requirements and specifications Impact on rework ROI: Medium
Requirements Management Tools Technology repository to store specifications of requirements Provides a single repository for requirement descriptions Provides a framework for traceability and specifications (but you have to do the work) Does not enforce the quality of the functional specification itself ( Mary had a little lamb problem) Impact on rework ROI: Low - Medium
Business Process Analysis Technology and process to define business requirements Allows modeling of business requirements and decision making. Models are semi-rigorous (UML) Typically high level Generates use cases Impact on rework ROI: Medium - High
Requirements-Based Testing Technology models requirements and design, automatically produces all test documentation Allows modeling of business requirements and decision making during requirements/design work. Models are rigorous. Ambiguous and incorrect logic quickly identified All test cases and documentation generated from models Impact on rework ROI: High Defect escapes reduced to zero Test costs reduced 30% or more
Case Study: Office of Financial Management Implemented the following technologies: 1. Requirements-based testing Functional validation of specifications Automatic design of test cases 2. Keyword test automation Rapid authoring of automated scripts Full regression on each software release
Results TEMS (travel and expense management) Large reduction in initial defects Testing time cut significantly No defects when released to production 100% functional coverage with 1246 test cases Regression tests: All test cases executed in less than a week. Dataview Executive visibility (governor and staff) No defects on implementation Regressions run in hours, not days
Summary Reduced budgets do not automatically mean fewer software deliveries Assess your software development processes Identify the areas with the most rework. Typically requirements and testing Invest in technology and processes that reduce those rework costs.
What do our clients say? 17,000 hardware items, over 60,000 business rules. We went from hundreds and hundreds of defects logged to nearly zero. I m not sure what I would do without Critical Logic. Dr. John Will, Director Configuration Engineering, Sun Microsystems, Inc. Critical Logic has helped us deliver our software faster, allowing us to develop a wider range of products and services to our customers that are essentially defect free. Patti Rosenthal, Senior VP, Wells Fargo Bank Critical Logic reduced our requirements elicitation investment by 50%. Russ Nelson, VP of Product Development, Salesforce.com
Thanks Coding Business Need Requirements & Design RFTS Business Application Testing For more information on Critical Logic technologies and services www.critical-logic.com 650-343-4524