Performance Measurement of Software Application Development & Maintenance

Transcription

1 2004 IFPUG Conference Performance Measurement of Software Application Development & Maintenance The David Consulting Group 1

2 Measurement Must Consider Continuous Process Improvement Strategic Positioning (Business & Technical) Deliver Value Improve Productivity Reduce Time to Market Increase Quality Satisfy Customer Improve Competitive Position Lower Costs Increase Market Share Improve Margins Shareholder Value Increase Revenues 2

3 Presentation Topics Measurement for Process Improvement Baseline your Performance Model Performance Improvements 3

4 Measurement for Process Improvement QUANTITATIVE QUALITATIVE Measures how you are doing Deliverable Size Effort/Cost Duration Quality Process Methods Skills Tools Environment Identifies what you are doing Measured Performance Capability Maturity Standard of performance Baseline of Performance 4

5 What Gets Measured? Business Related Measures Unit Delivery Cost Time To Market Customer Satisfaction Contribution Measures the impact of IT on the business Process Related Measures Effectiveness Integration Compliance Identifies trends and helps to monitor progress Project Related Measures Project Tracking Estimating Change Management Effective utilization of measures in a pro-active format 5

6 Keys to Measurement Program Measure From User Viewpoint Base Metrics on the Goals of the Process Being Measured Keep Measures Simple & Flexible 6

7 Utilize Results in Decision Making Improvements resulting from current and future initiatives must be measured The basis for measuring improvements may include: Industry data Organizational baseline data It is necessary for the organization to put a stake in the ground relative to current performance level in order to improve development practices 7

8 Presentation Topics Measurement for Process Improvement Baseline your Performance Model Performance Improvements 8

9 Baseline Activities Identify sample set (typically project oriented) Collect baseline data Project measures (e.g., effort, size, cost, duration, defects) Project attributes (e.g., skill levels, tools, process, etc.) Analyze data Performance comparisons (identification of process strengths and weaknesses) Industry averages and best practices Performance modeling (identify high impact areas) Report results 9

10 Identify Sample Set & Collect Data Research MEASURES Software Size Level of Effort Time to Market Delivered Defects Cost CHARACTERISTICS Skill Levels Automation Process Management User Involvement Environment Analysis PERFORMANCE LEVELS PROFILES Results Correlate Performance Levels to Characteristics Substantiate Impact of Characteristics Identify Best Practices 10

11 Quantitative Data Start End Effort Schedule Delivered Project Date Date FP (Mths) (Months) Cost Defects Project abc 8/18/03 1/9/ $375, Project xyz 3/15/03 12/10/ $134,640 1 Project 123 5/27/02 5/9/ $401,958 3 Project 890 8/15/03 10/19/ $85,800 0 Capstone 12/1/02 4/1/ $20,880 6 CRS ROCI Release /19/04 3/4/ $19,200 0 Project data is collected for: Start and end date duration Size expressed in function points Effort labor Defects pre and post implementation S-Tracker Upgrade 8/1/03 8/22/ $14,400 1 Low and Grow CLIP Enhancement 3/12/03 5/28/ $19,800 2 Account Status Change 9/15/03 2/9/ $95,880 0 Reversal Automation 8/27/03 1/5/ $25,560 4 NBR Cash Differentiation 8/5/03 1/5/ $53,460 1 CROI Multi-Accounts 8/1/03 9/15/ $12,000 0 Low and Grow CLIP Delay 11/13/02 1/19/ $5,400 1 Dialer Redesign 12/22/02 5/18/ $10,800 2 CROI Enhancements (Feb 03) 2/2/03 2/13/ $3,

12 Analyze Project Attributes MANAGEMENT DEFINITION DESIGN Team Dynamics Clearly Stated Requirements Formal Process High Morale Formal Process Rigorous Reviews Project Tracking Customer Involvement Design Reuse Project Planning Experience Levels Customer Involvement Automation Business Impact Experienced Development Staff Management Skills Automation BUILD TEST ENVIRONMENT Code Reviews Formal Testing Methods New Technology Source Code Tracking Test Plans Automated Process Code Reuse Development Staff Experience Adequate Training Data Administration Effective Test Tools Organizational Dynamics Computer Availability Customer Involvement Certification Experienced Staff Automation 12

13 Software Practices Profile Profile scores reflect the goodness of the development practices for a given project. Six categories are evaluated and scored. The higher the score the higher the probability of a successful delivery. Profile Project Score Mgmt Def Des Build Test Env Project Project Project Project Project Project Project

14 Strengths And Opportunities (An Example) Definition Strengths Requirements are clearly stated and stable Development and customers are experienced in applications Opportunities for Improvement More formal requirements gathering process on larger projects More consistent use of prototyping on larger projects A formal review process 14

15 Establish A Baseline Size is expressed in terms of functionality delivered to the user Software Size Performance Productivity A representative selection of projects is measured Organizational Baseline Rate of delivery is a measure of productivity Rate of Delivery Function Points per Person Month 15

16 Baseline Data Measures Performance CUSTOMER IT Product Deliverable Performance Indicators Industry Baselines B A D C D Functional Size Duration Cost Quality Average and Best in Class Performance Levels FUNCTIONALITY DELIVERED PRODUCTIVITY MEASURES COMPARATIVE ANALYSIS Business Value Operational Efficiency Ability to Compete Customer Satisfaction Delivery Rate Time to Market Cost per Unit of Work Defect Density Performance Comparison Identification of Risks Profile of Performance Best Practices Identification 16

17 Evaluating Performance Without Size Project Cost Quality (000 s) (Defects Released) PO Special $ Vendor Mods $ Pricing Adj. $ 40 5 Store Sys. $ Which project produced the best results? 17

18 Evaluating Performance With Size Project Size Cost Rate Quality Density (000 s) (Defects Released) PO Special 250 $500 $2, Vendor Mods 765 $760 $ Pricing Adj. 100 $ 80 $ Store Sys $990 $ Size --- Function Points Rate --- Cost per Function Point Density -- Defects per FP 18

19 Compare to Industry Benchmarks Industry baseline performance Software Size Rate of Delivery Function Points per Person Month 19

20 Function Points Per Person Month Average of Recent Projects Across Different Platforms Client Server 17 Main Frame 13 Web 25 e-business Web 15 Vendor Packages 18 Data Warehouse 9 20

21 Function Points Supported By One FTE Average of Support Provided for Corrective Maintenance by One FTE Client Server 642 Main Frame 978 Web 756 e-business Web 438 Vendor Packages 740 Data Warehouse

22 Track Improvements Software Size Track Progress Rate of Delivery Function Points per Person Month Year 2 grouping of projects

23 Presentation Topics Measurement for Process Improvement Baseline your Performance Model Performance Improvements 23

24 Model Improvements Model the impact of implementing selected process improvements Evaluate the impact on productivity Modeling is performed from several perspectives: Management Improvements, Design Improvements, Definition Improvements, Build Improvements, Test Improvements, Environment Improvements and SEI CMM Specific Improvements EXAMPLE: Improvements are measured from the following baseline -- Average Project Size: 133 Function Points (FPs) Average Productivity: 10.7 FP/Effort Month (EM) Average Time-to-Market: 7.3 Months Average Cost/FP: $ Projected Delivered Defects/FP:

25 Modeled Improvements Current improvement initiatives (SEI) are appropriately targeted at the majority of weak spots revealed by the baseline results. 25

26 Strive for Continuous Improvement PERFORMANCE PRODUCTIVITY COSTS DEFECTS RESOURCES TIME TO MARKET CAPABILITIES BUSINESS VALUE DELIVERABLES SKILL LEVELS TECHNOLOGY PROCESS MEASURED BASELINE Industry Averages Best Practices PERFORMANCE Sub Performance Organization Baseline RISKS SOFTWARE PROCESS IMPROVEMENT IMPROVEMENT INITIATIVES / BEST PRACTICES 26