Leading Indicators for Project Management Project Headlights Dave Card David.card@dnv.com
Agenda Motivation Headlights Strategies for Leading Indicators Common Leading Indicators Back-up Lights Summary Slide 2
Motivation The outcome of a project always involves uncertainty, especially if more than one dimension of performance is considered Measurement results often viewed as snapshot in time, implications of current conditions not understood Systematic view of measurement needed to anticipate and understand project performance, enables definition of leading indicators or headlights Slide 3
Measures and indicators No measure is intrinsically a leading indicator Leading Indicator = f (measure, time, interpretation) Not leading indicators: - Customer Satisfaction - Earned Value Slide 4
Types of Indicators RR Leading Indicators Current Indicators Trailing Indicators Slide 5
Headlights (Leading Indicators) Under specific conditions, an individual measure or collection of measures may be predictive of future performance Headlights should be planned into the project can be expensive to mount as an option No generic answer as to exactly what to measure for a specific project Many common measurement practices obscure the actual situation, providing back-up lights instead Slide 6
Requirements for Leading Indicators Timely data collection and analysis Knowledge of what is important to success Measures with leading indicator properties (strategies) Interpretation and use of the measures as leading indicators Slide 7
Projects are Systems Many interacting internal and external factors Influence of any individual factor varies over time Measure factors likely to affect the performance factor of interest, not just the performance factor directly Common tendency to avoid recognition of problems as opposed to searching for potential problems Slide 8
Interactions Among Factors Functional Size Product Size Likely Causes Process Performance Likely Effects Effort Customer Satisfaction Schedule Adapted from J. McGarry, D.Card, et al., Practical Software Measurement, Addison Wesley, 2002 Product Quality Slide 9
Strategies for Leading Indicators One measure predicts future values of another measure Values of a measure predict future values of the same measure A measure tracks a basic constraint or limit to performance A measure captures risk or uncertainty Slide 10
Three Common Leading Indicators Process Compliance failure to follow the defined plan and process usually results in failure to meet budget, schedule, and quality objectives Requirements Volatility uncertainty about the project objectives usually results in delays, rework, and inadequate testing Risk Exposure project activities must reduce risk in order to reach a successful conclusion Slide 11
Measurement Influences LEADING INDICATOR LEADING INDICATOR Compliance Requirements Effort LEADING INDICATOR Schedule Risk Defects TRAILING INDICATOR Unusual results in one dimension may predict problems in others! Slide 12
Quantification of Risk and Uncertainty Risk of undesirable events Lack of information Variability in performance Slide 13
Risk Exposure 1.0 0.8 Risk Exposure 0.6 0.4 0.2 0 Time Risk Referent (Planned Exposure) Current Estimated Exposure Slide 14
Planning Uncertainty into a Project 14 On-Time Late Don t Know 12 10 Frequency (# Milestones) 8 6 Frequency 4 Project End 2 0 02/28/97 03/30/97 04/30/97 05/30/97 06/30/97 07/30/97 08/30/97 09/30/97 10/30/97 11/30/97 12/30/97 More Months Slide 15
Variation in Performance Common Causes Determine Overall Level of Performance Performance Measure Special Causes Produce Unusual Differences Unmanaged Variation = Unmanaged Risk Time or Sequence Slide 16
Process Variability Inspections of State Transition Diagrams Upper Limit Center Actuals From D.Card, Controlling the Object-Oriented Design Process, CNRC Conference on Quality Assurance of Object-Oriented Software, February 2000 Slide 17
Longitudinal Predictions Involves chains of activities (e.g., inspections) or continuing activities (e.g., requirements changes) that span the product life-cycle Values of performance factor in one activity relate to subsequent activities May be described analytically, empirically, or simulated Slide 18
Example Defect Profile Defect Profile 800 700 600 Post deliverey defects are those reported within 6 months following release of the software to the field. Defects Discovered 500 400 300 Defects per Phase Expected Defects per Phase 200 100 0 REQUIREMENTS DESIGN CODE INTEGRATION David N. Card, Managing Software Quality with Defects, COMPSAC Proceedings, August 2002 DRY_RUN FORMAL TEST POST RELEASE Slide 19
Potential Constraints Staff Availability Annual Budget Specialized Facilities SPEED LIMIT 50 Slide 20
Common Back-up Lights Cumulative measures Percentages Focus on a single factor Ambiguous and inconsistent measurement definitions Slide 21
Cumulative View Typical View of Cost Cost Performance Index 1.00 0.95 0.90 0.85 0.80 0.75 Series1 1 4 7 10 13 16 19 Month Cost Performance Index = Sum to date of Budgeted Cost of Work Performed Sum to date of Actual Cost of Work Performed Slide 22
Individual View Individuals chart with Shewhart Control Limits 1.4 1.2 Individual CPI Individuals Value 1 0.8 0.6 0.4 Changes can be detected as they occur Series1 Center = 0.97 UCL = 1.2466 LCL = 0.6934 Zone A Above Zone B Above Zone A Below Zone B Below 0.2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Month Run Slide 23
Process Performance Models All effective PPMs are leading indicators Not all leading indicators are valid PPMs Slide 24
Summary Consider the project as a system Plan Headlight measures into the project Avoid measurement practices that obscure the situation Ensure that measures are well-defined Remember leading is relative Don t forget about constraints and risks Get managers to think in terms of leading indicators Slide 25