ASQ Vancouver 25th Anniversary Quality and Business Excellence Celebration Statistical Analysis of New Product Development (NPD) Cycle-time Data Including Applications of Results Steve Pratt, MEng., PE, CSSBB Director of Engineering, Alpha Technologies
Alpha Technologies Company Background Alpha is a Full Service Power Systems Provider to Multiple Markets Pratt Slide 1
Alpha s Power Solutions Products Power Modules Indoor Power Systems Outdoor Power Systems Pratt Slide 2
Phase-Gate New Product Development (NPD) Process Deliverables: Product Concept Business Case Preliminary Plan Deliverables: High-Level Design Requirements Complete Project Plan Deliverables: Design Outputs DFx Reviews Preliminary Test Reports Deliverables: Pilot Doc Pack Pilot Build Sales Forecasts Deliverables: Compliance Certifications Training MarCom docs Deliverables: Sustaining Eng Repair/Support Cost Reduction Phase 1 Concept Phase 2 Planning Phase 3 Development Phase 4 Qualification and Pre- Production Phase 5 LA and Production Ramp-up Phase 6 GA and MOL Pratt Slide 3
Concurrent Engineering via Cross- Functional Core Teams Supply Chain Quality Service Program Manager Manufacturing Product Management Engineering Pratt Slide 4
Motivation for Study Continuous Improvement: general perception that NPD takes too long lack of proper management of resources desire to establish performance benchmarks Pratt Slide 5
NPD Data Collected and Analyzed Schedule Data Recorded Dates: Start, Gate 1, Gate 2, Gate 3, Gate 4 (LA) & Gate 5 (GA) Program Plans: estimated time to LA, estimated time to GA Calculated: total time, time per phase, actual vs. plan Cost Data Timecard System: actual total effort (man-hours) Program Plans: estimated total effort Calculated: effort by function, effort by phase, remaining effort, actual vs. plan Pratt Slide 6
Tools Used in the Study Microsoft Excel and PowerPoint JMP Statistical Discovery Software Analyze-Distribution Platform: Calculations: Quantiles, Moments Plots: Histogram, Quantile Box, Normal Quantile Fit Distribution: Normal, Beta, Goodness of Fit Tests Fit Y by X Platform: Bivariate Analysis Fit Line, Fit Polynomial, Summary of Fit Fit Y by X Platform: One-way Analysis Calculations: Means and Standard Deviations Plots: Mean Diamonds Analysis of Variance (ANOVA) Fit Model Platform Calculations: Standard Least Squares, Summary of Fit, Effect Tests Plots: Actual by Predicted, Effect Leverage, Residual by Predicted Two-way ANOVA with interactions Pratt Slide 7
External Benchmarking Data PDMA Best Practices Research Provides industry-average NPD Cycle-Time based on complexity of programs: new-to-the world new-to-the firm next-gen improvements incremental improvements Pratt Slide 8
Average NPD Cycle-Time Benchmarking Source: Griffin A., Product development cycle time for business-to-business products, Industrial Marketing Management 2002;31: 291-304 Pratt Slide 9
Planned vs. Actual NPD Cycle-Time Pratt Slide 10
ANOVA to Identify Significant Factors for Schedule Data Pratt Slide 11
ANOVA to Identify Significant Factors for Effort Data Pratt Slide 12
Categorizing NPD Programs into 9 Buckets Product Types: Modules Indoor Systems OSP (power modules, shelves, controllers) (rack-based systems, racks, distribution) (outside plant power/battery systems) Program Complexities: A major program requiring advanced development B completely new, but no advanced development C new with incremental development Pratt Slide 13
NPD Summary Data Schedule (Gate 2 to LA) Pratt Slide 14
NPD Summary Data Schedule (Gate 2 to GA) Pratt Slide 15
NPD Summary Data Effort (Total Development Hours) Pratt Slide 16
NPD Schedule & Effort Continuous Probability Distributions Pratt Slide 17
Additional Analyses plan vs. actual schedule and effort schedule and effort variance by phase time spent per phase total effort by function and phase default team size and composition Pratt Slide 18
Applications Better Estimating Pratt Slide 19
More Accurate Gate 2 Point Estimates of Schedule and Effort Previous estimates were: - subject to negotiation - overly optimistic and aggressive - rose-colored recollections of past Pratt Slide 20
Alpha s Historical Schedule Estimation Accuracy Median MRE = 44% Pratt Slide 21
NPD Schedule Point-Estimation via Historical Mean Values Median MRE = 19% Pratt Slide 22
Basic Schedule Equation Using Historical Mean Effort Values Median MRE = 21% Pratt Slide 23
Informal Comparisons Equation Using NPD Mean Values Median MRE = 20% Pratt Slide 24
Mean Magnitude of Relative Error Schedule Estimates Pratt Slide 25
Alpha s Historical Effort Estimation Accuracy Median MRE = 36% Pratt Slide 26
NPD Effort Point-Estimation via Historical Mean Values Median MRE = 18% Pratt Slide 27
Mean Magnitude of Relative Error Effort Estimates Pratt Slide 28
Determine the Cone of Uncertainty Pratt Slide 29
Cone of Uncertainty for NPD Project Estimates Pratt Slide 30
Cone of Uncertainty for NPD Project Schedule Data Pratt Slide 31
Cone of Uncertainty Applied to the 9 NPD Project Buckets Pratt Slide 32
Schedule Cone of Uncertainty Quantified by Program Phase Pratt Slide 33
Effort Cone of Uncertainty Quantified by Program Phase Pratt Slide 34
Effort Estimate Ranges by Program Phase Pratt Slide 35
Applications Top-Down Project Planning Pratt Slide 36
Creation of Top-Down Program Schedules Pratt Slide 37
NPD Time Spent per Phase Pratt Slide 38
How Much Time Should Be Spent on Planning? Pratt Slide 39
Applications Performance Goal Setting Pratt Slide 40
SMART Performance Improvement Goals An ideal performance improvement measure is: fact-based and statistically valid objective and quantifiable deterministic at the start of a program consistent across all types of programs an enabler of continuous improvement unambiguous and easy to calculate Pratt Slide 41
RSD: a Normalized Measure of Dispersion Pratt Slide 42
Program Scoring: RSD as Performance Unit-of-Measure Pratt Slide 43
2013 NPD Cycle-Time Performance Pratt Slide 44
Earned-Value Milestones to Score WIP NPD Programs Pratt Slide 45
Applications Better Project Selection Pratt Slide 46
Deterministic Business Case Analysis Figures of Merit Pratt Slide 47
Probabilistic Business Case Analysis Beta Distribution Pratt Slide 48
Monte Carlo Simulation of Cash Flows ($000) Trial #1 Trial #2 Trial #3 Trial #4 Trial #5000 NRE 562 664 530 597 560 GM$ year1 220 274 274 241 348 GM$ year2 604 653 579 407 635 GM$ year3 558 447 644 651 451 NPV $556 $461 $681 $447 $621 Pratt Slide 49
Probability-Based Figures of Merit Pratt Slide 50
Applications Resource Management Pratt Slide 51
Default Team Size and Composition Effort divided by Schedule represents number of individuals Pratt Slide 52
Roadmap and Budget Planning Pratt Slide 53
Program Resource Management Engineering Pokémon board visual resource management tool 9 program type/complexity buckets default team size/composition Pratt Slide 54
NPD Programs are Represented by a Series of Magnetic Trays Pratt Slide 55
Program Trays Incorporate Default Team Size/Composition Pratt Slide 56
NPD Team Members are Represented by a Series of Cards Pratt Slide 57
Engineering Pokémon Additional Pieces Bonus Targets Add-on Resource Trays Ghost Cards Program Status Indicators Important Date Indicators Clear Plastic Overlays Pratt Slide 58
The Board in Action Pratt Slide 59
Resulting Improvements Pratt Slide 60
Improved New Product Development Better planning and management of resources more stability and focus / less chaos increased efficiency enhanced organizational understanding faster NPD cycle-time Pratt Slide 61
In Conclusion: It is never too late to start recording data! - even simple data such as key dates and time spent on activities can facilitate powerful analyses - capture data in real-time; don t rely on memory or post mortem activities Pratt Slide 62
Questions??? Pratt Slide 63
Thank You! STEVEN PRATT, MEng., PE Director of Engineering www.alpha.ca Pratt Slide 64