Life Cycle Risk Management in a Product Tailoring Environment. David R. Pinkley

Transcription

1 Life Cycle Risk Management in a Product Tailoring Environment David R. Pinkley

2 Life Cycle Risk Management Agenda Life Cycle Risk Management Product Tailoring Impact Architecture Optimization Heritage Reuse Risk Tracking Integration Readiness Anomaly Risk Rating Cumulative Risk Management Normalcy Bias: Lack of exposure to failure and small sample size of operating hours: Rejection of proposed failure modes Seizing on any ambiguities to infer less credibility Interpretation of warnings in the most optimistic way Bounded Rationality: Decision-making, rationality of individuals is limited: By information Cognitive state Finite decision times (Herbert A. Simon) Epistemic Failures due to erroneous technological assumptions, even thought there were good reasons to hold that assumption. (John Downer) Page 2

3 Lifecycle Risk Management Components Pre-Award!RFP!Selection Reqmts - Design!SRR!PDR!CDR Build and Test!MRR! TRR!PER Sell-Off!PSR!MRR Operations!FRR!ICR Architectural Optimization " Development " Implementation " Risk Profile/Strategy Heritage Reuse Compliance " Resource " Schedule " Technical Establish Baseline and Assess Compliance Residual Risk and Uncertainty Management Integration Readiness and Anomaly Rating Cumulative Risk Analysis Risk ID, Tracking, Control " Baseline " Realized " "Unknowns Bounding Integration Readiness " Next Step Integration " Product Liens " Constraints Cumulative Risk Tracking " Residual Risk " In-Phase Burn Down " Cumulative MS Impact " Process Effectiveness Anomaly Risk Rating " Severity " Cause " Corrective Action Page 3

4 Establishing A Compliant Baseline

5 Product Type Investment Product type total risk exposure Process and Product Architectures Risk Exposure Type 1: Large risk exposure Factors: Longest Mission Length, Custom Development Overlapping Mitigation Actions Type 2: Moderate risk exposure Factors: Long Mission Length, Heritage Development Part Quality/Margins/Redundancy Type 3: Low risk exposure Factors: Short Mission, Heritage Development Part Quality/Test Actions Type 4: Minimum risk exposure Factors: Non-flight prototype development Safety and Interface Focus Type 1 Mitigated - No SPFs - Large Margin - MA & MS - Full V&V - L1/L2 parts - Supplier OS - TRL assured - MRL assured Residual Minimal Type 2 Mitigated - Minor SPFs - Margin - MA - V&V - L2/L2- parts - Supplier risk - Heritage - Internal MRL Residual Internal Core BPS STDs Type 3 Mitigated - Part Quality - Test Margins - MA Audits - Supplier Risk - Heritage - Internal MRL Residual - Single string - Streamlined Process Type 4 Mitigated - Interfaces - Safety Residual Engineering Prototype Development Mission Success Assurance Activities Cost Probability of Success T1 T2 Flight Product Types T3 T4 IR&D Mission Success Product Type Risk Management has Demonstrated Equivalent Mission Success Page 5

6 Product Type Instantiation Optimizes Development Architecture Critical evaluation Programmatic Constraints: Acquisition, Requirements, Funding Mission Needs: Reliability, Life, Performance Acquisition Risk Strategy: Mission Class, Mission Type Risk Management Radical Spokes Risk Management Supplier Interaction EEE Parts Complexity Mission Class Risk Surface Legend Vector Characteristic People: Risk Process Informal, Sparse documentation Programmatic: Risk Process cost/schedule/technical focus Mission Success: Programmatic Plus Residual Risk Management COTS: Buying product as catalog item with little data Insight: Formal Data available throughout development Oversight: Customer/Supplier interactive development COTS: Consumer commercial and Industrial Grade L3: Hi-Rel parts with screening but little qualification data L2: Military grade parts will full screening and Qual program L1: Space grade parts with delta screening and qualification Single String: Simplex assemblies performing mission Block Redundant: Parallel active and standby assemblies No SPFs: No simplex assemblies beyond DFMR items People Programmatic Mission Success Minimum Practical Risk Oversight Insight Supplier Interaction COTS L1 L2 L3 COTS No SPFs Block Redundant Single String EEE Parts Complexity Optimal Development Architecture Translates Into Managed Risk and Controlled Cost Page 6

7 Product Type Instantiation Optimizes Mission Success Processes Process Application Level: Inductive/Deductive Analysis Rigor: Patent Defect Inspections, WCCA Circuits Oversight: Independent Internal and Customer Assessments Relationships: Test level, Build-in vs. Procedural Category Program Execution Risk, Oversight and Assurance Triage, Information & Lessons Learned Process 1 Design Assurance 2 Requirement Analysis and Validation 3 Parts, Materials and Processes 4 Environmental Compatibility 5 Reliability Engineering 6 System Safety 7 Configuration/Change Management 8 Integration, Test and Evaluation 9 Risk Assessment and Management 10 Independent Reviews 11 Hardware Quality Assurance 12 Software Assurance 13 Supplier Quality Assurance 14 Failure Review Board 15 Corrective/Preventative Action Board 16 Alerts, Information Bulletins Optimal Mission Success Processes Avoid, Eliminate, Precipitate, & Mitigates Risks Page 7

8 Early and Continual Assessment of Hardware-Software Reuse Key to Fulfilling Heritage Promise Assessment of TRUE Heritage Key to Managing Capability Based Risks Page 8

9 Residual Risk and Uncertainty Management

10 Residual Risk Augmented Classical Risk Management Augmented Risk Management o ID, Analysis, Plan, Track, Control o Acquisition, Baseline Constraints, Execution Systems Engineering o Risk Interfaces and Relationships o Product Type Risk Exposure Consequence Subcontract Management o Capture Subtle Requirement and Product Changes Peer, Life-Cycle Gate, & Milestones o Build Upon the Cumulative Risk Posture Program Failure Review Board o Root Cause Complexity and Systemic Risks Metrics o Trends and Outliers Residual Risk Management Ensures Baseline Compliance Page 10

11 Managing Uncertainty Across Product Types Retired Risks No Residual Risk Artifacts Known- Knowns Risk Artifacts Open Risks Uncertainty Uncertainties Known- Unknowns Accepted Risk Unknown- Knowns Execution Risk Unknown- Unknowns Hidden Risk Test as you Fly Validation Demonstrated TPM Performance Flight or test-validated analysis, simulations and models Open Residual Risks Analysis / test limitations Unverified Models/Simulations Envelope expansion Unverified failure modes Miscommunicate test/analysis Understanding of data/ envir Poor documentation combined with loss of corporate memory Bad assumptions Unfinished foundation research Untested new environments Inadvertent operations outside of limits Incremental knowledge Buildup Complete verification and validation Risk Handling Evaluate Deltas due to o Baseline limitations o Margin gaps o In-complete V&V o Analysis thoroughness Program communications / data sharing Incremental knowledge build-up w/ trending TRL level 6 by PDR Envir analysis/test rigor Simulators & test-beds fidelity, TAYF Design Margins Uncertainty Management Core to an Effective Risk Management Strategy Page 11

12 Process/Product Risk Exceptions Management Risk Avoidance/Mitigation SMA DRIVING ACTIVITIES System Safety Engineering Hazard Analysis, Inhibit Design, FTA, Flight/Operations/Range Safety Reliability & Maintainability Fault Tolerance, SPFs, Lifetime, Deratings, Circuit Margins Parts, Materials, & Processes PMPCB, Qual/Screening, Radiation, GIDEP, Compatibility, Allowables Hardware Quality Assurance Workmanship, Process Control, Non-conformances, Inspection Software Quality Assurance CMMI L3, Reliability, Safety, Firmware, Peer Review, IV&V DISCIPLINE PRODUCT LEVERAGE Safety Compliance Verification - Equipment & personnel hazards - Pyros, batteries, mechanisms Fault Tolerance/Avoidance Balance - Anticipated/unanticipated faults - Random & environmental origins No Infant Mortality & Wearout - Screening/Qual exceptions - Degradation, Contamination Processes Controlled, No Latents - Process control & capability - Latent and patent defects S/W Maturity,Test Effectiveness - Residual defects: phase, interval - Test efficacy, saturation Design/Production Flow-down - Process Requirem ents - Guidelines/ Lessons Learned - Workmanship - Part/ Material Standards FMEA,FTA,PRA - Redundancy, Cross- strapping - Graceful Degradation - SPF CIL Mitigation - Common Cause Margins - Derating, EOL, LLIs, Budgets Inspection and Screening - Patent Defect - Latent Defect - Infant Mortality Burn- in - Operational Profi le Confi dence Qualification - Environm ental Tolerance - Life Characterization - Radiation Characterization Reliability Growth - Categorization, Phase, Trending Process Capability - Control with Randomness - Coverage, Com pleteness - Meets User Ex pectations SME Residual Risk Identification, Management, Elimination/Mitigation Page 12

13 Integration Readiness and Anomaly Ratings

14 Next Step Readiness Assurance Manages Risks and Supports Verification Integration and Test Readiness Verification (I&TRV): Next level integration readiness Analyses: complete, Production: current, Non-conformances: addressed, Testing: compliant Mission Assurance/Systems Engineering: Manage residual risk and requirement verification Subcontracted Hardware: Integrate I&TRV with the pre-ship review process Review Items I&TRV Assessment - Drawings, Specs, Engineering - Change Proposals - PFS, SOW, ICD's - Waivers, MRB/FRB Results - Previous Unit Risks - Environmental Test Results - Reliability/Parts/Design Analysis - Telemetry/Calibration Data - Mass Data - Operational/Handling Constraints - Adequate Compliance Testing - STE and Documentation Readiness - Waivers/Liens Closure Plan - Receiving Organization Readiness - Pre-integration Critical Items - Operations and Handling Constraints I&TRV Manages Integration Risk In-Phase with the Development Page 14

15 Risk Rating and Tracking of Anomalies What: System/Hardware/Software/Test Anomaly and Non-Conformance Reports Capture: Anomaly/Failure Effect; Current Active Cause and Corrective Action Residual Risk: Post investigating/maturity transferred to program risk Ratings programmatically prioritizes anomalies and determine risk to mission success Failure Effect Rating (excluding redundancy) Failure Cause/Corrective Action Rating Severity R R Cause/Corrective Action Negligible (N) 1 1 Significant (S) 2 2 Catastrophic (C) 3 3 High Priority 4 Known cause/certainty of corrective action (No residual risk) Unknown cause/effective corrective action (No residual risk) Known cause/uncertainty in corrective action (Some residual risk) Unknown cause/uncertainty in corrective action (Residual risk) Risk Rating and Essential Tool in Anomaly Residual Risk Burn-Down Page 15

16 Cumulative Risk Analysis

17 Significant Risks Escapes Cumulative Risk Management Risk Mitigation Efficacy: Ensuring a low risk profile, eliminating and controlling risks Risk Triad: Programmatic (cost and schedule), Technical, and Cumulative residual risk Metrics: (1) Risk identification, (2) Burn-down efficacy, (3) Orthogonal in-phase detection, and (4) Risk escapes Proposal Planning Rqmts Def. Prel Design Detail Design Risk Closure Pre-Fab Fab & Assy I&T Del & Support Life Cycle Program Phase Post Delivery High Risks 3 Moderate 2 Risks 1 Low Risks 0 Total Risks Process Escapes Cumulative Product & Process Assessment of Contribution to Mission Success Page 17