Paul R. Croll Chair, IEEE SESC Computer Sciences Corporation pcroll@csc.com IEEE 1540 - Software Engineering Risk : Measurement-Based Life Cycle Risk PSM 2001 Aspen, Colorado
Objectives Describe Risk in the context of a life cycle process framework Describe IEEE 1540 s Risk process model and process requirements Describe other Standards that complement IEEE 1540 in managing risk in the acquisition and engineering of software intensive systems PSM 2001, Aspen, CO Tuesday, July 24, 2001, 1:15 PM - 1:55 PM Paul R. Croll - 2
Risk (RM) in the Life Cycle Context An organizational life cycle process responsibility of the organization using the process the organization ensures that the process exists and functions IEEE Standard 1540 assumes that the other management and technical processes of IEEE/EIA 12207 perform the treatment of risk PSM 2001, Aspen, CO Tuesday, July 24, 2001, 1:15 PM - 1:55 PM Paul R. Croll - 3
IEEE/EIA 12207 Life Cycle Process Tree PRIMARY ACQUISITION SUPPLY DEVELOPMENT OPERATION MAINTENANCE LIFE CYCLE Source: Singh97 SUPPORTING DOCUMENTATION CONFIGURATION MANAGEMENT QUALITY ASSURANCE VERIFICATION VALIDATION JOINT REVIEW AUDIT PROBLEM RESOLUTION Risk touched on in 12207 Risk focused on in 12207 ORGANIZATIONAL MANAGEMENT INFRASTRUCTURE IMPROVEMENT TRAINING TAILORING PSM 2001, Aspen, CO Tuesday, July 24, 2001, 1:15 PM - 1:55 PM Paul R. Croll - 4
Risk Objectives in IEEE/EIA 12207 Sprinkled throughout the Acquisition, Supply, Development, Operation, Verification, Joint Review, Problem Resolution, and Tailoring Processes Focused on in Process objectives Determine scope of risk management to be performed Identify risks to the project as they develop Analyze risks Determine mitigation priority Define, implement and assess mitigation strategies Define, apply and assess risk metrics PSM 2001, Aspen, CO Tuesday, July 24, 2001, 1:15 PM - 1:55 PM Paul R. Croll - 5
IEEE/EIA 12207 Process Interactions MANAGEMENT INFRASTRUCTURE ORGANIZATION IMPROVEMENT TRAINING M PROJECT F F F F OPERATION E: 3 T F ACQ - ACQUISITION. SUB - SUBCONTRACTOR E - EXECUTE ACQUISITION SUPPLY U: 4 T U: 4 E T P JOINT E REVIEW E: ACQ E E: 3 T: SUB U E (T)E AUDIT E: 3 (I)V&V E: 3 P E E QA E: 3 V&V E: 3 MAINTENANCE E: 2,3 U DEVELOPMENT E: 1,2,3 F - FEEDBACK M - MANAGE P - PARTICIPATE T - TASK U - USE E:N - EXECUTE THE PROCESS NUMBERED N Source: Singh97 1 2 3 4 E PROBLEM DOCUMENTATION CM RESOLUTION TAILORING PDCA PSM 2001, Aspen, CO Tuesday, July 24, 2001, 1:15 PM - 1:55 PM Paul R. Croll - 6
IEEE/EIA 12207 Process Roles ACQUISITION ROLE SUPPLY ROLE OPERATING ROLE ACQUIRER SUPPLIER OPERATOR USER ACQUISITION PROCESS contract SUPPLY PROCESS employ employ employ OPERATION PROCESS employ employ employ S U P P O R T I N G ENGINEERING ROLE SUPPORTING ROLE DEVELOPER MAINTAINER EMPLOYER OF SUPPORTING PROCESSES use MAINTENANCE PROCESS use Documentation Configuration management Quality assurance Verification DEVELOPMENT PROCESS employ Validation Joint review Audit Problem resolution P R O C E S S E S ORGANIZATIONAL ROLE MANAGER ORGANIZATIONAL PROCESSES Infrastructure Improvement Training Source: Singh97 PSM 2001, Aspen, CO Tuesday, July 24, 2001, 1:15 PM - 1:55 PM Paul R. Croll - 7
Risk Process Overview Information Needs Decisions ❶ Technical and Processes ❺ Perform Risk Treatment Feedback and Risk Action Requests ❷ Plan and Implement Risk Improvement Actions ❼ ❸ Manage the Project Risk Profile Evaluate the Risk Process Analysis Monitoring PSM 2001, Aspen, CO Tuesday, July 24, 2001, 1:15 PM - 1:55 PM Paul R. Croll - 8 ❹ ❻ Source: IEEE Standard 1540:2001 IEEE 2001. All rights reserved.
Risk Process Overview Information Needs Decisions ❶ Technical and Processes ❺ Perform Risk Treatment Feedback and Risk Action Requests Define the information requirements for RM information needed and priority risk ❷areas Plan and of concern Implement RM policies Risk required risk acceptability thresholds Make decisions regarding risks Make recommendations for improving the RM process Improvement Actions ❼ ❸ Manage the Project Risk Profile Evaluate the Risk Process Analysis Monitoring PSM 2001, Aspen, CO Tuesday, July 24, 2001, 1:15 PM - 1:55 PM Paul R. Croll - 9 ❹ ❻ measurement focus Source: IEEE Standard 1540:2001 IEEE 2001. All rights reserved.
Risk Process Overview ❷ measurement focus Plan and Implement Risk Establish RM policies to support information required by decision makers how RM is to be performed Information Needs and how RM activities Feedback Decisions will be coordinated Improvement Actions ❶ tools or techniques Technical to be used how risk is to Processes be communicated Establish the RM ❺ process ❼ ❸ Manage the Project Risk Profile Perform Risk Treatment Establish responsibility for RM Assign RM resources Establish RM process evaluation Evaluate the Risk Process and Risk Action Requests Analysis Monitoring PSM 2001, Aspen, CO Tuesday, July 24, 2001, 1:15 PM - 1:55 PM Paul R. Croll - 10 ❹ ❻ Source: IEEE Standard 1540:2001 IEEE 2001. All rights reserved.
Risk Process Overview ❷ measurement focus Plan and Implement Risk Information Needs and stakeholder(s) Feedback Decisions perspective(s) Improvement Actions Create a consistent current and historical view of the risks present and their treatment Define the technical and managerial risk management context ❶ risks Technical areas of concern objectives, Processes assumptions and constraints Manage the Project Risk Profile Perform Risk Treatment Evaluate the Risk Process and Risk Action Requests Establish ❺ risk thresholds Establish and maintain the project risk profile Communicate risk status to stakeholders ❼ ❸ Analysis Monitoring PSM 2001, Aspen, CO Tuesday, July 24, 2001, 1:15 PM - 1:55 PM Paul R. Croll - 11 ❹ ❻ Source: IEEE Standard 1540:2001 IEEE 2001. All rights reserved.
Risk Process Identify risks defined by RM context Estimate Overview risk likelihood and consequences Evaluate and prioritize the risks and their interactions against thresholds Technical and Processes Recommend risk Information treatment Needs where applicable Document in risk action request Decisions measures of treatment effectiveness ❺ contingency plans ❶ Perform Risk Treatment Feedback and Risk Action Requests ❷ measurement focus Plan and Implement Risk Improvement Actions ❼ ❸ Manage the Project Risk Profile Evaluate the Risk Process Analysis Monitoring PSM 2001, Aspen, CO Tuesday, July 24, 2001, 1:15 PM - 1:55 PM Paul R. Croll - 12 ❹ ❻ Source: IEEE Standard 1540:2001 IEEE 2001. All rights reserved.
Risk Process Overview Information Needs Decisions ❶ Technical and Processes ❺ Perform Risk Treatment Feedback and Risk Action Requests evaluates risk action requests and determines acceptability of risks ❸ If risk reduction Plan and actions are to be Manage taken, ❷ Implement the management selects, plans, monitors, Projectand Risk Risk Profile controls treatment to decrease risk exposure Improvement Actions ❼ Evaluate the Risk Process Analysis Monitoring PSM 2001, Aspen, CO Tuesday, July 24, 2001, 1:15 PM - 1:55 PM Paul R. Croll - 13 ❹ ❻ Source: IEEE Standard 1540:2001 IEEE 2001. All rights reserved.
Risk Process Overview Information Needs Decisions ❶ Technical and Processes ❺ Perform Risk Treatment Feedback and Risk Action Requests Improvement Actions ❼ Evaluate the Risk Process Analysis Once a risk treatment has been selected ❸ if Plan and Manage ❷a 12207 Life Cycle Process is employed, Implement the + risk Project Risktreatment is managed using the problem Risk Profile management approach of the Process if a non-12207 Life Cycle Process is ❻employed, Monitoring + a detailed Risk Treatment Plan must be developed and implemented PSM 2001, Aspen, CO Tuesday, July 24, 2001, 1:15 PM - 1:55 PM Paul R. Croll - 14 ❹ Source: IEEE Standard 1540:2001 IEEE 2001. All rights reserved.
Risk Process Overview Information Needs Decisions ❶ Technical and Processes ❺ Perform Risk Treatment Feedback and Risk Action Requests ❷ measurement focus Plan and Implement Risk Improvement Actions ❼ ❸ Manage the Project Risk Profile Evaluate the Risk Process Review and update individual risk states ❹and the management context Assess Analysis effectiveness of risk treatments Seek out new risks Monitoring PSM 2001, Aspen, CO Tuesday, July 24, 2001, 1:15 PM - 1:55 PM Paul R. Croll - 15 ❻ Source: IEEE Standard 1540:2001 IEEE 2001. All rights reserved.
Risk Process Overview ❷ measurement focus Plan and Implement Risk Information Needs Improvement Actions Decisions ❼ ❸ ❶ Manage the Project Risk Profile Technical and Processes Perform Risk Treatment Evaluate the Risk Process Feedback and Risk Action Requests ❺Capture RM information Assess and improve the RM process collect RM information assess the quality of the process Analysis Monitoring PSM 2001, Aspen, CO Tuesday, July 24, 2001, 1:15 PM - 1:55 PM Paul R. Croll - 16 ❹ identify opportunities for improvement provide feedback to management make improvements to the process Generate lessons learned ❻ Source: IEEE Standard 1540:2001 IEEE 2001. All rights reserved.
IEEE 1540 and ISO/IEC 15026 ISO/IEC 15026:1998, Information Technology System and Software Integrity Levels Defines a process for establishing integrity levels that are used to contain risk within acceptable values the system integrity level reflects the worst case risk that is associated with the as-designed system all appropriate risk dimensions are addressed Requires employment of a risk management process PSM 2001, Aspen, CO Tuesday, July 24, 2001, 1:15 PM - 1:55 PM Paul R. Croll - 17
IEEE 1540 and ISO/IEC 15939 ISO/IEC 15939:FDIS, Information Technology Software Measurement Process Identifies the activities and tasks that are necessary to successfully identify, define, implement, and improve a software measurement process Two core activities Plan the Measurement Process Perform the Measurement Process Two supporting activities Establish and Sustain Measurement Commitment Evaluate Measurement PSM 2001, Aspen, CO Tuesday, July 24, 2001, 1:15 PM - 1:55 PM Paul R. Croll - 18
IEEE 1540 and ISO/IEC 15939-2 References to risk in ISO/IEC 15939 Plan the Measurement Process Identify Information Needs Annex A: The measurement information model Measurable Concept PSM 2001, Aspen, CO Tuesday, July 24, 2001, 1:15 PM - 1:55 PM Paul R. Croll - 19
IEEE 1540 and IEEE 1012 IEEE Std 1012-1998, IEEE Standard for Software Verification and Validation Uses integrity levels to determine appropriate V&V activities These integrity levels could be determined in the baseline risk profile PSM 2001, Aspen, CO Tuesday, July 24, 2001, 1:15 PM - 1:55 PM Paul R. Croll - 20
IEEE 1540 and IEEE 1228 IEEE Std 1228-1994, IEEE Standard for Software Safety Plans Addresses planning for a software safety program that provide a systematic approach to reducing software risks Requires that a risk assessment be performed to identify potential safety risks Requires that risk treatment alternatives be addressed for uncontrolled risks PSM 2001, Aspen, CO Tuesday, July 24, 2001, 1:15 PM - 1:55 PM Paul R. Croll - 21
IEEE 1540 and IEEE 1058 IEEE Std 1058-1998, IEEE Standard for Software Project Plans requires the specification of a risk management plan identification, analysis and prioritization of project risk factors procedures for contingency planning, risk monitoring, and changes in risk status PSM 2001, Aspen, CO Tuesday, July 24, 2001, 1:15 PM - 1:55 PM Paul R. Croll - 22
IEEE 1540 and IEEE 982.1 and 982.2 IEEE Std 982.1-1988, IEEE Standard Dictionary of Measures to Produce Reliable Software measures appropriate for use in risk management IEEE Std 982.2-1988, IEEE Guide for the Use of IEEE Standard Dictionary of Measures to Produce Reliable Software guidance regarding measures appropriate for use in risk management PSM 2001, Aspen, CO Tuesday, July 24, 2001, 1:15 PM - 1:55 PM Paul R. Croll - 23
For more information... Paul R. Croll Computer Sciences Corporation 5166 Potomac Drive King George, VA 22485-5824 Phone: +1 540.663.9251 Fax: +1 540.663.0276 e-mail: pcroll@csc.com For IEEE Standards: http://standards.ieee.org/catalog/ For the IEEE Software Engineering Standards Committee: http://computer.org/standard/sesc/ PSM 2001, Aspen, CO Tuesday, July 24, 2001, 1:15 PM - 1:55 PM Paul R. Croll - 24
Questions? PSM 2001, Aspen, CO Tuesday, July 24, 2001, 1:15 PM - 1:55 PM Paul R. Croll - 25
References [IEEE 982.1] IEEE Std 982.1-1988, IEEE Standard Dictionary of Measures to Produce Reliable Software, Institute of Electrical and Electronics Engineers, Inc. New York, NY, 1988. [IEEE 982.2] IEEE Std 982.2-1988, Guide for the Use of IEEE Standard Dictionary of Measures to Produce Reliable Software, Institute of Electrical and Electronics Engineers, Inc. New York, NY, 1988. [IEEE 1012] IEEE Std 1012-1998, IEEE Standard for Software Verification and Validation, Institute of Electrical and Electronics Engineers, Inc. New York, NY, 1998. [IEEE 1228] IEEE Std 1228-1994, IEEE Standard for Software Safety Plans, Institute of Electrical and Electronics Engineers, Inc. New York, NY, 1994. PSM 2001, Aspen, CO Tuesday, July 24, 2001, 1:15 PM - 1:55 PM Paul R. Croll - 26
References - 2 [IEEE 1058] IEEE Std 1058-1998, IEEE Standard for Software Project Plans, Institute of Electrical and Electronics Engineers, Inc. New York, NY, 1998. [IEEE 1540] IEEE Standard 1540-2001, IEEE Standard for Software Life Cycle Processes Risk, Institute of Electrical and Electronics Engineers, Inc. New York, NY, 2001. [IEEE/EIA 12207] IEEE/EIA Standard 12207.0-1996, Industry Implementation of International Standard ISO/IEC12207:1995 (ISO/IEC 12207) Standard for Information Technology Software life cycle processes, Institute of Electrical and Electronics Engineers, Inc. New York, NY, 1998. [Singh97] Raghu Singh, An Introduction to International Standard ISO/IEC 12207, Software Life Cycle Processes, 1997. PSM 2001, Aspen, CO Tuesday, July 24, 2001, 1:15 PM - 1:55 PM Paul R. Croll - 27
References - 3 [ISO/IEC 15026:1998] ISO/IEC 15026:1998, Information Technology System and Software Integrity Levels, ISO/IEC, 1998. [ISO/IEC 15939] ISO/IEC 15026:FDIS, Information Technology Software Measurement Process, ISO/IEC, 2001. [Singh97] Raghu Singh, An Introduction to International Standard ISO/IEC 12207, Software Life Cycle Processes, 1997. PSM 2001, Aspen, CO Tuesday, July 24, 2001, 1:15 PM - 1:55 PM Paul R. Croll - 28