ISO, CMMI and PMBOK Risk Management: a Comparative Analysis

ISO, CMMI and PMBOK Risk Management: a Comparative Analysis Cristine Martins Gomes de Gusmão Federal University of Pernambuco / Informatics Center Hermano Perrelli de Moura Federal University of Pernambuco / Informatics Center Abstract All sectors of society are worried about quality, either of product or service quality. In the software industry is not different, the need to manage risk, has a strong relation with quality and increases with system complexity. Unanticipated problems frequently cause major problems to projects, such as cost overruns, schedule delays, quality problems, and missing functionality. Making informed decisions by consciously assessing what can go wrong, as well as the likelihood and severity of the impact, is at the heart of risk management. This paper presents a comparative analysis of quality approaches to risk management. It has the purpose to provide a general overview of all the risks aspects treated in ISO standards, CMMI model version 1.1 and PMBOK. Keywords Risk management, risk assessment, risk control, ISO, Capability Maturity Model Integrated, CMMI, project management body of knowledge, PMBOK. Introduction Risk management can be traced to the eighteenth century Era of Enlightenment, a time of searching for knowledge and exploring the unknown. Today risk management is a general procedure for resolving risks. In Hall (1998), risk management is said to resolve a risk if, when it is applied to any instance, the possible consequences are all acceptable. Software development is often plagued with unanticipated problems that cause projects to miss deadlines, exceed budgets, or deliver less than satisfactory products. While these

problems cannot be eliminated totally, some of them can be controlled well by taking appropriate preventive action. As Pressman (1995) said, risk management deals with these problems before they occur. Organizations may be able to avoid a large number of problems if they use systematic risk management procedures and techniques early in projects. Risk Management is generally one of the main topics of interest for researchers and practitioner s working in the area of project management. Several risk management approaches have been proposed and used by SEI (1990), Charette (1990), Fairley (1994) since Boehm (1991), but it still has a long way to go. While some organizations defined their own risk management approaches others do not manage their risks explicitly and systematically. Risk management based on intuition and individual efforts alone is rarely effective and consistent. Risk Management From an overall business perspective, the success of many organizations is becoming increasingly dependent on the success or failure of the software they build. In this environment, managing risk is not only a sound development practice, but also a vital business practice (Hall, 1998). Risk Management is generally one of the main topics of interest for researchers and practitioner s working in the area of project management: SEI Software Engineering Institute (1990) defines the risk management process through the Continuous Risk Management Model that includes five distinct phases: Risk Identification, Risk Analysis, Risk Track, Risk Control and Risk Monitor connecting by an ongoing risk communication effort. Charette (1990) defined the Risk Software Engineering including two phases: Risk Evaluation (Risk Identification, Risk Estimative and Risk Evaluation) and Risk Management (Risk Planning, Risk Control and Risk Monitoring). Boehm (1991) presents a process with two main phases: Risk Assessment (Risk Identification, Risk Analysis and Risk Prioritization), and Controlling Risk (Risk Management Planning, Risk Resolution and Risk Monitoring). Fairley (1994) presents the project risk management with seven steps: (1) Identify risk factors; (2) Assess risk probabilities and effects; (3) Develop strategies to mitigate identified risks; (4) Monitor risk factors; (5) Invoke a contingency plan; (6) Manage the crisis; (7) Recover from the crisis. Kleim and Ludin (1997) proposed a four-phase process: Identification, Analysis, Control and Reporting based on PDCA Plan Do Act Check for software quality. Chapman and Ward (1997) describe a generic risk management process consisting of nine steps: (1) Define the key aspects of the project; (2) Focus on a strategic approach to risk management; (3) Identify where risks might arise; (4) Structure the information about risk assumptions and relationships; (5) Assign ownership of risks and responses; (6) Estimate the extent of uncertainty; (7) Evaluate the relative magnitude of the various risks; (8) Plan responses; and (9) Manage by monitoring and controlling execution. The PMI Project Management Institute (2000) presents six phases for risk management: Risk Management Planning, Risk Identification, Qualitative Analysis Risk, Quantitative Risk Analysis, Risk response Planning and Risk Monitoring and Control (PMI, 2000).

The CMMI (2001) Capability Maturity Model Integration defines risk management process with three phases: Risk Assessment, Risk Control and Risk Reporting. These representative backgrounds for risk management processes present a general agreement regarding what is included in the process with the differences depending on variation in the level of detail and on the assignment of activities to steps and phases. When risk management methods are used, they are often simplistic and users have little trust in the results of their risk analysis results. Given the increasing interest in risk management in the software industry, for applying risk management more widely, is necessary to provide comprehensive support for risk management, guidelines for application, support communications between the stakeholders and be credible. Risks The common definition of risks, either by dictionaries or usages of the term risk, associates several different meanings. It can refer to a possibility of loss, injury, or destruction. Knight (1921) in his dissertation said that risk is an exposure to uncertain events with well-known probabilities, while uncertainty is the exposure to events with unknown or less well-determined probabilities. The math definitions for risk is a random variable, describing the financial exposure of an economic entity to uncertain events and dictionary definitions for risk are so broad that it is far to define risk as anything that is related to the possibility of loss. Clearly, there is some value in having such an extended and encompassing concept to facilitate initial discussion about risk. However, there is this wide range of meanings associated to the word risk can also prevent adequate precision in more detailed analysis or risks unless this ambiguity is explicitly addressed and removed, as Humphrey (1990) said. Risk Management Process Definition The application of concepts and principles of risk management in software development has required adaptation. This section presents the analysis of the different perspectives of the activities that compose the risk management process in the standards and models. The standards and models treated in this paper are: SEI Software Engineering Institute (Higuerag, 1994), PMBOK Project Management Body of Knowledge (PMI, 2000), CMMI Capability Maturity Model Integrated (2001), ISO/IEC 12207 (2002), ISO/IEC 15504 (1999) and ISO/IEC 9000-3 (1991). SEI Risk Management The Software Engineering Institute, a leading source of methodologies for managing software projects, develops a Continuous Risk Management (CRM) paradigm that consists of five distinct phases (identification, analysis, response planning, tracking, and control) linked by an ongoing risk communications efforts (Higuerag, 1994). The CRM icon is illustrated in Figure 1.

Control Track Identify Communicate Plan Analyze Figure 1. Continuous Risk Management paradigm. The risk identification involves determining what risks might affect the project development though brainstorming and interviewing among developers, subject matters experts, customers, stakeholders, and outside experts. It is commonly accepted that risk categories helps to systematically organize and identify possible risks. Then, the risk list must be analyzed in such a way to determine the probability of it occurrence. One of the tasks of the planning is to establish a set of risk-control functions to bring the risks items under control (e.g. software metrics). Finally, the last cited activity: communication. Provide information and feedback internal and external to the project on the risk activities, current risks, and emerging risks. Each risk nominally goes through these functions sequentially, but the activity occurs continuously, concurrently (e.g., risks are tracked in parallel while new risks are identified and analyzed), and iteratively (e.g., the mitigation plan for one risk may yield another risk) throughout the project life cycle (Higuerag, 1994). ISO/IEC 9000-3 Risk Management ISO/IEC 9000-3 (1991) is an application guide of ISO 9001 for the development, supply and maintenance of software. ISO 9001 is part of the series of norms ISO 9000. These norms specify the minimum requirements, so that the companies can assure the quality of its products and services. In June of 1993 Norm ISO/IEC 9000-3 with lines of direction for application of ISO 9001 to the development was created, supply and maintenance of software. For each item of ISO 9001 a correspondent in ISO/IEC 9000-3 exists that details it and adjusts to software. The lines of direction proposals in ISO 9000-3 cover questions as the common agreement between the parts (contracting and contracted) of functional requirements and the use of consistent methodologies for the software development and project management as a whole, of the conception until the maintenance. All the orientations of ISO/IEC 9000-3 address to the contractual situation, where one another Company contracts the Company in question to develop a Product of Software. ISO/IEC 9000-3 does not possess a specific process for risk management, but it presents the activity of controlling risks in acquisition management.

ISO/IEC 12207 and ISO/IEC 15504 Risk Management The ISO/IEC 12207 (2002) is the first international norm that describes in details the processes, activities and tasks that involve the supply, development, operation and maintenance of software products. The main purpose of this norm is to serve of reference for the too much standards that come to appear. Launched in August of 1995, it is cited in almost all the works related to the Software Engineering since then, also to those relative ones to the quality. The standard is voluntary; that is, it does not in itself impose any obligation upon anyone to follow it. Yet, it may be imposed by an organization through internal policy directive or by individual parties through contractual agreements. ISO/IEC 12207 is being modified to be in accordance with ISO/IEC 15504 (1999) (SPICE - Software Process Improvement and Capability determination) Part 5: An Assessment Model and Guidance Pointer. As a result of this alteration, ISO/IEC 12207 will go to substitute the dimension of processes of ISO/IEC 15504 - Part 5, or either, the existing processes currently in ISO 15504 will be enclosed in ISO/IEC 12207 through its annex ISO/IEC PDAM 12207 - Amendment the ISO/IEC 12207 (2002). Project SPICE objectified the creation of norms for the evaluation of processes and the continuous improvement of these processes, being based on the best characteristics of evaluation models as CMM - Capability Maturity Model (ISO/IEC 15504, 1999). The improvement of processes is carried through evaluations, that they describe practical usual of the organization, an organizational unit or a project. The analysis of the results is made in relation to the necessities of the business of the organization, having raised negative and positive aspects, as also the involved risks in the process. The best practices for managing risk, in accordance with ISO/IEC 15504 (1999), are the following ones: Establishing the target of risk management: to determine the target of risk management that will be used by the project, in accordance with the politics of organizational risk management. Risk Identification: identifying risks in the beginning and during projects execution. Analyze and prioritize risks: to evaluate the probability of occurrence, the impact, the time of occurrence, the cause and the relations between the risks, determining the priorities. Defining the strategy for risk management: to define a strategy appropriate to managing risk or a set of risks, in a project level and organizational level. Defining metrics for risks: for each risk or set of risks, to define the metric ones for gauging of the change in the situation of the risk and the progress of the activities of reduction. Implementing strategy of risk management: to execute the strategy defined for managing risk. Evaluating the results of the strategy of risk management: in daily paydefinitive points of control, to apply metric the definite ones to evaluate the waited progress and the level of success of the strategy of risk management.

Executing the corrective actions: when the progress waited in the reduction of the risk is not reached, to execute corrective actions to correct or to prevent the impact of risks. CMMI Risk Management SEI - Software Engineering Institute, under the coordination of Humphrey (1987), generated the first version of model CMM - Capability Maturity Model. In 1991, SEI evolved the structure of maturity of process for the SW-CMM - Capability Maturity Model for Software. As result of the evolution of model SW-CMM, in 2000 was launched model CMMI - Capability Maturity Model Integrated, which adds, beyond the representation for periods of training (SW-CMM), the continuous representation. In the continuous representation six levels of capacity exist, assigned for the numbers of 0 the 5 that they correspond: level 0 - Incomplete, level 1 - Executed, level 2 - Managed, level 3 - Defined, level 4 - Managed Quantitatively and level 5 Optimized. The components of model CMMI can be grouped in three categories: SG - Specific Goals and GG - Generic Goals component is required and considered essential so that the organization reaches the improvement of the process; SP - Specific Practices and GP - Generic Practices component is waited and can help to reach the specific and generic objectives; Sub-practical, extension of disciplines, elaboration of generic practices, headings of practices and objectives helps to understand the model. Risk management process in CMMI (2001) included three phases, as shown in Figure 2: Risk Assessment Identification Risk Management Risk Control Planning Risk Reporting Figure 2. CMMI Risk Management Process. Risk Assessment: Identification - Listing the risks; Analysis - Determining the probabilities and impacts and Prioritization - Ranking the risks for action. Risk Control: Planning - Determining how and when to take action; Resolution - Taking risk mitigation action and Monitoring - Measuring the outcome. PMBOK Risk Management The PMI - Project Management Institute is an association of professionals of projects management that exists since 1969. This association created in 1986 the first version of the PMBOK - Project Management Body of Knowledge. The PMBOK is a

guide who inside describes the set of knowledge and best the practical ones of the profession of projects management. It is a generic material that serves for all the knowledge areas (PMI, 2000). The PMBOK (PMI, 2000) organizes the processes of management in five groups, as presented in Figure 3: initiating processes, planning processes, executing processes, controlling processes and closing processes. The PMBOK processes are organized by knowledge areas that if they inside relate to an aspect to be considered of the projects management. Inside of these areas of knowledge the five groups of the above-described processes can occur (PMI, 2000). Figure 3. Processes group in a Phase. In the 2000 edition, PMBOK (PMI, 2000) presents six processes in risk management, as follow: Risk Management Planning: deciding how to approach and plan the project s risk management activities. Risk Identification: determining the risks that might affect the project and documenting their characteristics. Qualitative Risk Analysis: analyzing conditions and risks qualitatively to determine and prioritize their impact on project objectives. Quantitative Risk Analysis: determining the probability and consequences of risks and estimating their impact on objectives. Risk Response Planning: determining how to enhance opportunities and minimize threats to objectives. Risk Monitoring and Control: executing risk response plans, monitoring risks, identifying new risks and evaluating the effectiveness of responses. The PMBOK stresses that attention to risk management will contribute to project success and to be successful, organizations must address risk management throughout the life of the project. This requires gathering data on project risks and their characteristics. Comparative Analysis After the introduction of several approaches of risk management, this section presents the results of a comparative study, based in a comparative analysis among the activities of SEI Continuous Risk Management and the ISO standards, CMMI and PMBOK, as illustrated in table 1.

Continuous Risk Management Paradigm was chosen for being a reference, always constant in the literature of Software Engineering, a model boarded since 1990 and used by great governmental institutions. The definitions of the processes are differentiated in relation to the nomenclature used for the description of the activities, to the subdivision of these activities in tasks and the definition of the target of determined activities. Differently of the approach presented for CMMI model (2001) and PMBOK (PMI, 2000), the model of SEI Risk Management, does not present a specific concern in defining a Risk Management Planning. ISO/IEC 12207 (2002) and ISO/IEC 15504 (1999) standards also define this activity, but they do not determine the method to be used for execution process. ISO/IEC 9000-3 (1991) standard only presents the necessity of definition of a planning of the development of the project to transform the specification of the requirements of the purchaser into a software product. Of this form, it is implicit the necessity of survey of the resources for risk management. Risk Identification produces lists of the project-specific risk items likely to compromise a project s success. It has the same objective in all the studied approaches. Perhaps the most important aspect of this activity is the requirements of formal documentation. The documentation has the purpose to facilitate the reuse of information of previous projects in new projects. It is important to remember that risk identification is an activity that must be carried throughout project execution. After the risks identification the next activity is the analysis of the same ones, which it has as objective the prioritized and cataloging. Risks Analysis assesses the loss probability and the loss magnitude for each identified risk item, and it assesses compound risks in risk-item interactions. It only appears as the same activity in all the approaches with exception of the PMBOK, where this technique can be divided into two major categories: qualitative and quantitative analysis. It is important to point out that all the approach define the necessity to measure the risks, but do not make reference to qualitative or quantitative analysis. After the analysis of the aspects of the identified risks, one lists classified for the importance of the risks can be elaborated to optimize the performance efforts. Risk prioritization produces a ranked ordering of the risk items identified and analyzed.

Table 1. Risk Management approaches: comparative analysis. Response Planning helps to prepare mitigation and contingency plans. This activity appears in all studied approaches. The CMMI (2001) establishes as development of reply to risk its reduction. One of the most important aspects of planning risk management is to become the possible use of resources most efficient to maximize the potential of performance of the project. The risk most important must has its action planned initially while the too much risks must have the cost of comparative mitigation to its impact, for the evaluation of the benefits of the effort to be unfastened. After the definition of mitigation and contingency plans, becomes necessary to monitoring risks. Risk monitoring involves tracking the project s progress toward resolving its risk items and taking corrective action when appropriate. This activity is the most different in all studied approaches. The PMBOK (PMI, 2000) approach says that corrective actions are necessary to controlling risk. The others like ISO standards are based in a PDCA Plan- Do-Check-Act cycle, than means a worried quality assurance. Through this comparative study we can observe that, the main involved stages in the risk management in projects, can be summarized as follows: Risks evaluation: the activity to evaluate contemplates the activities of risks identification that is no more then the classification as Boehm (1991) and Charette (1990), the risks analysis that evaluate the probability of occurrence and the impact of the risk and the prioritized that has the purpose to generate an ordinance of the risks, in accordance with the carried through analysis previously. Risk Control: controlling risks is formed by planning, or either, definition of strategies to control the identified risks, for the resolution of the risks that has the objective to simulate situations where the risks are decided and by the monitoring risks that involves the accompaniment of the progress of the project and when the exactly necessary taking of corrective actions, like PMBOK (PMI, 2000) and CMMI (2001).

Conclusion Risk management adds to project management a structural approach for risk identification and risk analysis to initiate the response planning. The risk planning creates the perspective of getting alternatives and contingencies to risk mitigation, while the functions of monitoring and controlling risk management combine with the function of control of project management. This paper presents an overview of risk management approaches since 1990. Discusses risk management processes in standards and models of quality, based in a comparative analysis of their activities. We analyzed the granularity and coverage of five processes as shown in table 1. From this comparative study it is clear that there is not a process standard to risk management, what exist are best practices consolidated in the project management: risk identification, analysis, prioritizing and controlling. In none of the approaches, treated here, exists the indication of processes and methods that can be used. The choice is of the organization, which in the reality adjusts its existing processes already to the activities of risk management and this can be a serious problem. Each organization has particularities and is essential to treat the organization as one to define the better risk management process. Finally, the definition of a plan to managing risk in an adequate manner is not an easy task, but it is essential for the improvement of the process of software development and therefore the guarantee of the quality. References Hall, E. M. (1998). Managing Risk. 2 nd Ed. USA: Addison Wesley. Pressman, R. S. (1995). Software Engineering: A practitioner s approach. 4 th Ed. McGraw- Hill. Charette, R. (1990). Application strategies for risk analysis. New York: MultiScience Press Boehm, B. W. (1991). Software Risk Management: principles and practices, IEEE Software, Volume 8. No1. Fairley, R. (1994). Risk Management For Software s Projects. IEEE Software. Kleim, R.L. and Ludin, S. (1997). Reducing Project Risk. Gower. Chapman, C. and Ward, S. (1997). Project Risk Management: Processes, Techniques and Insights. John Wiley & Sons. ISO 9000-3. (1991). Guidelines for the application of ISO 9001 to the development, supply and maintenance of software. International Standard Organization. ISO/IEC 15504. (1999). ISO 15504 Part 5: An Assessment Model and Indicator Guidance. ISO/IEC JTC1 SC7. International Standard Organization.

ISO/IEC 12207. (2002). ISO/IEC 12207 Information Technology. Amendment to ISO/IEC 12207. ISO/IEC JTC1 SC7. International Standard Organization. Humphrey, W. (1987). Characterizing the software process: a maturity framework, Technical Report. Software Engineering Institute SEI, Carnegie Mellon University, USA. Available on the World Wide Web: http://www.sei.cmu.edu PMI - Project Management Institute. (2000). A Guide to the Project Management Body of Knowledge. Available on the World Wide Web: http://www.pmi.org/pmi/publictn/pmboktoc.htm. Knight, F.H. (1921). Risk, uncertainty and profit. Houghton Mifflin, Boston. Humphrey, W.S. (1990) Managing the Software Process. Addison Wesley. CMMI - Capability Maturity Model Integration (2001), version 1.1 Pittsburgh, PA. Software Engineering Institute, Carnegie Mellon University. USA. Higuerag, P.R. (1994). An Introduction to Team Risk Management, Technical Report. Software Engineering Institute, Carnegie Mellon University, USA. Available on the World Wide Web: http://www.sei.cmu.edu