Exhibit 1: Structure of a heat map

Size: px
Start display at page:

Download "Exhibit 1: Structure of a heat map"

Transcription

1 Integrating risk and performance management processes Werner Bruggeman Geert Scheipers Valerie Decoene 1. Introduction Years ago, Kaplan & Norton interviewed managers about their time consumption and they found out that 85% of senior managers spent less than one hour per month on discussing about strategy. Most managers even spent less time on managing risks. Traditionally, companies management systems were focused on shareholder value, revenue growth, productivity, cost control and quality (Kaplan, 2009). Few companies explicitly incorporated risk into their management system. All of these changed due to the eruption of the financial crisis in Companies started wondering about the risks they were subject to. Risk management is not new; in 1988 the Basel I norm was introduced for banks and this norm was renewed in 2007 by the Basel II norm hence institutionalizing risk management for banks. Companies in other industries could rely on ERM (Enterprise Risk Management) and COSO (Committee of Sponsoring Organizations of the Treadway Commission) standards to organize their risk management; ERM is the process applied to strategy setting and across the organization to identify potential risk events, manage the risks within the organization s risk appetite and provide reasonable assurance regarding the achievement of the organization s objectives (COSO, 2004). Many companies established their own risk department and risk professionals had their own organizations. Despite all these precautions to handle risks more appropriately, a lot of companies were affected by the emanations of the financial crisis because of their exposure to risks. Main reason for this failure was the companies failure to explicitly account for risks when formulating their strategies, and their failure to monitor and manage the risks they had assumed (Kaplan, 2009, p 2). 2. Linking Risks to strategy To tackle these problems Kaplan and Norton decided to further expand their strategy execution framework and incorporate risk management into it. Risk management needs to be incorporated in the company s corporate DNA because it permeates every area and aspect of business and it has to be anticipative. By using the BSC-based strategy and performance management process, companies would be able to alleviate some of the excessive risk taking that they pursue for short-term financial gain and they could adopt a holistic view of risks. 1 Delaware Performance Management - Belgium The Netherlands France United States Asia Pacific

2 Kaplan (2009) classified risks into three categories based on their degree of predictability, controllability, management and magnitude of their consequences to the enterprise: 1. The lowest level level 3 comprises routine operational and compliance risks. These are risks arising from errors in routine, standardized and predictable processes that expose the company to a substantial loss. Companies try to minimize the level 3 defects and strive for zero deficiency by training its staff and the establishment of standard operating procedures and internal controls. We can say that level 3 risks are known and avoidable; hence management strives to achieve full compliance and zero defects. 2. The second level of risks is about strategy risks being risks inherent in the company s strategy; possible strategy risks are financial risk; customer, brand, and reputation risk; supply chain risk; innovation risk; environmental risk; human resources risk; and information technology risk; called the known unknowns. Such a whole list of possible risks could imply a complex risk management process, specific to each type of risk. However, the strategy map gives an overview of the companies strategic objectives and the interrelationships between them and can serve as a natural framework for identifying, mitigating and systematically managing the risks to a company s strategic objectives in an integrated and comprehensive manner (Kaplan, 2009, p 3). Hence, risk management can be incorporated into a company s strategy map: a strategy map identifies the Key Success Factors (KSFs) to realize the company s strategy. Those KSFs are made operationally by Key Performance Indicators (KPIs) and strategic initiatives are formulated to close the gap between current and desired performance. This part of the strategy map represents a positive idea: how are we moving forward to achieve an outstanding performance? Starting from the same strategy map, managers can think about events that could prevent an objective from being achieved. These events are called risks and are made operationally by Key Risk Indicators (KRIs). Actions can be formulated to keep these KRIs on track and hence identify, avoid and overcome the barriers that the strategy may encounter. Risk thus represents the negative idea but both risk and strategy are sides of the same coin and should be managed in an integrated way. 3. The third level of risks is called the unknown unknowns, black swans (Nassim Nicholas Taleb, 2007) or global enterprise risks. These risks are unpredictable, unprecedented occurrences that create existential risk. Companies should gain an insight in which unlikely event(s) could lead to their downturn. They can do this by organizing active discussions of unlikely events and their consequences these meetings are called tail-risk meetings since the likelihood of these events is in the tail of the probability distribution or by scenario planning; managers gain an insight in the correlated consequences of future events and are forced to think about their strategy in another way. These discussions or scenario planning can be incorporated in the sixth stage of the strategy execution system as defined by Kaplan and Norton: Test and Adapt the Strategy. Organizations have to test the robustness of their strategy. 2 Delaware Performance Management - Belgium The Netherlands France United States Asia Pacific

3 The strategic risk management process consists of 5 building blocks: 1. Strategy formulation and risk identification: Each company should articulate a clear and sound strategy. This strategy should be made visible by using a strategy map according to the BSC-principles (Kaplan & Norton, 1996). The strategy map gives an overview of the company s critical success factors to realize its strategy and the relationships among them. The KSFs represent hence a positive idea referring to the company s desired future. However, the strategy map provides also a framework to think about the negative side of the strategy story. It can help to identify the possible barriers to the realization of the strategy: risks. A risk is the possibility that an event will occur and adversely affect the achievement of the company objectives (COSO, 2004). Each KSF identified to realize the strategy can be linked with a risk; an event representing a barrier to the realization of the strategy. 2. Strategic and operational risks: A company should identify its key risk areas. Risks can occur at the strategic and operational level; strategic risks are about the events endangering the realization of the company s strategy and operational risks are about the possible negative effects arising from operational malfunctions. 3. Risk assessment: development of the heat map and the key risk indicators. A heat map is constructed by estimating two parameters: the likelihood of an event and the magnitude of the event s consequences. Each event gets a color code according to its position on the matrix (see exhibit 1 Exhibit 1: Structure of a heat map The heat map hence can be used to prioritize; risk events with a high score should get priority for the limited budget available for mitigating and preventing risks (Kaplan, 2009). For each risk a KRI can be identified and according to that, strategic initiatives are proposed. So the strategy map and BSC provide a framework for capturing both strategic initiatives concerning the realization of the strategy and the overcoming of possible barriers to its realization. 3 Delaware Performance Management - Belgium The Netherlands France United States Asia Pacific

4 4. Identification of control activities: When all risks and KRIs are identified, strategic initiatives can be formulated to keep the risks under control. The risk existing before taking corrective action is called the inherent risk; risk after executing a control activity is called residual risk. Concerning taking corrective actions to control risks, 4 possible strategies can be followed: (1) acceptance: the company accepts that it runs certain risks; (2) avoidance: the company avoids certain risks by shutting down some activities; (3) sharing: the company will share some of its risks with third parties by for example sourcing an activity out; and (4) reduction: the company will control the risks by applying appropriate control procedures for processes, employees, etc 5. Risk governance: Risk committee and risk management process: For a company to be resistant against crises and build out a robust strategy, risk management should be integrated in the strategy management process. Applying the Strategy Focused Organization model of Kaplan and Norton (Kaplan and Norton, 2001) Risk management should be developed along five dimensions (see exhibit 2): top management must be committed to manage risks effectively, company risks should be identified in operational terms, risk management strategies should be cascades vertically and horizontally in the organization, risk management should be everybody s job and risk management should be governed as a continual process. Exhibit 2: The five principles of Strategic Risk Management In organizing Risk Management as a continuous process companies may assign the risk management responsibility to the Office of Risk Management (ORM). Kaplan and Mikes (2011) prescribe that specific types of risk are best managed by specific staff functions, carrying out critical risk management processes (see exhibit 3) but 4 Delaware Performance Management - Belgium The Netherlands France United States Asia Pacific

5 the responsibility of the Office of Risk Management may be to coordinate and oversee the risk-management activities across the enterprise. Exhibit 3: Critical risk management processes according to type of risk 3. Strategic Risk Management at De Lijn, Belgium De Lijn is a public carrier, providing urban transport by bus and tram, founded in 1990 when the Belgian federal government transferred the responsibility for urban transport to the three Belgian regions. De Lijn found that they lacked insight in the risks that could withhold them from realizing their strategy. De Lijn already had an internal audit department but according to the rules of the Institute of Internal Auditors (IIA) the internal auditor should evaluate the risk management system but should not install it himself; this should be the task of the strategic management officer. In line with earlier work of Kaplan (2009) De Lijn decided to integrate its risk management process into its strategy process under the coaching of Delaware Performance Management. During an earlier phase of their strategy and visioning process, De Lijn already had developed a clear vision, mission and strategy. The strategy was made visible by use of a strategy map and cascaded down to the different entities (= Flemish provinces) and functional domains of De Lijn, assuring alignment of the corporate and lower level strategy maps. In a next phase the objective was to link the risk management process to those strategy maps. In the next months of the project, De Lijn implemented the 5 building blocks of the integrated risk management system: (1) Risks were identified using the preexisting strategy maps and for each KSF a KRI was established; (2) De Lijn categorized its possible risks into 2 categories being strategic and operational risks; (3) De Lijn assessed each risk s possible impact and likelihood in a heat map; (4) De Lijn identified control activities for the most prioritized risks; and (5) De Lijn appointed the responsibility of risk management to its Office of Strategy Management (OSM). 5 Delaware Performance Management - Belgium The Netherlands France United States Asia Pacific

6 1. Strategy formulation and risk identification: De Lijn already disposed of a clear and sound vision and strategy which was translated into a strategy map. The strategy map provided De Lijn with an insight into its Key Success Factors (KSFs) and their interrelationships. This map was at the same time the framework to identify possible risks that could withdraw De Lijn from realizing its strategy. For each KSF two parameters could be defined: from a positive standpoint each KSF could be characterized by a Key Performance Indicator (KPI) but from a negative standpoint each KSF could be linked to a risk, being an event withholding the company from realizing its strategy. Each risk could be characterized by a Key Risk Indicator (KRI). Example: the ultimate goal of the company was to make sure that its customers arrive safely on their destination. To realize this goal, the company has to deliver an excellent and reliable service. Excellent and reliable service was considered as an important KSF to realize the strategy. This KSF was linked with a KPI: the % of satisfied customers measured by the customer survey. However, on time delivery could be disturbed by certain risks such as a crash of the IT-systems. The appropriate KRI to track this risk was then the frequency with which the IT-systems are updated. 2. Strategic and operational risks: De Lijn made a distinction between risks threatening the realization of the strategy being strategic risks and possible negative effects arising from operational malfunctions being operational risks (see exhibit 4). Example: important strategic objectives in the strategy map were related to improving travelling comfort and experience, reliability of travelling times and service. The strategic risk related to these objectives was discontinuity of the services. A related operational risk was a major mistake by an employee. This risk was linked to the schedule monitoring process. Exhibit 4: Identifying strategic and operational risks 6 Delaware Performance Management - Belgium The Netherlands France United States Asia Pacific

7 3. Risk assessment: Managers of De Lijn estimated the likelihood and possible impact of each identified risk to position them in a heat map matrix. The heat map gave managers a support for prioritizing the different risks. Special attention was given to the risks in the red zone (i.e. high probability and high impact). These were monitored more carefully than risks in the green zone (i.e. low probability and low impact). 4. Identification of control activities: For the highly prioritized risks, De Lijn set out control activities to handle and control those risks carefully. These control activities gave De Lijn the opportunity to estimate the impact of the strategic initiatives to the inherent risk and hence calculate the residual risk. Example 1: De Lijn s employees are affiliated with a labor union. The strong unionized atmosphere makes it complex to control the organization, could impede the implementation of strategic changes and increased the chance for strikes. To minimize this risk the management decided on two important risk mitigating initiatives. They implemented a program of ameliorating the relationship with the unions and decided to let them participate in the strategy process. Example 2: The Lijn s high visibility in the media caused a risk that people could create a negative image about the organization. In order to minimize this risk De Lijn formulated some important new strategic initiatives (e.g. the implementation of a stakeholder management program and the organization of a unified external communication platform). 5. Risk governance. De Lijn decided to implement risk management as a continuous process. De Lijn secured this continuity by integrating its risk management system into its strategy management process and assigned risk management as being one of the responsibilities of the Officer of Strategy Management (OSM). The OSM became responsible for coordinating, coaching and monitoring of the company wide risk management processes. The OSM s role and responsibilities were clearly differentiated from the responsibilities of the internal auditor, who had to stay independent. The OSM reported to the General Manager while the internal auditor had to report to the board. The roles assigned to the OSM were roles that typically cannot be undertaken by the internal auditor (see exhibit 1). The internal auditor s role is to evaluate the risk management system and not to install it. Exhibit 1: Roles of the internal auditor 7 Delaware Performance Management - Belgium The Netherlands France United States Asia Pacific

8 Key learnings This case highlights the added value of integrating the strategy and risk management process. Risks can be identified and controlled by using the corporate strategy map as framework since strategy and risk management are two sides of the same coin; strategy management is about the bright future; where are we going to?;and risk management is about identifying, avoiding and overcoming the hurdles that the strategy may encounter along the way. Enterprise Risk Management is too much organized as a stand-alone process. ERM only focuses on the risks and has no attention for the company strategy. Strategy Focused Organizations use a more holistic approach: both strategy and risks are managed in relationship with each other. Furthermore, the case made clear that KPIs and KRIs should be identified separately. Strategy and risks are something completely different and hence should be measured differently. However KPIs, KRIs and strategic initiatives and risk mitigating initiatives should be linked to the corporate strategy map. Furthermore, the case illustrates how the risk management function can be integrated into the Office of Strategy Management. Besides its roles in developing, aligning and communicating the strategy and monitoring strategy the OSM function may also be responsible for coordinating, coaching and implementing the Enterprise Risk Management and make it an integrated part of the company s strategy and performance management process. Literature: Kaplan R.S., Risk Management and the Strategy Execution System, Balanced Scorecard Report, November-December Kaplan R.S. and A. Mikes, Managing the Multiple Dimensions of Risk: part I, Balanced Scorecard Report, July-August 2011, vol. 13, N 4. Mikes A. and Kaplan R.S., Managing the Multiple Dimensions of Risk Part II, Balanced Scorecard Report, September-October 2011, Vol. 13, N 5. About Delaware Performance Management Delaware Performance Management has a long experience in coaching companies in developing and implementing new strategies and driving organizational change. During their coaching projects We follow an integrated methodology and use the latest developments in the area of strategy and performance management. Delaware Performance Management is one of the founding partners of the Strategy Management Collaboratives (SMC), the applied research and consulting network in association with the Harvard Business School Professor Robert Kaplan and David Norton. In this knowledge network Delaware Performance Management serves as the Center of Excellence of Organizational Performance Management. 8 Delaware Performance Management - Belgium The Netherlands France United States Asia Pacific

Creating a Strategy-Focused Organization

Creating a Strategy-Focused Organization Creating a Strategy-Focused Organization Werner Bruggeman Valerie Decoene Geert Scheipers In recent years, organizations have sought to develop more comprehensive performance measurement systems to provide

More information

Strategic Risk Assessment. A first step for improving risk management and governance. COVER STORY. By Mark L. Frigo and Richard J.

Strategic Risk Assessment. A first step for improving risk management and governance. COVER STORY. By Mark L. Frigo and Richard J. Strategic Risk Assessment ILLUSTRATION: TIM LEE/WWW.LEEILLO.COM A first step for improving risk management and governance. By Mark L. Frigo and Richard J. Anderson December 2009 I STRATEGIC FINANCE 25

More information

Corporate Risk Management System Policy TransContainer OJSC

Corporate Risk Management System Policy TransContainer OJSC APPROVED by the Board of Directors TransСontainer OJSC (protocol No. ) as of 2013 Chairman of the Board of Directors TransContainer OJSC Z.B. Ryzhmanova Corporate Risk Management System Policy TransContainer

More information

Strategic Risk Management for School Board Trustees

Strategic Risk Management for School Board Trustees Strategic Management for School Board Trustees A Management Process Framework May, 2012 Table of Contents Introduction Page I. Purpose....................................... 3 II. Applicability and Scope............................

More information

WFP ENTERPRISE RISK MANAGEMENT POLICY

WFP ENTERPRISE RISK MANAGEMENT POLICY WFP ENTERPRISE RISK MANAGEMENT POLICY Informal Consultation 3 March 2015 World Food Programme Rome, Italy EXECUTIVE SUMMARY For many organizations, risk management is about minimizing the risk to achievement

More information

Integrating risk indicators into corporate performance management tool

Integrating risk indicators into corporate performance management tool Integrating risk indicators into corporate performance management tool Jelena Raid Swedbank Estonia Liivalaia 8, Tallinn, Estonia Abstract Tallinn Technical University Raja 15, Tallinn, Estonia In operational

More information

CRISC Glossary. Scope Note: Risk: Can also refer to the verification of the correctness of a piece of data

CRISC Glossary. Scope Note: Risk: Can also refer to the verification of the correctness of a piece of data CRISC Glossary Term Access control Access rights Application controls Asset Authentication The processes, rules and deployment mechanisms that control access to information systems, resources and physical

More information

Practice Guide COORDINATING RISK MANAGEMENT AND ASSURANCE

Practice Guide COORDINATING RISK MANAGEMENT AND ASSURANCE Practice Guide COORDINATING RISK MANAGEMENT AND ASSURANCE March 2012 Table of Contents Executive Summary... 1 Introduction... 1 Risk Management and Assurance (Assurance Services)... 1 Assurance Framework...

More information

STANDARDS OF SOUND BUSINESS AND FINANCIAL PRACTICES. ENTERPRISE RISK MANAGEMENT Framework

STANDARDS OF SOUND BUSINESS AND FINANCIAL PRACTICES. ENTERPRISE RISK MANAGEMENT Framework STANDARDS OF SOUND BUSINESS AND FINANCIAL PRACTICES ENTERPRISE RISK MANAGEMENT Framework September 2011 Notice This document is intended as a reference tool to assist Ontario credit unions to develop an

More information

Enterprise risk management: A pragmatic, four-phase implementation plan

Enterprise risk management: A pragmatic, four-phase implementation plan Enterprise risk management: A pragmatic, four-phase implementation plan Prepared by: John Brackett, Managing Director, Risk Advisory Services, RSM McGladrey, Inc. 704.442.3820, john.brackett@mcgladrey.com

More information

The College of New Jersey Enterprise Risk Management and Higher Education For Discussion Purposes Only January 2012

The College of New Jersey Enterprise Risk Management and Higher Education For Discussion Purposes Only January 2012 The College of New Jersey Enterprise Risk Management and Higher Education For Discussion Purposes Only Agenda Introduction Basic program components Recent trends in higher education risk management Why

More information

Designing a Metrics Dashboard for the Sales Organization By Mike Rose, Management Consultant.

Designing a Metrics Dashboard for the Sales Organization By Mike Rose, Management Consultant. Designing a Metrics Dashboard for the Sales Organization By Mike Rose, Management Consultant. Metrics can serve as critical measures of success for any organization and, in particular, the sales force

More information

THE INTERNATIONAL JOURNAL OF BUSINESS & MANAGEMENT

THE INTERNATIONAL JOURNAL OF BUSINESS & MANAGEMENT THE INTERNATIONAL JOURNAL OF BUSINESS & MANAGEMENT Performance Management Model for SMEs Rusaneanu Alexandra Ph.D. Student, Faculty of Cybernetics, Statistics and Economic Informatics, Bucharest University

More information

Operational Risk Management Program Version 1.0 October 2013

Operational Risk Management Program Version 1.0 October 2013 Introduction This module applies to Fannie Mae and Freddie Mac (collectively, the Enterprises), the Federal Home Loan Banks (FHLBanks), and the Office of Finance, (which for purposes of this module are

More information

The Balanced Scorecard (BSC)

The Balanced Scorecard (BSC) The Balanced Scorecard (BSC) Framework, Implementation Methodology and Recommended Application - Executive Brief (April 2012) - Introduction Balanced Scorecard is an integrated, organization-wide management

More information

Deriving Value from ORSA. Board Perspective

Deriving Value from ORSA. Board Perspective Deriving Value from ORSA Board Perspective April 2015 1 This paper has been produced by the Joint Own Risk Solvency Assessment (ORSA) Subcommittee of the Insurance Regulation Committee and the Enterprise

More information

OAC Presentation to UNESCO Member States

OAC Presentation to UNESCO Member States OAC Presentation to UNESCO Member States Scope and Purpose of Audit and Risk Committees 29 June 2016 1 Content: 1. Context 2. Audit and Risk Management in UNESCO today 3. Relationship between Entreprise

More information

Presentation Objectives Why is Internal Audit here? Concepts (Enterprise Risk Management, Strategic Risk, Strategic Risk Management, etc.

Presentation Objectives Why is Internal Audit here? Concepts (Enterprise Risk Management, Strategic Risk, Strategic Risk Management, etc. Internal Audit 1 January 13, 2012 Presentation Objectives Why is Internal Audit here? Concepts (Enterprise Risk Management, Strategic Risk, Strategic Risk Management, etc.) Summary Internal Audit 2 January

More information

IFAD Policy on Enterprise Risk Management

IFAD Policy on Enterprise Risk Management Document: EB 2008/94/R.4 Agenda: 5 Date: 6 August 2008 Distribution: Public Original: English E IFAD Policy on Enterprise Risk Management Executive Board Ninety-fourth Session Rome, 10-11 September 2008

More information

Integrated Risk Management:

Integrated Risk Management: Integrated Risk Management: A Framework for Fraser Health For further information contact: Integrated Risk Management Fraser Health Corporate Office 300, 10334 152A Street Surrey, BC V3R 8T4 Phone: (604)

More information

A Risk-Based Audit Strategy November 2006 Internal Audit Department

A Risk-Based Audit Strategy November 2006 Internal Audit Department Mental Health Mental Retardation Authority of Harris County ENTERPRISE RISK MANAGEMENT A Framework For Assessing, Evaluating And Measuring Our Agency s Risk A Risk-Based Audit Strategy November 2006 Internal

More information

Beyond risk identification Evolving provider ERM programs

Beyond risk identification Evolving provider ERM programs Beyond risk identification Evolving provider ERM programs March 2016 At a glance PwC conducted research to assess the state of enterprise risk management (ERM) within healthcare providers and found many

More information

Analyzing Risks in Healthcare. February 12, 2014

Analyzing Risks in Healthcare. February 12, 2014 Analyzing s in Healthcare February 12, 2014 1 Content What is Enterprise Management (ERM) ERM Benefits ERM Standards / ISO 31000:2009 ERM Process Register ERM Governance Model s Q&A 2 What is Enterprise

More information

Hand IN Hand: Balanced Scorecards

Hand IN Hand: Balanced Scorecards ANNUAL CONFERENCE T O P I C Risk Management WORKING Hand IN Hand: Balanced Scorecards AND Enterprise Risk Management B Y M ARK B EASLEY, CPA; A L C HEN; K AREN N UNEZ, CMA; AND L ORRAINE W RIGHT Recent

More information

Consulting Performance, Rewards & Talent. Making Employee Engagement Happen: Best Practices from Best Employers

Consulting Performance, Rewards & Talent. Making Employee Engagement Happen: Best Practices from Best Employers Consulting Performance, Rewards & Talent Making Employee Engagement Happen: Best Practices from Best Employers The Challenge Companies across the globe are taking the initiative to administer and manage

More information

Enterprise Risk Management

Enterprise Risk Management Enterprise Management ERM provides a framework for risk management, which typically involves identifying particular events or circumstances relevant to the organization's objectives (risks and opportunities),

More information

Administrative Guidelines on the Internal Control Framework and Internal Audit Standards

Administrative Guidelines on the Internal Control Framework and Internal Audit Standards Administrative Guidelines on the Internal Control Framework and Internal Audit Standards GCF/B.09/18 18 February 2015 Meeting of the Board 24 26 March 2015 Songdo, Republic of Korea Agenda item 24 Page

More information

Applying Risk Assessment to Your Audit Plan Break-out Session T3, Tuesday, October 26 2:00-2:50pm

Applying Risk Assessment to Your Audit Plan Break-out Session T3, Tuesday, October 26 2:00-2:50pm Applying Risk Assessment to Your Audit Plan Break-out Session T3, Tuesday, October 26 2:00-2:50pm Mike Brown Senior Vice President, Corporate Audit State Street Corporation Rich Reynolds Partner PricewaterhouseCoopers

More information

Policy and Procedure Statement

Policy and Procedure Statement Policy and Procedure Statement SUBJECT: Enterprise Risk CATEGORY: General Administration NO. 502-G PREAMBLE Risk exists in all activities and cannot be avoided, nor can it always be eliminated. However,

More information

POL ENTERPRISE RISK MANAGEMENT SC51. Executive Services Department BUSINESS UNIT: Executive Support Services SERVICE UNIT:

POL ENTERPRISE RISK MANAGEMENT SC51. Executive Services Department BUSINESS UNIT: Executive Support Services SERVICE UNIT: POL ENTERPRISE RISK MANAGEMENT SC51 POLICY CODE: SC51 DIRECTORATE: Executive Services Department BUSINESS UNIT: Executive Support Services SERVICE UNIT: Executive Support Services RESPONSIBLE OFFICER:

More information

Fraud Risk Management

Fraud Risk Management Fraud Risk Management Overview Discussion Questions 1) Does your organization follow a specific risk management model? If so, which one? Do you think this model adequately addresses the risks your organization

More information

Enterprise Risk Management

Enterprise Risk Management Cayman Islands Society of Professional Accountants Enterprise Risk Management March 19, 2015 Dr. Sandra B. Richtermeyer, CPA, CMA What is Risk Management? Risk management is a process, effected by an entity's

More information

THE SOUTH AFRICAN HERITAGE RESOURCES AGENCY ENTERPRISE RISK MANAGEMENT FRAMEWORK

THE SOUTH AFRICAN HERITAGE RESOURCES AGENCY ENTERPRISE RISK MANAGEMENT FRAMEWORK THE SOUTH AFRICAN HERITAGE RESOURCES AGENCY ENTERPRISE RISK MANAGEMENT FRAMEWORK ACCOUNTABLE SIGNATURE AUTHORISED for implementation SIGNATURE On behalf of Chief Executive Officer SAHRA Council Date Date

More information

A proven 5-step framework for managing supplier performance

A proven 5-step framework for managing supplier performance IBM Software Industry Solutions Industry/Product Identifier A proven 5-step framework for managing supplier performance Achieving proven 5-step spend framework visibility: benefits, for managing barriers,

More information

A To Do List to Improve Supply Chain Risk Management Capabilities

A To Do List to Improve Supply Chain Risk Management Capabilities A To Do List to Improve Supply Chain Risk Management Capabilities Investigators: Debra Elkins General Motors R&D Center Mailcode 480-106-359 30500 Mound Road Warren, MI 48090 Phone: 586-986-2750 FAX: 586-986-0574

More information

ENTERPRISE RISK MANAGEMENT FRAMEWORK WHAT IS ERM? JOIN. ENGAGE. LEAD.

ENTERPRISE RISK MANAGEMENT FRAMEWORK WHAT IS ERM? JOIN. ENGAGE. LEAD. ENTERPRISE RISK MANAGEMENT FRAMEWORK WHAT IS ERM? JOIN. ENGAGE. LEAD. Enterprise Risk Credit Risk Market Risk Operational Risk Regulatory Compliance Securities Lending INCREASED FOCUS ON ERM Although the

More information

ERM Program. Enterprise Risk Management Guideline

ERM Program. Enterprise Risk Management Guideline ERM Program Enterprise Management Guideline Table of Contents PREAMBLE... 2 When should I refer to this Guideline?... 3 Why do we need a Guideline?... 4 How do I use this Guideline?... 4 Who is responsible

More information

Saldanha Bay Municipality. Risk Management Strategy. Inclusive of, framework, procedures and methodology

Saldanha Bay Municipality. Risk Management Strategy. Inclusive of, framework, procedures and methodology Inclusive of, framework, procedures and methodology Contents 1 Introduction 1 1.1 Legislative Framework and best practice 1 1.2 Purpose of Enterprise Risk Management 2 1.3 Scope and Applicability 3 1.4

More information

Policy 10.105: Enterprise Risk Management Policy

Policy 10.105: Enterprise Risk Management Policy Name: Responsibility: Complements: Enterprise Risk Management Framework Coordinator, Enterprise Risk Management Policy 10.105: Enterprise Risk Management Policy Date: November 2006 Revision Date(s): January

More information

Keynote: How to Implement Corporate Performance Management (CPM), Pervasive BI & ROI: Hard & Soft

Keynote: How to Implement Corporate Performance Management (CPM), Pervasive BI & ROI: Hard & Soft Atre Group, Inc. Keynote: How to Implement Corporate Performance Management (CPM), Pervasive BI & ROI: Hard & Soft Shaku Atre Atre Group, Inc. 2222 East Cliff Drive, Suite#216 Santa Cruz, CA 95062 831.460.9300

More information

The New International Standard on the Practice of Risk Management A Comparison of ISO 31000:2009 and the COSO ERM Framework

The New International Standard on the Practice of Risk Management A Comparison of ISO 31000:2009 and the COSO ERM Framework The New International Standard on the Practice of Risk Management A Comparison of ISO 31000:2009 and the COSO ERM Framework Dorothy Gjerdrum, ARM-P, Chair of the ISO 31000 US TAG and Executive Director,

More information

Strategy Innovation. Strategic Results. Strategy Management Excellence. Strategy Execution

Strategy Innovation. Strategic Results. Strategy Management Excellence. Strategy Execution Strategy Innovation Strategic Results Strategy Management Excellence Strategy Execution Performance Management STRATEGY MANAGEMENT CERTIFICATION PROGRAM April 13 to 16, 2015 in Munich, Germany An international

More information

ENTERPRISE RISK MANAGEMENT FOR BANKS

ENTERPRISE RISK MANAGEMENT FOR BANKS ENTERPRISE RISK MANAGEMENT FOR BANKS Seshagiri Rao Vaidyula, Senior Manager, Governance, Risk and Compliance Jayaprakash Kavala, Consultant, Banking and Financial Services 1 www.wipro.com/industryresearch

More information

The Role of Internal Audit in Risk Governance

The Role of Internal Audit in Risk Governance The Role of Internal Audit in Risk Governance How Organizations Are Positioning the Internal Audit Function to Support Their Approach to Risk Management Executive summary Risk is inherent in running any

More information

Enterprise Risk Management

Enterprise Risk Management 2013 Government Accounting and Auditing Update Enterprise Risk Management Understanding and Implementing an ERM Framework Mike Sargent, Director- CliftonLarsonAllen May 2013 cliftonlarsonallen.com Discussion

More information

COBIT 5 for Risk. CS 3-7: Monday, July 6 4:00-5:00. Presented by: Nelson Gibbs CIA, CRMA, CISA, CISM, CGEIT, CRISC, CISSP ngibbs@pacbell.

COBIT 5 for Risk. CS 3-7: Monday, July 6 4:00-5:00. Presented by: Nelson Gibbs CIA, CRMA, CISA, CISM, CGEIT, CRISC, CISSP ngibbs@pacbell. COBIT 5 for Risk CS 3-7: Monday, July 6 4:00-5:00 Presented by: Nelson Gibbs CIA, CRMA, CISA, CISM, CGEIT, CRISC, CISSP ngibbs@pacbell.net Disclaimer of Use and Association Note: It is understood that

More information

Applying Integrated Risk Management Scenarios for Improving Enterprise Governance

Applying Integrated Risk Management Scenarios for Improving Enterprise Governance Applying Integrated Risk Management Scenarios for Improving Enterprise Governance János Ivanyos Trusted Business Partners Ltd, Budapest, Hungary, ivanyos@trusted.hu Abstract: The term of scenario is used

More information

Avondale College Limited Enterprise Risk Management Framework 2014 2017

Avondale College Limited Enterprise Risk Management Framework 2014 2017 Avondale College Limited Enterprise Risk Management Framework 2014 2017 President s message Risk management is part of our daily life, something we do regularly; often without realising we are doing it.

More information

FlyntGroup.com. Enterprise Risk Management and Business Impact Analysis: Understanding, Treating and Monitoring Risk

FlyntGroup.com. Enterprise Risk Management and Business Impact Analysis: Understanding, Treating and Monitoring Risk Enterprise Risk Management and Business Impact Analysis: Understanding, Treating and Monitoring Risk 2012 The Flynt Group, Inc., All Rights Reserved FlyntGroup.com Enterprise Risk Management and Business

More information

THE CONTRIBUTION OF THE BALANCED SCORECARD AS A STRATEGIC MANAGEMENT TOOL IN MANAGEMENT SUPPORT

THE CONTRIBUTION OF THE BALANCED SCORECARD AS A STRATEGIC MANAGEMENT TOOL IN MANAGEMENT SUPPORT Visión de Futuro Año 7, Nº1 Volumen Nº13, Enero - Junio 2010 URL de la Revista: www.fce.unam.edu.ar/revistacientifica/ URL del Documento: http://www.fce.unam.edu.ar/revistacientifica/index.php?option=com_content&view=article&id=184&itemid=51

More information

Maryland Association of Boards of Education Insurance Programs

Maryland Association of Boards of Education Insurance Programs Insurance Programs ENTERPRISE RISK MANAGEMENT John Magoon, ARM (P, E), CBCP, MBCI Risk Management Officer, MABE jmagoon@mabe.org 443 603 0399 A PERFECT DAY Our Goals 1.2 1 0.8 0.6 0.4 0.2 0 Actual Goal

More information

BALANCE SCORE CARD - A STRATEGIC PROJECT MANAGEMENT TOOL FOR INFRASTRUCTURE DEVELOPMENT PROJECTS

BALANCE SCORE CARD - A STRATEGIC PROJECT MANAGEMENT TOOL FOR INFRASTRUCTURE DEVELOPMENT PROJECTS BALANCE SCORE CARD - A STRATEGIC PROJECT MANAGEMENT TOOL FOR INFRASTRUCTURE DEVELOPMENT PROJECTS Eng. Udana Amarasuriya Department of Civil Engineering, University of Moratuwa, Sri Lanka; Associate Member,

More information

Successfully identifying, assessing and managing risks for stakeholders

Successfully identifying, assessing and managing risks for stakeholders Introduction Names like Enron, Worldcom, Barings Bank and Menu Foods are household names but unfortunately as examples of what can go wrong. With these recent high profile business failures, people have

More information

Disclaimer. www.diahome.org

Disclaimer. www.diahome.org Disclaimer The views and opinions expressed in the following PowerPoint slides are those of the individual presenter and should not be attributed to Drug Information Association, Inc. ( DIA ), its directors,

More information

Clarius Group Risk Management Policy and Framework

Clarius Group Risk Management Policy and Framework 1. Introduction Clarius Group Risk Management Policy and Framework 1.1 Definition Risk is the chance of something happening that will have an impact on objectives. Risk provides the opportunity (upside)

More information

Office of the Auditor General AUDIT OF IT GOVERNANCE. Tabled at Audit Committee March 12, 2015

Office of the Auditor General AUDIT OF IT GOVERNANCE. Tabled at Audit Committee March 12, 2015 Office of the Auditor General AUDIT OF IT GOVERNANCE Tabled at Audit Committee March 12, 2015 This page has intentionally been left blank Table of Contents Executive Summary... 1 Introduction... 1 Background...

More information

Enterprise Risk Management Panel Discussion

Enterprise Risk Management Panel Discussion Enterprise Risk Management Panel Discussion Facilitators Bill Cole, VCU and VCUHS CAE Michael Bordoni, former Emory University CAE, now DHG (Dixon Hughes Goodman LLP) Risk Advisory Services Partner Gary

More information

ENTERPRISE RISK MANAGEMENT ASSESSMENT GUIDE

ENTERPRISE RISK MANAGEMENT ASSESSMENT GUIDE ENTERPRISE RISK MANAGEMENT ASSESSMENT GUIDE WHITEPAPER CONTENTS CONTENTS INTRODUCTION 1 IS YOUR RISK MANAGEMENT PROCESS REALLY ASSESSING RISK? 1 IS YOUR RISK ASSESSMENT CONTEXT DRIVEN? 2 DOES YOUR RISK

More information

Sound Practices for the Management of Operational Risk

Sound Practices for the Management of Operational Risk 1 Sound Practices for the Management of Operational Risk Authority 1.1 Section 316 (4) of the International Business Corporations Act (IBC Act) requires the Commission to take any necessary action required

More information

Enterprise Risk Management in Colleges and Universities

Enterprise Risk Management in Colleges and Universities Enterprise Risk Management in Colleges and Universities Cherry Bekaert & Holland, L.L.P. Neal Beggan, CISA, CRISC Shane Hester, CPA, CISA Cherry, Bekaert & Holland, L.L.P. The Firm of Choice. 1 Cherry,

More information

POLICY. Number: 7311-10-005 Title: Enterprise Risk Management. Authorization

POLICY. Number: 7311-10-005 Title: Enterprise Risk Management. Authorization POLICY Number: 7311-10-005 Title: Enterprise Risk Management Authorization [ ] President and CEO [ X] Vice President, Finance and Corporate Services Source: Director, Enterprise Risk Management Cross Index:

More information

Designing Sales Management s Dashboard: Integrating the Balanced Scorecard into Sales Performance Management February 2008

Designing Sales Management s Dashboard: Integrating the Balanced Scorecard into Sales Performance Management February 2008 RESEARCH BRIEF Designing Sales Management s Dashboard: Integrating the Balanced Scorecard into Sales Performance Management February 2008 Michael Rose, Ph. D. The Sales Management Association +1 312 278-3356

More information

How To Use Risk It

How To Use Risk It Risk IT A set of guiding principles and the first framework to help enterprises identify, govern and effectively manage IT risk. In business today, risk plays a critical role. Almost every business decision

More information

www.pwc.com ERM006 ERM and Business Continuity Management: Together at Last RIMS Annual Conference April 13, 2016

www.pwc.com ERM006 ERM and Business Continuity Management: Together at Last RIMS Annual Conference April 13, 2016 www.pwc.com ERM006 ERM and Business Continuity Management: Together at Last RIMS Annual Conference April 13, 2016 Your presenters Phil Samson Principal PricewaterhouseCoopers, Dallas Leads s Risk Management

More information

STRATEGIC PLANNING AND BALANCED SCORECARD AT PLIVA

STRATEGIC PLANNING AND BALANCED SCORECARD AT PLIVA Mislav Vučić PLIVA d.d., Strategic Planning Office mislav.vucic@pliva.hr STRATEGIC PLANNING AND BALANCED SCORECARD AT PLIVA How and for what purpose PLIVA uses Balanced ScoreCard Abstract: The strategic

More information

White Paper. Change Management: Driving the Long-Term Success of Your Workforce Management Solution

White Paper. Change Management: Driving the Long-Term Success of Your Workforce Management Solution White Paper Change Management: Driving the Long-Term Success of Your Workforce Management Solution How Do You Measure the Success of a Technology Project? When your organization embarks on a technology

More information

Risk appetite. The strategic balancing act

Risk appetite. The strategic balancing act Risk appetite The strategic balancing act Designing risk management without defining your risk appetite is like designing a bridge without knowing which river it needs to span. Your bridge will be too

More information

UNITED STATES DEPARTMENT OF EDUCATION OFFICE OF INSPECTOR GENERAL

UNITED STATES DEPARTMENT OF EDUCATION OFFICE OF INSPECTOR GENERAL UNITED STATES DEPARTMENT OF EDUCATION OFFICE OF INSPECTOR GENERAL Evaluation and Inspection Services Memorandum May 5, 2009 TO: FROM: SUBJECT: James Manning Acting Chief Operating Officer Federal Student

More information

Formulating and Implementing an HP IT program strategy using CobiT and HP ITSM

Formulating and Implementing an HP IT program strategy using CobiT and HP ITSM Formulating and Implementing an HP IT program strategy using CobiT and HP ITSM Mathias Sallé HP Research Laboratories mathias.salle@hp.com Steve Rosenthal Management Software Organization steve.rosenthal@hp.com

More information

Getting to strong Leading Practices for value-enhancing internal audit By Richard Reynolds and Abhinav Aggarwal - PricewaterhouseCoopers LLP

Getting to strong Leading Practices for value-enhancing internal audit By Richard Reynolds and Abhinav Aggarwal - PricewaterhouseCoopers LLP Getting to strong Leading Practices for value-enhancing internal audit By Richard Reynolds and Abhinav Aggarwal - PricewaterhouseCoopers LLP Today's unpredictable business climate and challenging regulatory

More information

The Balanced Scorecard. Background Discussion

The Balanced Scorecard. Background Discussion The Balanced Scorecard Background Discussion Contents History and Evolution Important Business Drivers Key Concepts Case Studies & Success Stories 1 Business Intelligence (BI) and Knowledge Management

More information

PERFORMANCE MANAGEMENT METHOD FOR CONSTRUCTION COMPANIES

PERFORMANCE MANAGEMENT METHOD FOR CONSTRUCTION COMPANIES 24th International Symposium on on Automation & Robotics in in Construction (ISARC 2007) Construction Automation Group, I.I.T. Madras PERFORMANCE MANAGEMENT METHOD FOR CONSTRUCTION COMPANIES Namho Kim

More information

Governance and Risk Management in the Public Sector. Fernando A. Fernandez Inter-American Development Bank (202) 623-1430 e-mail: fernandof@iadb.

Governance and Risk Management in the Public Sector. Fernando A. Fernandez Inter-American Development Bank (202) 623-1430 e-mail: fernandof@iadb. Governance and Risk Management in the Public Sector Fernando A. Fernandez Inter-American Development Bank (202) 623-1430 e-mail: fernandof@iadb.org 1 Agenda Governance, why is it important? Compliance

More information

Safety Metrics, The Big Picture -Terry L. Mathis President, ProAct Safety

Safety Metrics, The Big Picture -Terry L. Mathis President, ProAct Safety Safety Metrics, The Big Picture -Terry L. Mathis President, ProAct Safety What is the best measure of safety performance? Is it the traditional recordable rate, severity rate, cost of accidents, near miss

More information

Operational Risk Management in a Debt Management Office

Operational Risk Management in a Debt Management Office Operational Risk Management in a Debt Management Office Based on Client Presentation January 2008 Outline The importance of operational risk management (ORM) International best practice A high-level perspective,

More information

The Changing Landscape for Trade Compliance Enterprise Risk (and Opportunity) Management

The Changing Landscape for Trade Compliance Enterprise Risk (and Opportunity) Management The Changing Landscape for Trade Compliance Enterprise Risk (and Opportunity) Management API International Trade and Customs Conference H. Michael Leightman, Partner Customs and International Trade Practice

More information

The Demise of Cost and Profit Centers

The Demise of Cost and Profit Centers 07-030 The Demise of Cost and Profit Centers Robert S. Kaplan Copyright 2006 by Robert S. Kaplan Working papers are in draft form. This working paper is distributed for purposes of comment and discussion

More information

LONDON Operation Excellence Dashboard Metrics and Processes

LONDON Operation Excellence Dashboard Metrics and Processes LONDON Operation Excellence Dashboard Metrics and Processes Wednesday, June 25, 2014 08:30 to 09:30 ICANN London, England CAROLE CORNELL: Okay. I m sorry. Let s begin. I m going to play with this as I

More information

COMPARATIVE STUDY BETWEEN TRADITIONAL AND ENTERPRISE RISK MANAGEMENT A THEORETICAL APPROACH

COMPARATIVE STUDY BETWEEN TRADITIONAL AND ENTERPRISE RISK MANAGEMENT A THEORETICAL APPROACH COMPARATIVE STUDY BETWEEN TRADITIONAL AND ENTERPRISE RISK MANAGEMENT A THEORETICAL APPROACH Cican Simona-Iulia Management, Faculty of Economics and Business Administration, West University of Timisoara,

More information

fs viewpoint www.pwc.com/fsi

fs viewpoint www.pwc.com/fsi fs viewpoint www.pwc.com/fsi June 2013 02 11 16 21 24 Point of view Competitive intelligence A framework for response How PwC can help Appendix It takes two to tango: Managing technology risk is now a

More information

The Balanced Scorecard and Corporate Social Responsibility: Aligning Values for Profit

The Balanced Scorecard and Corporate Social Responsibility: Aligning Values for Profit Balanced Scorecard and CSR -------------------------------------------------------------- This story was printed from News, located at http://www.greenbiz.com/news/. --------------------------------------------------------------

More information

The Search for a Holy Grail: Best Practices for Defining KPIs

The Search for a Holy Grail: Best Practices for Defining KPIs The Search for a Holy Grail: Best Practices for Defining KPIs (Selected Slides) Kent Bauer Partner and Managing Director The Performance Group Data Warehousing & BI Conference Orlando, Florida September

More information

In accordance with risk management best practices, below describes the standard process for enterprise risk management (ERM), including:

In accordance with risk management best practices, below describes the standard process for enterprise risk management (ERM), including: Enterprise Risk Management Process and Procedures Scope In accordance with risk management best practices, below describes the standard process for enterprise risk management (ERM), including: Risk identification

More information

Board oversight of risk: Defining risk appetite in plain English

Board oversight of risk: Defining risk appetite in plain English www.pwc.com/us/centerforboardgovernance Board oversight of risk: Defining risk appetite in plain English May 2014 Defining risk appetite in plain English Risk oversight continues to be top-of-mind for

More information

UNITED STATES DEPARTMENT OF EDUCATION OFFICE OF INSPECTOR GENERAL

UNITED STATES DEPARTMENT OF EDUCATION OFFICE OF INSPECTOR GENERAL UNITED STATES DEPARTMENT OF EDUCATION OFFICE OF INSPECTOR GENERAL AUDIT SERVICES August 24, 2015 Control Number ED-OIG/A04N0004 James W. Runcie Chief Operating Officer U.S. Department of Education Federal

More information

Using COSO Small Business Guidance for Assessing Internal Financial Controls

Using COSO Small Business Guidance for Assessing Internal Financial Controls Using COSO Small Business Guidance for Assessing Internal Financial Controls By János Ivanyos, Memolux Ltd. (H), IIA Hungary Introduction New generation of general models referring to either IT or Internal

More information

How To Transform It Risk Management

How To Transform It Risk Management The transformation of IT Risk Management kpmg.com The transformation of IT Risk Management The role of IT Risk Management Scope of IT risk management Examples of IT risk areas of focus How KPMG can help

More information

RiskManagement ESIEE 06/03/2012. Aloysius John March 2012

RiskManagement ESIEE 06/03/2012. Aloysius John March 2012 RiskManagement MOTIS ESIEE 06/03/2012 Aloysius John March 2012 Risk Management is a Introduction Process for Project manager to identify factors that may more or less affect the success or the achievement

More information

ENTERPRISE RISK MANAGEMENT POLICY

ENTERPRISE RISK MANAGEMENT POLICY ENTERPRISE RISK MANAGEMENT Approved by the Audit Committee on 14 February 2003 and adopted by resolution of the Board on 28 March 2003 Revisions approved by the Audit and Risk Committee on 14 February

More information

POLICY : CORPORATE RISK MANAGEMENT

POLICY : CORPORATE RISK MANAGEMENT APPENDIX 5 POLICY : CORPORATE RISK MANAGEMENT 1 Scope This is a Service wide policy. 2 Aims and Objectives Lancashire Combined Fire Authority provides services to a diverse range of people and organisations,

More information

Risk Assessment & Enterprise Risk Management

Risk Assessment & Enterprise Risk Management Risk Assessment & Enterprise Risk 1 Healthcare Corporate Governance Today s environment requires building a culture of risk awareness and management of risk across the organization, while formulating less

More information

Risk Management Policy Adopted by:

Risk Management Policy Adopted by: Risk Management Policy Adopted by: Infigen Energy Limited Infigen Energy (Bermuda) Limited Infigen Energy RE Limited in its capacity as Responsible Entity of Infigen Energy Trust Adopted: 17 December 2009

More information

fmswhitepaper Why community-based financial institutions should practice enterprise risk management.

fmswhitepaper Why community-based financial institutions should practice enterprise risk management. fmswhitepaper Why community-based financial institutions should practice enterprise risk management. By Michael D. Cohn, CPA, CISA, CGEIT Director, WolfPAC Solutions Group Unique Insights Implementation

More information

Risk Management Policy

Risk Management Policy Risk Management Policy June 2015 1 2 Contents 1. Policy Objectives and Background... 4 1.1. Policy Background... 4 1.2. Policy Objective... 4 1.3. Policy Sponsor and Maintenance... 4 2. Risk Types and

More information

Enterprise Risk Management Update Executive Summary December 2010

Enterprise Risk Management Update Executive Summary December 2010 Enterprise Risk Management Update Executive Summary December 2010 Risk is integral in the pursuit of improvement. Risk, in general, is seldom avoidable and cannot always be mitigated. Accordingly, risk

More information

Internal Auditing Guidelines

Internal Auditing Guidelines Internal Auditing Guidelines Recommendations on Internal Auditing for Lottery Operators Issued by the WLA Security and Risk Management Committee V1.0, March 2007 The WLA Internal Auditing Guidelines may

More information

Risk Management Policy and Process Guide

Risk Management Policy and Process Guide Risk Management Policy and Process Guide Status: pending Next review date: December 2015 Page 1 Information Reader Box Directorate Medical Nursing Patients & Information Commissioning Operations (including

More information

Enterprise Risk Management Integrated Framework. Executive Summary

Enterprise Risk Management Integrated Framework. Executive Summary Enterprise Risk Management Integrated Framework Executive Summary September 2004 Copyright 2004 by the Committee of Sponsoring Organizations of the Treadway Commission. All rights reserved. You are hereby

More information

ACG s Growth Strategy and High Performance Business Consulting Services

ACG s Growth Strategy and High Performance Business Consulting Services ACG s Growth Strategy and High Performance Business Consulting Services ACG delivers telecom market share/forecast reports, consulting services, business case analysis, product and service message testing.

More information