HOW TO MODEL AND EVALUATE THE INCIDENT REPORTING PROCESS OF A COMPANY?

HOW TO MODEL AND EVALUATE THE INCIDENT REPORTING PROCESS OF A COMPANY? Marinka Lanne and Kaarin Ruuhilehto VTT Technical Research Centre of Finland, P.O. Box 1300, FI-33101 Tampere, Finland; Tel.: þ358 20 722 3633, Fax: þ358 20 722 3499, e-mail: marinka.lanne@vtt.fi, kaarin.ruuhilehto@vtt.fi Companies need to control several different sectors of corporate safety and security including, for example, occupational health and safety, environmental safety, premises security, crime prevention, rescue operations and emergency planning, information security, and personnel security. All these sectors contribute towards the total corporate safety and security of a company. By collecting information about different types of incidents, a company can learn about how to prevent future incidents, and thus make the company safer. Incidents, as referred to here, include accidents, near-misses, and other deviations such as hazardous situations. A company can internally collect information about those event chains that can lead to accidents. The time and place, the object and consequences, and causes and preventive and corrective actions, are all examples of important information collected about incidents. Even if a company recognizes the importance of preventive data, it might be difficult to organize the collection, handling, and utilization of this information. Computer software is nowadays essential, but it can not address all the aspects associated with the incident reporting process. The organization of the incident reporting process was investigated at VTT using three case studies during 2005-2006. Key questions of the study were: 1) how to model all the parts of the incident reporting process, and 2) how to evaluate the quality of incident reporting process. All the actions, the different actors, and the different documents and information flow relating to the incident reporting process were modelled. The important phases of the process were: a) identifying an incident and acting immediately (understanding hazards), b) informing the supervisor about the incident (motivational factors), c) handling the incident reports (systematic documentation), d) investigating the incident (understanding analysis frameworks), e) decision making about solutions and improvements by utilizing the collected data, f) implementing and monitoring the corrective and preventive actions (reacting), and g) observing the general view and evaluating the incident reporting process. The study contained three case companies: an oil refinery, a chemical factory, and an amusement park. Three incident reporting processes were modelled with the data collected from the companies. The three case models were used to develop an evaluation tool for the incident reporting process. This paper discusses the modelling and evaluation of the incident reporting process. The main purpose is to show that modelling of the incident reporting process gives an opportunity for a company to evaluate and improve the quality of the incident reporting process. Based on the research carried out, principles and ideas for assessing the different phases of the incident reporting process are presented. KEYWORDS: incident reporting, near-miss reporting, incident analysis INTRODUCTION This article concerns incident reporting process and system at the organizational level. An incident reporting system comprises disclosing and analysing the incident, and also using information from previous incidents to improve corporate safety and security. The system includes incidents related to all sectors of the corporate safety and security: for example, occupational, environmental safety, production process safety, premises security, crime prevention, fire safety, and information security. In this context incident can be defined as an unusual or unexpected event, which either resulted in, or had the potential to result in: injury to personnel, significant damage to property, adverse environmental impact, or a major interruption of process operations. This definition is expressed in the Incident Investigation Guidelines book of the Center for Chemical Process Safety (Philley et al. 2003). An incident can be either an accident or a near miss. An accident is an incident which poses harmful consequences affecting people, the environment, property, or the operation process. A near miss is an incident which probably would have posed harmful consequences if the circumstances had been slightly different. It has been demonstrated in various studies (e.g. Bird and Germain 1966; Heinrich et al. 1980) that there is a relationship between the number of near misses, with minor incidents and major accidents. Various iceberg 1

models show that for each major accident there are a large number of minor accidents, and even more near misses. Many industrial companies already recognize that they can learn from their near misses (Jones et al. 1999). Nowadays, accidents themselves are too few in number to be able to obtain enough information about their causes. For example, Jones et al. (1999) have demonstrated, by way of a practical example from industry, that there is an inverse proportionality between the number of reported near misses and the number of accidents, and they assert that the rate of near miss reports is an important numerical indicator of industry s safety awareness. For learning from incidents, it is important to investigate and understand how incidents arise. The entire incident reporting process and system must be planned carefully, as several issues affect the functioning of the reporting process. At first, the organization needs to ensure that all the employees both white and blue collar understand and identify incidents. A major challenge for the organization is to motivate people to report their observations to supervisors, or directly to an incident reporting system. Know-how about the incident reporting system and also motivational issues affect the reporting activity. Van der Schaaf and Kanse (2004) have grouped the demotivating factors influencing incident reporting as follows: 1) fear of disciplinary action or of other people s reactions, 2) uselessness, 3) acceptance of risk, and 4) practical reasons (as too time consuming or difficult to submit a report). Indemnity against disciplinary proceedings, confidentiality or de-identification, and rapid, useful, accessible, and intelligible feedback are important factors associated with the reporting culture, according to Reason (1997). The adopted theory or model about accidents and their causes characterizes of the reporting process. Accident models focused on human behaviour in the control of danger and on safety management systems are pertinent frameworks with respect to learning from incidents. (Koornneef, 2000) Lucas (1991) also stresses the importance of the organization s model of why humans make mistakes, as part of the overall safety culture. Although many research studies about incident reporting have been performed, precious few of them focus on process modelling and evaluating the process quality. The evaluation of the entire incident reporting process provides the organization with important data about the process quality. Systematic methods and regular checking of the reporting process are required for the evaluation. The evaluation aims at continuous improvement. According to Oktem (2003), auditing a nearmiss system s performance is the first and the most critical step in understanding what is working well and what needs to be improved. This paper discusses the modelling and evaluation of the incident reporting process. The main purpose is to show that modelling of the incident reporting process gives an opportunity for a company to evaluate and improve the quality of the incident reporting process. From the national perspectives, the main purpose of the study was to create a simple tool for organizations to plan, evaluate and improve their incident reporting process. OBJECTS AND METHODS The objective of the study was to define the phases of the incident reporting process, and then to create a self-audit tool for evaluating the process phase by phase. The study started in the autumn of 2005 and ended in December 2006. At first, a literature review of incident reporting procedures was performed. Interviews and workshops at the three participating case-study companies were used to create a model of their reporting processes and gain information on the motivating (and demotivating) factors associated with reporting culture within the organizations. Finally, a phase model of incident reporting process was generated, upon which the guidelines and self-audit tool for incident reporting were created. The research work was carried out at VTT, the Technical Research Centre of Finland. The three case studies were from the participating companies: an oil refinery, a chemical factory, and an amusement park. The main funding was received from The Finnish Work Environment Fund, and the Ministry of Social Affairs and Health. The workshops for modelling the incident reporting processes in the case companies were carried out in groups of 6 11 persons. A two-day workshop was held in each company, and the participants consisted of upper management, line management, employees, and safety and security professionals. In the workshop sessions, the actual incident reporting processes were described on large sheets of paper several flip-over pads. The following questions were asked by the research scientists: What are the basic functions of your incident reporting process? What needs to be done and by whom? Which tools are used in the process? Which documents are to be made? What is the information, and how does it flow during the process? After the workshop sessions, the researchers computerized the described process models. A general model of the incident reporting process was created as a combination of the case models and the relevant literature. The self-audit tool was then created, based on the general model. A new project has been proposed that plans to test the tool with new case organizations during the year 2007. Furthermore, there were additional interviews and development visits to the companies. In one of the case companies an employee interview for 16 persons was arranged, as was an additional group work meeting for over 50 persons (mostly workers and line management). The group work meeting and the interviews focused on completing the incident reporting form, and especially the underlying motivations for what incidents were reported. RESULTS The main results of the study were the phase model of the incident reporting process and the self-audit tool for evaluating the process. The following list presents the phase 2

model of the incident reporting system developed in the study. It provides short descriptions of the basic phases and important operations connected to these phases:. Detection of an incident: the phase when somebody recognizes an incident. Especially important are: understanding of hazards; and also methods and training to identify incidents.. Acting immediately: the phase when the identifier tries to control the situation and limit the harmful consequences. Guidelines and training on how to act in an incident situation are especially important.. Recording the incident: the phase when the identifier completes the reporting form and/or informs the supervisor about the incident. Especially important are: methods and tools for informing about the incident (reporting system); and motivational factors.. Receiving and handling the incident report: the phase when the appointed person receives the incident report and processes it. Especially important are: systematic documentation about the incident; classification system; evaluation of the investigation requirements; identifier feedback; and the wider dissemination of the information.. Investigating the incident: the phase when the causes of the incident are systematically determined. Especially important are: an understanding of analysis frameworks, and the methods, tools and training for the investigation are especially important.. Data analysis and utilizing the collected data: the phase in which all the data collected from different incidents are dissected and analysed (statistically), and the incorporated information is used for decision making. Especially important are: analysis methods and tools; databases and applications; the reliability and validity of the data and the analysis; and the versatility of utilization.. Decision-making about solutions and improvements: the phase when the identifier and investigators suggested improvements are evaluated and implementation decisions are made.. Implementing and monitoring changes: the phase when the actual implementation process is planned, executed and evaluated. Especially important are: a risk assessment of the action plan; and a comparison of the intended and actual results of action.. Evaluating and improving the entire incident reporting process: the overall procedure whereby the process is evaluated and improved systematically, phase by phase. Especially important are: the functionality of the process; evaluation of the objectives; objective setting; and flow of information; and responsibilities, abilities and skills. According to the study, the greatest challenges of the incident reporting process are related to the investigation phase, the utilization of information, and also to the evaluation of the effects of implemented operations. There can be different levels of investigation: basic level where one trained supervisor evaluates the incident factors by interviewing the identifier and by combining different data, and expanded level where an investigation group models and evaluates the incident factors by interviewing the incident identifier and by combining different data. The basic investigation is the minimum level and it generates data for statistical analysis. The expanded investigation becomes valid when the incident has had the potential to pose significant harmful consequences. Consequences can be evaluated with the aid of the typical risk matrix. A more detailed investigation on the expanded level has to be performed when the incident involves a complex chain of events or multiple causes. In the case companies, the EHS team (environment, health and safety experts) has a significant role in incident investigations. The team brings not only expertise and an analytical approach to investigation, but at the same time it removes the line management from the learning process. When the number of incident reports increases, it becomes difficult to handle all the investigations with so few experts. Hence, there is a need to shift the responsibility for basic level investigations to the line management. A big challenge is to ensure that every line manager adopts coherent conceptual basics about the origin of an incident, and that they have adequate skills to investigate the incident. EHS experts have and will continue to have an important role in the expanded level investigations, and in the investigation quality evaluation of the basic level investigations. Database and incident reporting applications have an important role in the incident reporting process because they enable rapid data analyses. All the case companies involved in the study were implementing new applications or updating the old ones during the study. There was, however, no clear common understanding and concept about incident models, and therefore the application providers brought their own models to the companies. Hence, the application providers made decisions that the companies should have done. The biggest challenge with these software-related processes was to get the application to serve the incident process and to avoid a situation in which the application and creation of the database overshadows the investigations and actions. For example, classification scales must be coherent with the conceptual basics. Moreover, the basic objective of learning lessons to improve system performance demands information and communication that is, people getting together regularly to discuss the gathered data. The self-audit tool for evaluating the incident reporting process helps the organization perform a systematic evaluation. The tool consists of the evaluation frame (Figure 1) and the associated checklists. The description of the incident reporting process is an essential part of the evaluation. Evaluation and improvement are carried out by following the steps listed below: 1. Describe the incident reporting process of the company. Record the basic phases and functions. Explain why 3

Figure 1. The self-audit tool for evaluating the incident reporting process these functions are done and who is responsible for completing them. Consider what methods and tools are connected to the process, and how the information flows. The evaluation frame helps to identify the phases of the process. 2. Evaluate which phases need improvement, using the evaluation frame. 3. Check the improvement needs in more detail. Use the checklist. 4. Make a plan for implementing the suggested improvements. 5. Ensure that the required decisions are made in the normal decision-making process of the company. 6. Implement the changes. 7. Incorporate a new evaluation into the general auditing plan of the company DISCUSSION The goal of the incident reporting process is to learn from errors and improve performance. It is important to keep the focus on learning and improving when the objectives of incident reporting are made. Quantitative objectives are associated with, for example, the number of reported incident or incidents investigated. It can also be related to the number of suggested improvement operations, and/or to the speed of processing. Qualitative objectives can be bound to the content of the incident report, to investigation methods, to the quality of improvement suggestions, to the influence of the improvement operation, and to the reliability and validity of the collected data. Consequently, there are many other factors than number of incident reports that affect the quality of the incident reporting process. 4

During the study, a discussion on a need to link the incident reporting process and the risk assessment process arose. Harms-Ringdahl [2004] recognizes a need for intensifying the flow of information between accident investigation and risk assessment, which supports this idea. In this study, the incident reporting process and the risk assessment process were linked by using the same classification titles for describing the event types and by defining the consequences with the same risk matrix. Incident data was also used to update the results of the risk analyses and to update the checklists that are used in risk assessment. Systematic evaluation of the incident reporting process provides the organization with important data about the quality of the process. The evaluation frame and the checklist created within this study can help organizations identify not only those parts of the process that are working well, but more importantly, those which need improvement. The study indicates that modelling the incident reporting process helps the organizations to better understand the process and highlight ideas for improvement. It is important to involve different points of view to the evaluation process. The identifiers and declarers, investigators, decision-makers and those who utilize all the associated information may have different experiences with the system. The evaluation of the reporting system, as such, motivates the personnel to continuous improving. The self-audit tool developed in this study to assist the evaluation provides a framework and important check points to evaluation discussions. REFERENCES Bird, F. E. and Germain, G. L., 1966. Damage Control, New York, American Management Assoc. Inc. Harms-Ringdahl, L., 2004. Relationships between accident investigations, risk analysis, and safety management, Journal of Hazardous Materials 111: 13 19. Heinrich, H. W., Petersen, D. and Roos, N., 1980. Industrial Accident Prevention: A Safety Management Approach, New York, McGraw-Hill. Jones, S., Kirchsteiger, C. and Bjerke, W., 1999. The importance of near miss reporting to further improve safety performance, Journal of Loss Prevention in the Process Industries, 12: 59 67. Lucas, D.A., 1991. Organisational Aspects of Near Miss Reporting. In the book: van der Schaaf, T.W., Lucas, D.A. and Hale, A.R. (eds), Near Miss Reporting as a Safety Tool, Oxford, Butterworth-Heinemann Ltd, 127 136. Philley, J., Pearson, K. and Sepeda, A., 2003. Updated CCPS Investigation Guidelines book, Journal of Hazardous Materials, 104: 137 147. Reason, J., 1997, Managing the Risk of Organisational Accidents, Hampshire, Ashgate Publishing Ltd. van der Schaaf, T. and Kanse, L., 2004. Checking for biases in Incident Reporting. In the book: Phimister, J.R., Bier, V.M. and Kunreuther, H.C, Accident Precursor Analysis and Management: Reducing Technological Risk Through Diligence, 119 126. 5