Integrated Data Management for handling hazard of change situations: a sample case of operational implementation

Transcription

1 Integrated Data Management for handling hazard of change situations: a sample case of operational implementation M. Cassani a*, V. Licata a, D. Baranzini b, S. Corrigan b, E. De Grandis c, A. Ottomaniello c, a KITE Solutions, Laveno Mombello (VA), Italy b Trinity College Dublin, APRG-School of Psychology, Dublin, Ireland c Air Dolomiti, Safety Department, Dossobuono di Villafranca, Verona, Italy Abstract: The ability to manage organizational change is crucial to ensure a permanent and high level of performance of safety critical organizations. There is the need to go beyond implementation of the basic concepts and methodological approaches of Preliminary and System Hazard Assessment coupled with sound Human Factors analysis. Changes are the prototypical case of need of such a deeper analysis that becomes consequently a prospective type of evaluation of risk. In addition, the Company management attitude towards the implementation of a prospective assessment to the risks associated with relevant changes within an organization is equally necessary. All these aspects are considered part of a classical safety assessment method. A number of innovative elements have to be developed in order to adequately account for changes. In this perspective the theoretical framework that sustains this approach is named LIFE (Lean Integrated Forecast Estimator). In brief, LIFE is a prospective change assessment method (evolution of prospective safety approaches) that will verify and model the impact of future change events that will be affecting a system and will EU funded MASCA Project. In this paper, two major aspects are discussed: 1) the process of implementation of the LIFE process of analyzing the risk associated with change within an organization, by means of a support tool that guides the analyst in applying the methodology; and 2) a case study of operation implementation that considers a relevant change in operations planned by an air carrier and the preliminary stages of implementation of the safety assessment of the change, according to the proposed novel approach. Keywords: Aviation Safety, Safety Management System, Management of Change, Human Factors 1. INTRODUCTION Aviation organizations experience permanent changes due to expansion, contraction, introduction of new equipment, new procedures. In particular, commercial competition, driven partly by the low-cost business model, is driving aviation organizations globally to change the way in which they do business, cutting costs and developing a leaner enterprise. The low-cost carriers, relatively new entrants to the business, grew their companies around this business model. Older organizations (including the legacy carriers ) do not have the opportunity to build their organization from scratch therefore they have to change an established stable system with its strong cultural and institutional supports, without compromising core operational goals. This change can take different forms including downsizing, consolidation, bankruptcy and new business formation. At the same time, in the operational sector, new requirements for regulation coming from ICAO (2009) demand that all aviation organizations develop a safety management system, requiring transparency in the development of internal organizational processes. The requirements for safety management require a proactive strategic approach, anticipating risks and demonstrating a capacity to keep safety at the centre of change that is driven by commercial competition, and ensuring that safety evidence itself becomes an effective driver of change, even in the ultra-safe system that aviation has become. Nevertheless, organizations find it difficult to integrate their different functional units in a common programme of change; there is no clear consensus about what it means to be proactive ; there is no integrated framework for managing all the human related functions in an operational system; influential change programmes like lean enterprise and six sigma have no systemic methodologies for managing complex human functions in systems. Embedding the Lean way of working into the culture can take several years and requires constant support and guidance from management.

2 Changes can introduce new hazards, may impact the appropriateness and effectiveness of risk mitigation And therefore an objective of change is to reduce the risk the organization faces in meeting its strategic challenges, comparing current performance to projected demands. Therefore it falls back into highlighting the need to support a culture of change and change management which is an issue Human and Organizational factors approach have often confronted in the past. 2. THE LIFE APPROACH 2.1 The LIFE concept A number of innovative elements have to be developed in order to adequately account for changes. The change analysts has to evaluate and estimate the degree of safety that may be affected by the change under study; the ability or degree of resilience of the organization in order to adapt to the aspects that may not be imagined by the safety analyst a-priory and may still result from the change, etc. The theoretical framework that sustains this approach is named LIFE (Lean Integrated Forecast Estimator). In brief, LIFE is a prospective change assessment method (evolution of prospective safety approaches) that will verify and model the impact of future change events that will be affecting a system and will generate change solutions accordingly. 2.2 The LIFE method in brief The LIFE is a step by step procedure dedicated to the analysis of future states, or changes, under with a company or organization might be exposed to. From this analysis the most likely hazards potentially generated by such upcoming changes will be prioritized, shortlisted and assessed for their prevention of minimization. Within this rationale the following key steps are considered and described in Figure 1 below. Figure 1 The LIFE process In general a model of a change, called Expected Model of Change (EMC) (Figure 1) has to be developed. It defines the type of upcoming type Change from an operational or business point of view. The EMC will be studied via dedicated workshops to identify the type of potential system related hazards triggered (directly or indirectly caused) by the occurrence of the EMC interfacing somehow with the company or organization. If such hazards, called EMC hazards, will be shortlisted as serious threat, then a dedicated simulation exercise will be generated to verify hazard reductions in terms of frequency and/or severity (as shown by the Step 6 in

3 Figure 1). From this assessment and simulation of future risk states (determined by the expected change), a pool of requirements will be generated and deployed to anticipate a set of possible responses if the EMC will become reality. In this perspective a company or organization time to adapt to the change is reduced by anticipating a number of pre-set response that will ensure more organizational resiliency and capability. An organizational readiness to anticipated changes is in place. Such new capability is called anticipatory resilience. Overall, such theoretical framework would give leverage to any change oriented risk analysis. In particular a case study will be presented here below. 2.2 LIFE Step-by-Step The LIFE methodology accounts for a number of prospective change activities typically described in an overall approach of safety assessment of change management (McDonald et al, 2012). Such activities are grouped into a number of steps. Table 1 below describes the basic key activities per each of the 10 steps of the LIFE operation. Table 1. Basic activities of the LIFE operation LIFE Steps Key actions in LIFE Competency requirements STEP 1 Expected Model of Change STEP 2 Model of EMC risk due to imminent change STEP 3 Review of performance management STEP 4 EMC risk estimation and impacts STEP 5 Exit STEP 6 Simulation of a controlled change (system to be ) to reduce EMC risks STEP 7 Redo STEP 6 STEP 8 Strategic/Process/Capability model to be STEP 9 Strategic/Process/Capability requirements STEP 10 implementation and monitoring Define the type of change that is incoming from an operational or business point of view Modelling the risks which are triggered directly by the incoming change: Expected Model of Change - EMC Risks Monitoring all relevant Key Performance Indicators (KPIs) and prepare to monitor their variance associated with change Assessing the EMC risks as defined in Step 2 With no EMC risk impacts, NO ACTION IS REQUIRED Exercise of simulation of relevant solutions to reduce hazards of the most relevant EMC risks identified in STEP 4 If there is no EMC risk reduction, then repeat STEP 6 If EMC risk reduction is confirmed in Step 6, a model of the system to be shall be defined according to MASCA requisites of strategy, process and capability Provision of Strategic, Process and Capability requirements to increase readiness to respond to the upcoming change Provision of means and plans (resources) to implementation and monitoring of strategic requirements Competence to describe, communicate and model the incoming change Competence to conduct focus groups with different organisational functions. Operational and statistical skills on relevant system Indicators Competence to conduct workshops and operational skills in Risk Matrix classifications, estimation and analysis n/a Simulation and estimation skills to reduce hazard frequency and/or severity of the selected EMC risks n/a Modeling skills, Problem solving skills, Communication skills, Estimation skills, Group decision making skills Competence in designing of strategic, process and capability requirements to implement Step 8 in the organisation Project management skills Project Auditing skills

4 Notably, the first column of Table 1 defines the LIFE steps, the second column the key actions per each step and the third column defines the requisite of competence to conduct the various steps. Step 1 would describe the organizational change in a clear and concise written form and leads to the agreement that the change could have a relevant potential effect upon the organization. A capacity to summarise the change in a written and visual description together with group communication and conflict resolution skills should be in place. The model of the chance is called the Expected Model of Change or EMC. This would bring forward Step 2 where a model about the areas of impact of those risks accounted for by the potential chance should be in place. Such types of risks are called the EMC risks. Competence to conduct focus groups to shortlist the all relevant EMCs and their areas of impact should be available. Estimation and group decision making skills are favorable at this stage. Although not directly associated to the previous operations, the Step 3 is dedicated to setting up a strategy about the use of the key performance indicators that will be critical to future monitoring of systems indicators sensitive to any organizational response to the change and the actual impact of change within the organization. Operational and statistical skills on relevant system Indicators are relevant of such a type of activity. Ranking and shortlisting the EMC risks which demands immediate actions is the key action in Step 4 where any selected EMC risk will be studied in a more classical approach in terms of hazard frequency and severity estimation. Competence to conduct workshops and operational skills about risk matrix classifications, estimation and qualitative or semi-quantitative risk assessments should be available at this level. If EMC risks have relevant impact according to the LIFE analysts, then Step 6 is applied. In particular a solution to reducing the hazard frequency and/or the severity of relevant EMC risks is simulated. The term simulation is here applied to define a group activity (a dedicated workshop) with all LIFE analysts in order to model a sustainable system change which can be put in place to increase the likelihood of reducing hazard frequencies or severities of any EMC risks. This is a purely prospective change assessment strategy where the company should imagine how to change its form, structure or functions in order to accommodate more proficiently the EMC and EMC risks which are estimated to occur in the future. In such view the organization thinks about an internal change to react to an external change that will occur in the future. Clearly, modeling and problem solving skills are paramount for this Step 6. Clearly if there is no EMC risk reduction, repetition of Step 6 is necessary as suggested by Step 7. Furthermore, Step 8 and 9 specify an effort to modeling strategy, processes and capabilities required to support the solution given, that is, the new system configuration and requirements as simulated in Step 6. Still modeling and problem solving skills are paramount for this phase. Finally Step 10 should be dedicated to implementation and monitoring of all key performance indicators according to the previous Step 3. Project management skills as well as project auditing skills are critical to keep under control the systems of change of the LIFE solutions. 3. THE IMPLEMENTATION OF NEW COMPANY PROCEDURES AND FLIGHT MANAGEMENT TOOLS: THE EFB SYSTEM The application of the LIFE methodology is now presented in the context of an operational case study for some of the LIFE steps defined accordingly. In particular, the model of change, or Expected Model of Change, in Step 1 (see figure 1 above) is given as use of the Electronic Flight Bag (EFB) technology. This is an operational change for an airline company approaching the introduction of such technology. Focusing on the flight operations area, the case study put in place a risk analysis technique to investigate potential future risk events that may be occurring if the EFB would be introduced in the airline company. In particular, Steps 1-4 of the LIFE methodology are here applied and presented in the case study below. In addition, with reference to Step 4 only the initial phase of the process of identification of hazards and risk evaluation is carried out, as the complete process has been described in a different paper presented in this Conference (De Grandis et al., 2012). Steps 5-10 are rapidly reviewed with respect to the LIFE approach as the EFB case study is at present only in its initial phase as it will be detailed in the following sections.

5 3.1 EFB implementation in AirDol The purpose of this section is to describe the evaluation of risks identified and connected to flight operations and the use of the EFB system. 3.2 Problem Statement The implementation of the EFB is being gradually introduced in the company, starting with a family of modern aircrafts in operation. The implementation and use of the EFB is a obviously an important change in the organization, as it is expected that there will be an overall return of effectiveness in the management of the operations by reducing the time spent in preparation of the different routes and loads as well as in optimizing flight times etc. This would also impact on the overall efficiency of the operations, reducing the amount of time loss in correlating the work of ground and flying staff. At the same time it is expected that there will be a reduction of human errors of different nature, in relation to the evaluation of routes and in preparing landing and take-off procedures, as well as in setting a variety of data and setting crucial quantities, such as the key decision speeds of go-non-go or rotate (take-off speed). However, it is important to consider that the introduction of the EFBs may generate other types or modes of human error which must be defined and studied in order to perform a complete assessment of the new risk scenarios to be assessed. The problem to be studied in this phase of implementation of the LIFE approach is only associated to the potential hazards and indicators of safety that result from the change. The full process of risk evaluation will be implemented in the future, when the EFB will be active and in operation within the organization and data collection will take place. In this initial stage, only the expert judgment of analysts and the familiarity with company rules and accepted behaviours (culture) is being utilized, in combination with the available data and past experience of certain associated organizations willing to share their historic data and past experience. Moreover, the process of implementation of the EFB within that company follows a precise process of progressive inclusion of electronic support. The initial phase of implementation implies the usage of electronic maps inserted in special hardware systems (i-pod, tablets, portable PC, etc.) to be carried on board by the pilots and connected externally to the cockpit controls and interfaces. The usage of hardware systems (PCs or tablets), external to the cockpit control systems and flight directors, implies that the EFB system cannot be utilized during flight critical phases of take-off and landing and can be extensively utilized and connected to the control system of the aircraft while on the ground and during the pre-flight phase. In a second phase of implementation, the EFB will be fully integrated within the control system of the aircraft so as to enable usage during the whole operational period. In this case therefore, in addition to the usage of the EFB for the maps and airport plans and procedures, it will be possible to extensively use the electronic support for calculating critical dynamic quantities, such as weight and balance and speeds. These two phases of implementation of the EFB require the use of specific methods and approaches in support of the hazard identification and risk evaluation. In this paper, the initial steps of phase one are described, highlighting the possible techniques and ways of implementation for the assessment of the KPIs of the second phase. 3.3 Qualitative Event Assessment In order to carry out the preliminary analysis of the usage of the EFB in the configuration of the hardware systems carried on board and their usage for the pre-flight phase a set of workshops and brain storming meetings have been carried out in order to define the set of hazards or undesirable operational states that may derive from the usage of the EFB systems. This process leads to a simple table of generic hazard identification in association to the usual activities and processes implemented in the company as Standard Operating Procedures (SOP). In the case of the EFB, this table is developed considering the pre-flight steps and activities carried out by the pilots (Table 2). These can

6 be further subdivided in two periods: cockpit preparation and final cockpit crew. They can be further subdivided in: Cockpit Preparation comprising: Cockpit Power up Walk Around (external inspection) Cockpit Preparation by pilot in command (CM 1 - captain) Cockpit Preparation by pilot (CM 2 - first officer) Cockpit Crew comprising: Final Cockpit Preparation The most relevant contribution of this step of activity is represented by the identification of the conditions generating hazards and the initial assessment of the potential consequences or outcome of the evolution of the hazard, independently of containment measures or consequences. Table 2. Generic hazard identification Activity or Issue Hazard Potential outcome Phase Cockpit Preparation Cockpit Power up Walk Around (external inspection) Cockpit Preparation CM 1 (Pilot in Command) Cockpit Preparation CM 2 (First Officer) Excessive workload of CM2 due to number of task to carry out during cockpit preparation Improper/inadequate loading of software Lack of adjournment of software Lack of familiarity with PC handling, time pressure on CM2 - Software initialisation not completed - Maps not available - Improper selection of portrait - Maps not available - Improper selection of portrait - Maps not available - Improper storage of PC - Maps not available - Flight cancellation or delay - Flight cancellation or delay - Flight cancellation or delay - Damage to cables or PC - Fire/smoke in the cabin - Flight cancellation or delay Phase Cockpit Crew Final Cockpit Preparation Pilot workload Out of charge batteries No updated paper maps or missing paper maps No Airfield Sketch. Lack of familiarity with airfield, worsen by visibility problems No Ground facilities (radar, light guidance system, etc.) No SID (Standard Instrumental Departure) No/Wrong SID, bad weather No approach chart in the case of emergency (bad weather, environment difficult, for example mountains) - Pilots unable to locate maps - No charts on show - Flying with wrong maps or without maps - No coordinates for Xcheck with FMS (impossible to see taxiway) - Getting lost on airfield - No info/news on obstacles - Missing performance - Flying wrong departure - Missing information in the case of emergency (increase of WL of crew) - Diversion Delay - Diversion Delay - Runway incursion - Flight cancellation - Ground collision (aircraft, infrastructures and vehicles) - Mid-air collision (ground and flight) - Wrong runway take-off - Loss of control in flight

7 3.4 Quantitative Assessment Methods for quantification of risks The quantification phase of the approach begins by combining the different hazards identified in the qualitative assessment process and associating these to the potential outcome resulting from the brain storming process. This leads to the definition of the hazards that need to be studied in order to position each of them on the risk matrix. This is a typical quantification process that enables to associate to each hazard a probability of occurrence and a severity of the resulting consequences (ARMS, 2011). Both these quantification approaches require the usage of a variety of methods. Usually, Expert Judgment () is applied as it seems fast and simple to implement. In reality, a formal implementation of the approach requires a rather lengthy and demanding process of definition of the quantities to be evaluated. can be utilized for probabilities and occurrences as well as severity of consequences. Other methods may be considered when may be too expensive to apply in a formal way or is considered too simple and shallow if simple judgment is utilized. Generally speaking the assessment of the severity of consequences is performed by simulation or by coupling simple simulation and some experimental analysis. In order to assess the probabilities of each occurrence a variety of methods exists than can be applied depending on the type of hazard being studied, e.g., system failures, human inadequate performances etc. Preliminary quantification of hazards and consequences In the case of the EFB case study, the initial assessment of the hazards and consequences started by combining the various hazards and building the actual undesirable operational states to be assessed. In the case of the EFB, the initial study identified 14 hazardous conditions (Table 3), out of the list of hazards resulting from the preliminary analysis. For each hazard the set of possible outcomes are also selected from the qualitative analysis. In order to assess the severity of the consequences some initial work of definition of the Control Measures is carried out and the measures are identified. The evaluation of the severity of the consequences is defined in relation to a reference set of severity values that are defined and fixed for all the safety studies performed for the organizations. In this case, a set of six severity levels has been utilized ( None, Minor, Low, Medium and Extreme ) in association to the level of damage resulting from the consequences of the hazard evolution. In this phase of study, full usage of has been utilized, also in consideration of the preliminary nature of the study. In order to evaluate the probability of occurrence of each consequence, the process of evolution from the initial hazard (undesirable operational state) to the final event must be considered. For this first phase of implementation of the EFB, amongst the variety of tools and techniques available for assessing the probability of occurrence, the most appropriate methods that have been applied are the use of and the approach approach known as (Colombari and Bello, 1980). While is the most common approach utilized for assessing probabilities, as discussed above, needs the availability of an actual procedure or the implementation of a specific activity to be carried out by operators in order to associate a specific probability of unsuccessful performance. In some cases, however, it has been considered more appropriate the usage of or possibly also the process of elicitation of knowledge and experience existing within the organization as well as in the database of reporting incidents and occurrences of various nature. As in the case of the evaluation of the severity, also for the assessment of the probabilities a standard set of values applied thorough the safety studies of the organization has been defined, with 7 discrete levels of probabilities, i.e., Extremely remote, Remote, Unlikely, Low, Possible, Likely, Frequent. The combination of the 7 discrete levels of probabilities with the 6 levels of severity defined for the quantification of the consequences enables the definition of the Risk Matrix that is utilized as reference for the assessment of all hazards selected for assessment for the organization. The use of the same Risk Matrix (RM) for all hazards selected for safety assessment ensures consistency of the analysis. The RM in use at AirDol has already been presented elsewhere (De Grandis et al., 2012) and will not here shown again.

8 However, it is important to recall that the combination of severity and probability results in sets of cells of the RM with a precise indication of activity to consider for ensuring the respect of safety margins. These are indicated by letters and have the following meaning: A Extreme risk requiring immediate mitigation; B risk requiring short term mitigation; C Acceptable risk with mitigation potentialities, requiring long term improvement; D Low risk requiring simple monitoring; and E Negligible risk, data collection required. In the present study, initial quantification has been performed and results are shown in Table 3. Table 3. Quantification of hazards and consequences. Hazard No. Description 1 Software initialisation not completed 2 Maps not available 3 Improper selection of portrait 4 Improper storage of PC 5 Pilots unable to locate maps Incident sequence description - Flight delay or canc. - Flight delay or canc. - Flight delay or canc. - Damage to cables/pc - Fire/smoke in cabin - Flight delay or canc. 6 Loss of SA - Diver./Alter./Delay 7 No charts on show - Diver./Alter./Delay Existing Control Maintenance Quality Maintenance Quality SOP/EOP Maintenance Quality SOP SOP/EOP SOP/EOP Maintenance TCAS EGPWS Outcome (Pre- Mitigation) Severity Low Low Low Likelihood Risk 3, C 2, C 1, C 2, C 1, C 6, C 2, B Additional mitigation required Outcome (Post- Mitigation) Severity Likelihood Risk Action s and owners Monitor. & Review requir. 8 Flying with wrong maps or without maps - Diver./Alter./Delay SOP/EOP 1, C 9 No coordinates for Xcheck with FMS (impossible to see taxiway) 10 Getting lost on airfield 11 Missing performance 12 Missing infor. in the case of emergency (WP of crew) 13 No info/news on obstacles 14 Flying wrong departure - Runway incursion - Ground collision with infras./ac/veh. - Wrong runway t-off - Runway incursion - Ground collision with infras./ac/veh. - Wrong runway t-off - Mid-air collision - Loss of ground sep. - Loss of control in flight - Loss of ground sep. - Mid-air collision - Loss of ground/air separation - Wrong runway t-off SOP EOP EOP EGPWS EOP EGPWS EOP SOP - EOP Extreme Medium Extre 3, C 1, D 3, C 6, C 3, C 3, C The implementation of the remaining steps of development of the LIFE approach would lead to the completion of the Hazard-Consequences table. This process is presently under development.

9 Considering the overall risk assessment, it is noticeable that each hazard generates more than one sequence/occurrence of different severity and probability. However, in Table 3, which represents the final table for risk assessment, only one of these sequences is considered. This is the occurrence that is associated with the highest risk, i.e., the highest value of combined severity and probability. This is considered acceptable as the additional mitigations required to reduce risk are combined with the initial hazard and their mitigation effects can be expanded to all possible consequences derived from such hazard. Therefore, it can be assumed that reducing to an acceptable level the risk of the occurrence with highest risk forces all the other occurrences to an acceptable level. As a consequence of these considerations, the hazard that presents the highest risk (Table 3) is the Absence of charts ( Hazard 7 ) and the associated sequence loss of separation between aircrafts, which leads to high risk requiring short term mitigation (level of risk B ), even if the severity is high and not extreme, as it would be if the sequence CFIT (Controlled Flight Into Terrain) was to be analyzed. 4. CONCLUSION The application of some of the steps in the LIFE methodology in the context of a real operational case study is necessary for the validation of the methodology. This paper has provided an overview of the process of implementation of the methodology and its application to the case of an actual implementation of management of change. The results obtained so far are promising and enable an organization, through a systematic and prospective risk and hazard assessment, to reduce the risk that the organization faces in meeting its strategic challenges, comparing current performances to projected demands. The use of methods such as Expert Judgment and are supported by extensive applications in other real safety assessment. It is however the responsibility of the safety manager of the organization to select the most appropriate techniques and methods to perform the evaluation of risk and depends on the level of expertise of the whole safety team involved in the analysis. The methodology remains the most innovative aspect of the proposed approach and the needs to account for changes in the organisation are the current most relevant issues to be dealt with by means of risk based approaches. Acknowledgements The MASCA project has received funding from the European Commission Seventh Framework Programme (FP7/ ) under grant agreement n References ARMS, (2011) The ARMS Methodology for Operational Risk Assessment in Aviation Organisations. visited Bello, G.C. and Colombari, C. (1980) The human factors in risk analyses of process plants: the control room operator model,. Reliability Engineering De Grandis E., Oddone, I., Ottomaniello, A., Cacciabue, P.C. (2012) Managing risk in real contexts with scarcity of data and high potential hazards: the case of flights in airspace contaminated by volcanic ash. These Proceedings of PSAM-11 - ESREL 2012, Helsinki, Finland, June ICAO - International Civil Aviation Organisation (2009) Safety Management Manual Doc 9859, AN/474. Second Edition, Montreal, Canada. McDonald, N., Ulfvengren, P., Ydalus, M., and Oder, E. (2012) A methodology for managing system change. This Proceedings of PSAM-11 - ESREL 2012, Helsinki, Finland, June