THE ICAO UNIVERSAL SAFETY OVERSIGHT AUDIT PROGRAMME CONTINUOUS MONITORING APPROACH (USOAP CMA)

Transcription

1 THE ICAO UNIVERSAL SAFETY OVERSIGHT AUDIT PROGRAMME CONTINUOUS MONITORING APPROACH (USOAP CMA) Nicolas Rallo Chief, Safety and Air Navigation Oversight Audit ICAO 2015 AMERCA Flight Standards Meeting April 7-8, 2015 Tysons, Virginia USA

2 ICAO s monitoring of safety oversight ICAO carries out audits and other monitoring activities to determine its Member States safety oversight capabilities mainly by assessing the effective implementation of the 8 Critical Elements in 8 audit areas (LEG, ORG, PEL, OPS, AIR, AIG, ANS and AGA) through Protocol Questions (PQs). 2

3 The 8 Critical Elements (CEs) 2 Operating Regulations 1 Legislation 3 State System & Functions 6 Licensing & Certification Obligations ESTABLISH IMPLEMENT 8 Resolution of Safety Issues 5 Guidance, Tools & Safety-critical Information 7 Surveillance Obligations 3

4 USOAP CMA audit areas Primary aviation legislation and civil aviation regulations (LEG) Chicago Convention Personnel licensing and training (PEL) Annex 1, and 19 Airworthiness of aircraft (AIR) Annexes 6,7,8, 16 and 19 Air navigation services (ANS) Annexes 2,3,4,5,10,11,12,15, 19 and PANS ATM Civil aviation organization (ORG) SAAQ Aircraft operations (OPS) Annexes 6, 9, 18, 19 and PANS OPS Aircraft accident and incident investigation (AIG) Annex 13, and 19 Aerodromes and ground aids (AGA) Annex 14, and 19

5 The components of USOAP CMA States Internal stakeholders External stakeholders Collection of safety information Determination of State safety risk profile Analysis of safety risk factors Evaluation of State s safety management capabilities Mandatory Information Requests (MIRs) PQ findings Significant Safety Concerns (SSCs) Corrective Action Plans (CAPs) Update of EIs and status of SSCs Prioritization and conduct of USOAP CMA activities USOAP CMA audits Safety audits ICAO Coordinated Validation Missions (ICVMs) Off-site validation activities 5

6 Prioritization of USOAP CMA activities USOAP CMA activities in States are prioritized by ICAO, based on State safety risk profiles and taking into consideration the approved budget and resources available. 6

7 Mandatory Information Requests (MIRs) - A Mandatory Information Request (MIR) are issued by ICAO mainly when Concerns are raised by internal/external stakeholders; - An MIR can lead to a finding or even an SSC. 7

8 Significant Safety Concerns (SSCs) An SSC occurs when the audited State allows the holder of an authorization or approval to exercise the privileges attached to it, although the minimum requirements established by the State and by the Standards set forth in the Annexes to the Chicago Convention are not met, resulting in an immediate safety risk to international civil aviation. Reference: EB 2010/7 dated 19 February

9 Significant Safety Concerns (SSCs) SSCs unresolved in 13 States 15 SSCs resolved through corrective actions taken by the States 30 SSCs resolved by immediate actions taken by the States prior to being posted on the ICAO website 9 9

10 As of January 2013, safety oversight information is available Evolution on the of ICAO Transparency public website. URL: STATES 1997: Voluntary Assessment Programme, Fully Confidential (Annexes 1 6 8) PUBLIC ICAO has identified a significant safety concern with respect to the ability of [State] to properly oversee the [insert airlines (air operators); airports; aircraft; or air navigation services, as applicable] under its jurisdiction. This does not necessarily indicate a particular safety deficiency in the [insert airlines (air operators); airports; aircraft; or air navigation services, as applicable] but, rather, indicates that the State is not providing sufficient safety oversight to ensure the effective implementation of applicable ICAO Standards. Full technical details of the ICAO findings have been 1999: USOAP Audit Summary Reports to all States (Annexes 1 6 8) 2005: USOAP CSA Audit results full transparency to all States 2006: SSC introduced, fast track notification to all States (restricted web site) 2001: Generic, non State specific LEI results globally and by region 2005: Public access to LEI, Critical Element results by State. All States provided consent 2006: Mechanism to make full USOAP results available to the public with State consent. 1 st cycle audits 45% of States made available to [State] to guide rectification, as well as to all ICAO Member States to facilitate any actions that they may consider necessary to ensure safety. [State] has undertaken to regularly report progress on this matter to ICAO. SSCs published on the CMA on line framework 2014 Unresolved SSCs made available to the public in the format and conditions approved by Council 10

11 Global Results by area and CE

12 Some keys questions for sustainability - PQ 2.051: Has the State established and implemented a mechanism to ensure that each safety oversight entity has sufficient financial resources to meet its respective national and international obligations? (average EI: 62%) - PQ 2.053: Has the State established a mechanism to ensure that each safety oversight entity/investigation authority has sufficient personnel to meet its respective national and international obligations? (average EI: 23%) - PQ 2.103: Is each safety oversight entity/investigation authority able to attract, recruit and retain sufficiently qualified/experienced technical personnel? (average EI: 42%) - CE 4 related questions (average CE4 EI: 48%)

13 Regional Cooperation ICAO encourages regional cooperation, and in particular through the establishment of Regional Safety Oversight Organizations (RSOOs) and Regional Accident Investigation Organizations (RAIOs). The challenge is not only to establish RSOOs/RAIOs but also to make them effective.

14 For any question, please contact: 14

15 International Aviation Assessment (IASA) Trends Kim B. Miller, Technical Support Branch Manager International Programs and Policy Division, Flight Standards 2015 AMERCA Flight Standards Meeting April 7-8, 2015 Tysons, Virginia USA

16 IASA Trends In the early 1990 s In many cases a country s general lack of awareness of its obligations under the Chicago Convention on Civil Aviation. A general lack of awareness of the International Civil Aviation Organization (ICAO) Annexes. Non-qualified inspectorate personnel, including personal friends, associates, and relatives. In some extreme cases a standing building without a national Civil Aviation Authority (CAA) being physically present.

17 IASA Trends In the mid 1990s ICAO publishes the critical elements for aviation safety oversight in Doc The eight critical elements are essentially the safety defense tools of a safety oversight system and are required for the effective implementation of safetyrelated policy and associated procedures. States are expected to implement safety oversight critical elements in a way that assumes the shared responsibility of the State and the aviation community. The effective implementation of critical elements by a CAA indicates the capability for safety oversight. ICAO Document 9734 Part A

18 IASA Trends Common IASA Deficiencies * CE-1 Primary aviation legislation: Legislation lacks legal standing, proposed drafts, cosmetic changes, they re almost done * CE-2 Specific operating regulations: Adoption of CFRs, old JARs, EASA, or other neighboring CAAs; poor, unenforceable translations, not implemented, contradictory * CE-3 State civil aviation system and safety oversight functions: Lack of appropriate funding, housing, equipment, adequate technical and non-technical staff, computers, recordkeeping, documents, transportation, airport access

19 IASA Trends Common IASA Deficiencies * CE-4 Technical personnel qualification and training : Personnel are not technically qualified for assigned duties. Experienced aviation personnel with no inspector regulatory training and/or consultant inspectors with no legal authority to perform function(s). Still at the airline, not appropriately seconded to the CAA. No training records. No formal system of training established or provided. CAA uses different sources of adhoc training for different employees. General adoption of generic training without bridging in to the national system, i.e differences training on country specific law, regulations, procedures. No formal system of training history retention.

20 IASA Trends Common IASA Deficiencies * CE-5 Technical guidance, tools and the provision of safetycritical information: Frequently wholesale adoption of guidance of another authority, i.e FAA Order , , Flight Standards Information Management System without customization for their use (reference to US FAR, in English, FAA specific sections/offices not removed). Inconsistent/conflicting guidance material. Personnel do not have access to mandatory continuing airworthiness information, manufacture s manuals, Master Minimum Equipment lists, ICAO publications among others.

21 IASA Trends Common IASA Deficiencies * CE-6 Licensing, certification, authorization and approval obligations: Incomplete or incorrect original certification or recertification, not in accordance with their own regulations, guidance, and international standards by non-qualified personnel. Certification lacks critical demonstration of abilities, i.e. demonstration flights and appropriate evacuation activities. No formal records of certification exist.

22 IASA Trends Common IASA Deficiencies * CE-7 Surveillance obligations: No formal surveillance plan, significant safety discrepancies overlooked, item(s) not reported, item(s) not captured, no documentation/records of surveillance. * CE-8 Resolution of safety concerns: No corrective action, no follow-up/close out action by CAA, no documentation/records of corrective action, followup/close out action.

23 The Consequences IASA Trends Improper and insufficient inspections prior to certification of air operators, maintenance organizations, & aviation training organizations Personnel licenses and certificates improperly issued, validated, & renewed Technical Personnel reluctant to make technical decisions because of lack of authority Procedures and documents improperly approved Specific operational authorizations issued without determining applicants abilities to carry them out safely No follow-up on identified safety concerns or

24 IASA Trends A lack of Political Will by national leaders has a critical affect on safety oversight - Frequently demonstrated by a: Lack of understanding of Signatory obligations Lack of resources (e.g., funding, housing, equipment, adequate support staff, computers, recordkeeping, documents, transportation, airport ramp access) Inability to attract, recruit, and retain technically qualified personnel Frequent changes in CAA top leadership Real or perceived job insecurity for technical staff Low morale and active searching for other employment by qualified technical personnel facilities,

25 IASA Trends Sustainability in the long term Obtaining a Category 1 rating under IASA is only the beginning The hardest part of a Category 1 rating is sustaining it Requires a continued commitment by the personnel of the civil aviation authority to do it right Requires the continued monitoring of the State s aviation oversight system, and to make systemic improvements and corrections when needed Requires a continued commitment of political will and economic resources by the national decision makers

26 IASA Checklist: A Practical Tool for Category 1 Sustainment LP Vanstory Aviation Safety Inspector Flight Standards Service 2015 AMERCA Flight Standards Meeting April 7-8, 2015 Tysons, Virginia USA

27 Sustainment Political will is paramount to sustainment of IASA Category 1 How to maintain resources and political will? Communication Protect your investments into your oversight system IASA checklist can provide a roadmap to sustainment

28 IASA Checklist The IASA checklist is readily available online through the FAA public website: As mentioned before, the IASA utilizes ICAO minimum requirements as outlined in their Standards These standards are woven into the 8 Critical Elements of a Safety Oversight System

29

30 8 Critical Elements of Safety Oversight Resolution of Safety Concerns Continued EIGHT CRITICAL SurveillanceELEMENTS Licensing and OF SAFETY Certification OVERSIGHT Qualified Technical Personnel Technical Guidance CAA Structure Specific Operation Regulations Primary Aviation Legislation

31 What is my Int l Aviation Landscape? Aviation is an industry and as in any industry it is always in flux new routes, new carriers, mergers, business failures, aging aircraft, business aviation Oversight needs to keep up with the aviation activities in your country. Keeping an updated General Information and Air Operator Complexity can assist monitoring changes required for oversight when it comes to international operations If you see significant changes, it may indicate changes in required resources/capabilities

32

33 Checklists The checklists for each of the critical elements is the basis for how CAAs are assessed in meeting international requirements As a result periodically updating and maintaining these lists can ensure awareness of what needs to be improved upon Additionally, reviewing this list and ICAO SARPs can assist during development of new regulations and requirements

34 Checklist References: ICAO standards and related documents Your domestic regulations/docs When there are changes to operations and/or regulations/requirements, this list can direct you to where you need to cross reference Check to see if you filed differences or if additional resources/capabilities are required these can be noted

35 Common Problems in Compliance Sustaining Compliance over time Lack of technical qualified personnel Type rated operations inspectors Personnel Retention Lack of certification and documentation Lack of clear and implementable regulations Checklists can assist in identifying any gaps in your system. This information can be used to target communications to those with resources. Specific information can focus resource requests and assist in maintaining Political Will

36 Questions?

37 Sustainment through Regional Organizations Suzette Nieuwoudt Project Coordinator ICAO COSCAP-SADC (RSOO-SASO) 2015 AMERCA Flight Standards Meeting April 7-8, 2015 Tysons, Virginia USA

38 Aviation Policy Issues

39 USOAP - SSCs IASA - Cat 2

40 AFRICA Effective Aviation Safety Oversight

41 EI OF 8 CEs

42 Indicators for Sustainment of Safety Performance Lagging Indicators o measure failure o reactionary measure Leading Indicators o measure performance o proactive LAGGING + LEADING = TO MEASURE SAFETY PERFORMANCE focus on impactful indicators rather than tracking intention

43 Areas of mutual concern Establish Implement

44 Regional Harmonisation 8

45 Regional Collaboration without duplication Boeing s Ageing Aircraft Maintenance Program Training ICAO endorsed Government Safety Inspector (GSI) Inspector Training System (ITS) (Decision Makers & Senior GSIs) Safety Management Systems (SMS) Resolution of Conflict/Safety Concerns (RSC) OPS/AIR/PEL/ATO/SUR 9

46 Regional Cooperation Cooperation with other Regional Initiatives RSOO Pre-ICVM Focused Technical Assistance AFCAC AFI-CIS (Cooperative Inspectorate Scheme) ICAO ESAF ROST (Regional Operational Safety Team) EASA SIASA (Support to Improvement of Aviation Safety in Africa) 10

47 Share Best Practices On the Job Training (OJT) during technical assistance missions OJT Aligned with ITS OJT level 1,11, 111 Experience opportunities Members of RSOO invited as observers 11

48 Share Best Practices (cont d.) Emphasize State specific CAP dependence on PQs All PQs relating to Findings (self-assessment questions) have to be satisfactorily answered CAP updating on istars All references to be verified as correct prior to them being entered onto the CAP on istars / OLF Completing State specific self-assessment questions CAP should be updated as progress is made to realistically reflect the State s progress. Assembling supporting documentation Certification History Files Individual Inspector Training Files ICVM Evidence Files 12

49 Sustaining standards Consistent and lasting regional harmonisation through cooperation coordination participation collaboration sharing amongst States within a specific Region, with ICAO, in existing aviation safety initiatives in the region, on the continent and globally, with international technical partners; and by one smart corrective action at a time

50 Thank you