Independent safety assessment by a CSM Assessment Body (RASBO)

Independent safety assessment by a CSM Assessment Body (RASBO) Planning, delivery, management and independent safety assessment report Presentation to ERA Arthur D. Little Limited Science Park Milton Road Cambridge CB4 0FH United Kingdom Tel.: +44 1223 427 100 www.adlittle.co.uk

Content 1 About Arthur D. Little 2 Introduction to independent safety assessment 3 Timing of independent safety assessment 4 Planning an independent safety assessment 5 Conducting the independent safety assessment 6 Management of findings 7 Contents of the safety assessment report 8 Cross acceptance 9 Value of independent safety assessment 2

1 Content About Arthur D. Little Arthur D. Little is the world's first Management Consultancy, focusing on technology intensive sectors Innovating business since 1886 Today we are the only premier global management consulting firm with a 125 year track record. Founded as a technology consultancy, throughout our history we have contributed to numerous ground breaking innovations Linking strategy, technology and innovation We are acknowledged as a thought leader in linking strategy, technology and innovation. We focus on technology intensive sectors delivering business transformation Working uniquely, different Our people and their side-by-side TM approach integrate cross-sector knowledge and next level thinking seamlessly into your business. How we work is innovative, what we deliver is positive change 3

1 Content About Arthur D. Little Safety and risk has been a part of the firm s heritage for over a century, highlighted by numerous landmark assignments Waste minimization services Circa 1905 Warning agents for natural gas 1952 NIOSH chemical standards 1979 Texas City investigation 1947 EPA RCRA Program Support 1976-1980 Mass Transit Railway Signaling ISA 1988 to date EPA Risk Assessment Support 1979-1985 Strategic Safety Management Training 1993 to date Delhi Metro ISA - 2003 to date Prototype Safety Case supporting UK Rail Privatisation 1993-1995 Development of Hong Kong MTR Safety Management 1989 to date Hooker Chemical Investigations 1979 Bhopal investigation 1985 Exxon Valdez 1989 Major HAZOP/SIL studies worldwide 1970 to date BP Forties Field Quantified Risk Assessment 1988-1990 Shell/Exxon UK Fife Plant Risk Studies - 1987 to date ENI Refining Risk Studies 1990 to date Greek Gas Pipeline Network Risk 1990 to date Safety critical systems 1995 to date Swiss and Munich Re EHS Management Systems 1999-2000 UK Nuclear Liabilities Assessment 1999-2000 Safety Development Programs 2008 to date 4

1 Content About Arthur D. Little We have undertaken signalling ISA assignments for many clients around the world European Signalling Products Generic ERTMS (EVC and RBC) CBTC GoA Level 4 European Signalling Projects LTA Singapore DTL1 MTR Corporation ATC Replacement Lantau Airport Railway Quarry Bay Congestion Relief Driverless Turnaround Tseung Kwan O LAR 4-tracking YAM O Signalling Disney Resort Line West Island Line Betuweroute ERTMS L2 trackside SA-NBS ERTMS L2 Delhi Metro, Chennai Metro Delhi Phase 1, 2 & 3 Chennai Metro KCR Corporation East Rail Resignalling 5

2 Content Introduction to Independent Safety Assessment An independent safety assessor provides additional assurance that a given significant change achieves the necessary level of safety Conducts independent safety assessment of the risk assessment process and safety demonstration of the significant change Does not perform the risk assessment required by the CSM nor the safety demonstration itself Provides confidence that the risk assessment and risk management activities have been properly conducted by the proposer Conclusions are not binding on the proposer but are an important input to be taken into account by them 7

2 Content Introduction to Independent Safety Assessment An independent safety assessor must be independent and impartial Must be independent from the design, risk assessment, risk management, manufacture, supply, installation, operation/use, servicing and maintenance" of the system under assessment Must be free from any pressure or incentive which may affect their judgement Must not deliver advice or solutions on how to address non-conformances or concerns identified by the assessment 8

2 Content Introduction to Independent Safety Assessment An independent safety assessor must be competent Risk management competence Technical competence Management systems competence 9

2 Content Introduction to Independent Safety Assessment Independent safety assessment is widely used, in many industries Rail Medical Nuclear Independent Safety Assessment Road Transport Defence Aviation 10

3 Content Timing of Independent Safety Assessment Independent safety assessment is best conducted throughout the risk management process, from start to finish Source: ERA 12

4 Content Planning an Independent Safety Assessment An independent safety assessment plan should describe, in practical terms, how the assessment is to be completed Scope of work Assessment approach Independent Safety Assessment Plan Assessment team Schedule of assessment activities Planned deliverables 14

Content 5 Conducting the independent safety assessment 5.1 General approach 5.2 Gathering data 5.3 Working papers 16

5.1 Content Conducting the Independent Safety Assessment General Approach The depth of assessment should be appropriately balanced, according to the significance of the change Balanced approach Heavyweight assessment to manage potential lower risks Lightweight assessment to manage potential high risks Significance of change Depth of assessment Pretty much OK Reflects an imprecise understanding of potential risks Vulnerable to surprises and omissions 17

5.1 Content Conducting the Independent Safety Assessment General Approach The depth of assessment can be guided by a thorough understanding of the management systems and processes and through an assessment of their strengths and weaknesses Understand Examine the management systems and processes in place Verify Test each system or procedure Emphasis given to possible weaknesses Focus on those elements that appear most critical Assess Analyse such processes for weaknesses 18

5.1 Content Conducting the Independent Safety Assessment General Approach Sufficient understanding of all relevant management systems should be obtained Text Software Design Technical Management Safety Management Requirements Capture Quality Management Hardware Design Verification and Validation 19

5.1 Content Conducting the Independent Safety Assessment General Approach Once the management systems are understood, their apparent effectiveness should be assessed 1 Consider potential impacts 2 Evaluate the management system 3 Set inspection priorities What are the potential impacts if the management system does not operate correctly? Are the specified processes likely to deliver their defined and/or necessary objectives? Are the systems and processes, coupled with their controls, sufficient to mitigate the potential impacts? Ensure potential issues representing high risk and/or weak management controls receive sufficient attention 20

5.1 Content Conducting the Independent Safety Assessment General Approach Verification should assess the correctness of the strengths and weaknesses determined in the previous analysis 21

5.1 Content Conducting the Independent Safety Assessment General Approach Verification should assess the correctness of the strengths and weaknesses determined in the previous analysis (continued) 22

5.2 Content Conducting the Independent Safety Assessment Gathering Data Assessment data can be gathered through a mixture of interviews and document review Interviews are a highly effective tool for gathering evidence 24

5.2 Content Conducting the Independent Safety Assessment Gathering Data Assessment data can be gathered through a mixture of interviews and document review (continued) Interviews are a highly effective tool for gathering evidence Documentary evidence is also gathered from review of the proposer s documents and records 25

5.3 Content Conducting the Independent Safety Assessment Working Papers Working papers should be maintained by each assessor, throughout the assessment Can be paper or electronic Record all assessment activities and their results Written whilst conducting the assessment activities Provide a basis for quality assurance 27

6 Content Management of findings A log of findings should be maintained, tracking the status of all issues identified during the assessment Documents all findings (non-conformities, inadequacies, etc) Tracks all such findings to closure: Original finding, responses from the proposer, updates from the assessor Dates of relevant updates / responses Current status (open / closed) Findings Log Updated as necessary throughout the assessment All findings should be closed or non-blocking prior to issue of a positive safety assessment report, or conditions noted accordingly 29

7 Content Contents of the safety assessment report The safety assessment report provides conclusions on the fulfilment of the safety requirements by the significant change Identification of the CSM assessment body Summary/reference of the independent safety assessment plan Definition of the scope of the assessment, including limitations Results of the assessment Details of the assessment activities performed to check compliance with the CSM for risk assessment Identified non-compliances and recommendations Details of cross acceptance Conclusions Does the risk assessment performed by the proposer comply with the requirements of the CSM? Are the risk controls sufficient to allow the change to safely fulfil its intended objectives? 31

8 Content Cross Acceptance An independent safety assessment often relies on the results of other assessments performed by third parties Has the third party assessment been performed with the necessary independence, impartiality and competence? Is the artefact that is subject to cross-acceptance being used in the same context as that assumed by the original assessment? Does the assessment reach clear conclusions? Are any restrictions raised by the assessment complied with? 33

9 Content Value of independent safety assessment Done well, independent safety assessment can significantly increase confidence in the safety of a change Good understanding & review of management processes allows significant weaknesses to be identified and corrected Effective review helps detect systematic errors Focus on management systems encourages proposer organisations to continuously improve 35