Assurance Cases. Software Engineering Institute Carnegie Mellon University Pittsburgh, PA Charles B. Weinstock January 26, 2015

Transcription

1 Software Engineering Institute Carnegie Mellon University Pittsburgh, PA Charles B. Weinstock January 26, 2015

2 Report Documentation Page Form Approved OMB No Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington VA Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. 1. REPORT DATE 26 JAN TITLE AND SUBTITLE 6. AUTHOR(S) Weinstock /Chuck 2. REPORT TYPE N/A 3. DATES COVERED 5a. CONTRACT NUMBER 5b. GRANT NUMBER 5c. PROGRAM ELEMENT NUMBER 5d. PROJECT NUMBER 5e. TASK NUMBER 5f. WORK UNIT NUMBER 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) Software Engineering Institute Carnegie Mellon University Pittsburgh, PA PERFORMING ORGANIZATION REPORT NUMBER 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR S ACRONYM(S) 12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release, distribution unlimited. 13. SUPPLEMENTARY NOTES The original document contains color images. 14. ABSTRACT 15. SUBJECT TERMS 11. SPONSOR/MONITOR S REPORT NUMBER(S) 16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF ABSTRACT SAR a. REPORT unclassified b. ABSTRACT unclassified c. THIS PAGE unclassified 18. NUMBER OF PAGES 19 19a. NAME OF RESPONSIBLE PERSON Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18

3 Copyright 2015 Carnegie Mellon University This material is based upon work funded and supported by the Department of Defense under Contract No. FA C-0003 with Carnegie Mellon University for the operation of the Software Engineering Institute, a federally funded research and development center. NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN AS-IS BASIS. CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT. This material has been approved for public release and unlimited distribution. This material may be reproduced in its entirety, without modification, and freely distributed in written or electronic form without requesting formal permission. Permission is required for any other use. Requests for permission should be directed to the Software Engineering Institute at permission@sei.cmu.edu. DM

4 Overview Maturity of ISO Assurance Case Standard Goal Structured Notation Example from Industry Confidence Work at the SEI Other Current Work on Closing Thoughts 3

5 Maturity of Assurance Case Technology Developed in late 90s in Europe Used for safety cases in Europe for over 20 years The UK Ministry of Defence requires generation of a compelling case to support claims that specific safety requirements are met: The safety case shall consist of a structured argument, supported by a body of evidence, that provides a compelling, comprehensible and valid case that a system is safe for a given application in a given operating environment. [DEFSTAN (Part 1)/4] ISO standard under development (ISO ) NRC Report: Software for Dependable Systems: Sufficient Evidence? 4

6 ISO/IEC : Assurance Case The assurance case is to be delivered and maintained with the system Claim: A proposition to be assured (e.g., The system is safe ) Evidence: A fact, datum, object, claim, or other assurance case Argument: A reason why the set of evidence shows that the claim is true Justification: A reason why a claim has been chosen Assumption: A claim that appears as evidence An Assurance Case is a quadruple a=(c,j,es,g) where c is a claim, j is a justification, es is a set of evidence, and g is an argument which assures c using es. This definition is recursive 5

7 Goal Structuring Notation (GSN) Kelly 1998 A specific notation for an assurance case consistent with Developed to help organize and structure safety cases in a readily reviewable form Used successfully for over a decade to document safety cases for aircraft avionics, rail signaling, air traffic control, and nuclear reactor shutdown Shows how claims are broken down into sub-claims, and eventually supported by evidence or while making clear the argumentation strategies adopted, the rationale for the approach (assumptions, justifications) and the context in which claims are stated A/J 6

8 Example: Battery Exhaustion Part One C1 Pump is safe for use on patients Cx1 Hazards: over infusion, under infusion,... S1 Argue over hazards to safe pump operation S2 Argue over hazards causing over infusion S3 Argue over hazards causing under infusion Cx4 Hazards: exhaustion of battery power, occlusion of line, faulty pump calibration,... C2 The battery exhaustion hazard has been adequately mitigated 7

9 Example: Battery Exhaustion Part Two C2 The battery exhaustion hazard has been adequately mitigated C4 When operating on battery power visual and auditory alarms are launched at least {x} minutes prior to battery exhaustion but no more than {x+y} minutes prior C3 Caregiver is notified sufficiently soon (but not too soon) prior to battery exhaustion S4 Argue over hazards causing failure to notify caregiver in a timely manner Cx3 A late warning won't give the caregiver time to stop current activities and plug the pump in. An early warning may be ignored. This depends on the clinical setting. Cx2 Caregiver doesn't notice alarm; amount of warning time too little for the anticipated clinical setting C5 C6 Visual and auditory alarms are loud enough to be heard and identified in the anticipated clinical setting {x} minutes warning prior to battery exhaustion is sufficient time to allow corrective action in the anticipated clinical setting 8

10 02 The Task We were asked to assure the safety of a system for gu iding aircraft onto ships in bad weather. This was to consider the whole sh ip/equipment/aircraft system of systems, taking into account: Human factors. The operating environment. Operating procedures. Maintenance & Management. An Operational Safety Case (OSC) was needed. Extracted from Assuring Operational Systems A Safety Case Study by Simon Di Nucci, Systems and Software Technology 2015 Carnegie Mellon Conference, University

11 This image cannot currently be displayed. 05 Approach - OSC Safety Strategy +-' >< Q) +-' c 0 () Safety Targets and Report Requirements Definition of flde<j.jately Safe Top Goal System is Pdequately Safe fa" Jli raaft to Locate& Ppproach Ship Shavvall elements and i nteradions are safe ~tern Description Jliraaft Desaiption () 0 :::J r-+ CD >< r-+ Goals ~tern Equipments are Safe to QJerate System is Safe in c:peration />II ~tern Safety Requirements are Satisfied ~tem/jliraaft!sh i p Interactions are Safe Extracted from Assuring Operational Systems A Safety Case Study by Simon Di Nucci, Systems and Software Technology 2015 Carnegie Mellon Conference, University

12 What confidence should be placed on an AC? Given the evidence, how confident should we be in the claim C1? Why? What does it mean to have confidence in the claim? What could be done to improve confidence? Why? 11

13 The Basis for Confidence in a Claim A classic philosophical problem: Justify belief in a hypothesis Use Induction Enumerative: Support increases as confirming instances are found? Using past experience as the basis for predicting future behavior 12

14 Eliminative Induction Support for a claim increases as reasons for doubt are eliminated CLAIM: The light turns on (when the switch is flicked). Bulb OK? Power? Wired?? Confidence increases as doubts are eliminated 13

15 What confidence should be placed on an AC? How confident in C1? Why? (Number of uneliminated doubts) What does it mean to have confidence? (Lack of doubt) What could be done to improve confidence? Why? (Elim. more doubts) C1 The system is safe C2 Hazard A has been eliminated C3 Hazard B has been eliminated Ev1 Evidence Ev2 Evidence Ev3 Evidence 14

16 A Small Example Rebutting defeaters (R) attack claim validity C1.1 The system is secure from sql injection Inference rule (IR) for validating a claim Cx2.1a A parameter is unrestricted if it can cause an unintended modification to the sql query when used R2.1 Unless there is unrestricted user input to a query IR2.2 If user input is properly restricted, then the system is safe from sql injection IR3.1 If all user input is sanitized then there is no unrestricted user input to a query Ev3.2 Evidence showing that all user input is properly sanitized before being used in an sql query UC3.3 Unless there is another way sql injection could occur Undercutting defeaters (UC) attack rule sufficiency Undermining defeaters (UM) attack evidence validity UM4.1 But the evidence is based upon faulty sanitation rules UM4.2 But the evidence is not for the system under consideration IR4.3 If these reasons are not true, then the evidence is valid

17 Key Ideas Confidence grows as doubts are identified and eliminated Doubts about a claim (rebutting defeater) Why claim may be invalid R2.1 Unless there is unrestricted user input to a query Doubts about evidence (undermining defeater) Why evidence may be invalid UM4.1 But the evidence is based upon faulty sanitation rules Doubts about reasoning (undercutting defeater) Premise ok; conclusion uncertain UC3.3 Unless there is another way sql injection could occur 16

18 Other State of the Art John Knight University of Virginia Confidence cases: a confidence argument created in parallel to the safety argument that documents the confidence in the structure and basis of the safety argument. Tim Kelly University of York Evidence elaboration: modeling evidence to better understand it and its evaluation for the purpose of explicit integration of the source data of evidence and the safety case argument. 17

19 Concluding Thoughts This has been a quick overview of assurance cases and confidence and an introduction to the concept eliminative argumentation as developed by the SEI. It is not a comprehensive review of all that is happening in the area. The SEI has been applying Baconian probabilities to confidence maps to show how much different portions of the argument contribute to overall confidence something that may prove useful for incremental certification. Assurance cases have been proven effective in the safety domain. The effectiveness of confidence cases and eliminative induction have yet to be demonstrated in practice. 18

20 Contact Information Charles B. Weinstock Principal Researcher Software Solutions Division Telephone: U.S. Mail Software Engineering Institute 4500 Fifth Avenue Pittsburgh, PA USA 19