Risk Assessment Tools for Identifying Hazards and Evaluating Risks Associated with IVD Assays Robert C. Menson, PhD AACC Annual Meeting Philadelphia, PA 22 July 2003
What Risks Must Be Managed? Risk to safety of patients, users, handlers Business Regulatory Product liability Menson & Associates, Inc AACC 2003 Edutrak 2
Elements of the Risk Management Process Risk Analysis Risk Evaluation Risk Control Assessm ent Management Post-Production Information Menson & Associates, Inc AACC 2003 Edutrak 3
Risk Assessment Tools Risk Matrix FTA=Fault Tree Analysis PHA= Preliminary Hazard Analysis FME(C)A=Failure Mode Effects (Criticality) Analysis HAZOP=Hazard Operability Analysis HACCP=Hazard Analysis and Critical Control Point Menson & Associates, Inc AACC 2003 Edutrak 4
Applications of Risk Analysis Design Production Premarket Notifications Complaints MDR Change Control Failure Analysis Etc. Menson & Associates, Inc AACC 2003 Edutrak 5
Risk Management Life Cycle Concept & Feasibility Development Scale-Up & Transfer Production System Level Assessment Design Assessment Process Assessment Customer Feedback Assessment Menson & Associates, Inc AACC 2003 Edutrak 6
Risk Management & Design Control Concept & Feasibility Planning Development Scale-Up & Transfer Production Design Control Requirements Plan Specifications Test Methods & Results Productions Methods Change Records Risk Assessment Preliminary Hazard Analysis Risk Management Plan Detailed Analysis (FMEA, FTA, HACCP, HAZOP) Risk Management Report Risk Reviews Menson & Associates, Inc AACC 2003 Edutrak 7
Key Concepts of Risk The frequency of the potential harm; How often the loss may occur; The consequences of that loss; How large the loss might be; The perception of the loss; How seriously the stakeholders view the risk that might affect them. Menson & Associates, Inc AACC 2003 Edutrak 8
Risk Management Terms Intended Use/Purpose Harm Hazard Risk Residual Risk Tolerable Risk Menson & Associates, Inc AACC 2003 Edutrak 9
Intended Use/Purpose Use of a Product, Process or Service in accordance with the specifications, instructions and information provided by the manufacturer ANSI/AAMI/ISO 14971:2000, definition 2.5 Menson & Associates, Inc AACC 2003 Edutrak 10
Harm Physical injury or damage to health of people, or damage to property or the environment. ISO/IEC Guide 51:1999, definition 3.3 Guidelines for inclusion of safety aspects in standards. Menson & Associates, Inc AACC 2003 Edutrak 11
Hazard Potential source of Harm ISO/IEC Guide 51:1999, definition 3.5 Menson & Associates, Inc AACC 2003 Edutrak 12
Risk Combination of the probability of occurrence of harm and the severity of harm. ISO/IEC Guide 51:1999, definition 3.2 Menson & Associates, Inc AACC 2003 Edutrak 13
Residual Risk Risk remaining after protective measures have been taken. ISO/IEC Guide 51:1999, definition 3.9 Menson & Associates, Inc AACC 2003 Edutrak 14
Tolerable Risk Risk which is accepted in a given context based on the current values of society. ISO/IEC Guide 51:1999, definition 3.7 Menson & Associates, Inc AACC 2003 Edutrak 15
Hazard Identification Laws Codes Standards Hazard Identifcation Events MDRs Accidents Etc List of Hazards Brainstorming PHA FTA FMEA Menson & Associates, Inc AACC 2003 Edutrak 16
Risk Matrix Frequency Severity Negligible Minor Major Severe Frequent L I H H Probable L I H H Occasional T I I H Remote T L I I Menson & Associates, Inc AACC 2003 Edutrak 17
ALARP Curve Increasing Probability of Occurrence Broadly Acceptable Region Intolerable Region ALARP Increasing Severity of Harm Maximum Tolerable Risk Menson & Associates, Inc AACC 2003 Edutrak 18
FMEA vs FTA FMEA Assumes component or part failure Identifies functional failure as a result of part failure FTA Assumes failure of the functionality of a product Identifies part/module failure as cause of functional failure Menson & Associates, Inc AACC 2003 Edutrak 19
FMEA vs. FTA FMEA Done for entire design Systematic way to predict new problems A bottoms-up analysis FTA Too difficult to do for entire design Systematic way to predict causes for usually know problems A top down analysis People expect the same results from FTA which is not true People do not expect the same results from and FMEA Often a fault tree is used for a problem or an accident. Menson & Associates, Inc AACC 2003 Edutrak 20
FTA Assumes fault and analyzes possible causes Connection tool for PHA* to subsystems or modules Top down Deductive Evaluate system (or subsystem) failures Considered more structured than FMEA Graphical presentation--visual picture *Preliminary Hazard Analysis Menson & Associates, Inc AACC 2003 Edutrak 21
Failures Primary--Due to internal causes that include poor design or use of inappropriate materials Secondary--Due to failures in the operation that include equipment failure Control--Due to failures in the systems that are in place to protect the quality and safety e.g. raw material outside specification failure of safety switch failure of test method Menson & Associates, Inc AACC 2003 Edutrak 22
FTA Basic Symbols Basic Flow FAULT Fault in a box indicates that it is a result of subsequent faults OR Connects a preceding fault with a subsequent fault that could cause a failure AND Connects two or more faults that must occur simultaneously to cause the preceding fault Menson & Associates, Inc AACC 2003 Edutrak 23
FTA During Design POOR FIT IN H AN D OR ASPECT RATIO TOO HIGH MAJOR DIAMETER TOO LARGE Competing Design Criteria ASPECT RATIO TO SMALL DOES NOT RESTRICT SPILLAGE OR BASE FLATNESS BASE SIZE TOO SMALL RIM TO FILL INSUFFCIENT Menson & Associates, Inc AACC 2003 Edutrak 24
FTA Lab Failure Other Outliers Patient Death OR Calibration Error Lab Error OR Systematic OR Interference Other Random Menson & Associates, Inc AACC 2003 Edutrak 25
FTA During Reliability HAZARD OR 4. x 10-9 SYSTEM FAILURE DRIFT> LIMIT AND 1. x 10-16 OR 4. x 10-9 CMPT A FAILS CMPT B FAILS CMPT C DRIFTS REFERENCE DRIFTS 5. x 10-9 2. X 10-8 3. x 10-9 1. x 10-9 Menson & Associates, Inc AACC 2003 Edutrak 26
What is FMEA? FMEA Failure Mode and Effects Analysis FMECA Failure Mode Effects and Criticality Analysis Menson & Associates, Inc AACC 2003 Edutrak 27
What is FMEA? Powerful prioritization tool Inductive High effective tool for identifying critical quality attributes High structured Methodical Breaks large complex designs into manageable steps Menson & Associates, Inc AACC 2003 Edutrak 28
FMEA Bottom up approach Evaluates specific failures Detailed analysis tool Use in conjunction with PHA and FTA Complements FTA May lead to different failure results Menson & Associates, Inc AACC 2003 Edutrak 29
Advantages of FMEA Less analyst dependent than FTA Allows direct criticality assessment of components Valuable troubleshooting aid Identifies areas of weak design Identifies areas of high risk Prevention planning Identifies change requirements Menson & Associates, Inc AACC 2003 Edutrak 30
Disadvantages of FMEA Does not consider operator error Tedious May not apply to all systems--especially software May require extensive testing to gain information May miss some failure modes Time pressures Information missing Menson & Associates, Inc AACC 2003 Edutrak 31
Definitions Criticality -- Weighting of hazard severity with the probability of failure Severity -- Seriousness of effect through its impact of the system function Occurrence -- Likelihood a specific failure will be caused by a specific cause under current controls Verification -- Ability of the current evaluation technique to detect potential failure during design Detection -- Ability of the current manufacturing controls to detect potential failure before shipping Menson & Associates, Inc AACC 2003 Edutrak 32
Definitions Risk Priority Number (RPN) = (S) x (O) x (D) or (V) Severity (S) Likelihood of occurrence (O) Likelihood of detection (D) Likelihood of verification (V) Menson & Associates, Inc AACC 2003 Edutrak 33
FMEA Example POTENTIAL FAILURE MODE AND EFFECTS ANALYSIS Imunnoassay Device FMEA Number Subsystem Page of X Component _Support Responsibility Prepared By Process FM EA D ate (O rig.) (rev.) Core Team: Device/ Potential Potential S Potential O Current D R Recommended R esponsibilit y Action Results Function Failure Effect(s) Cause(s) Controls P Action(s) and Target A ctions R Mode of Failure of Failure N C omplete Date Taken S O D P N Support System Holds Antibody - Separates A nalyte Fails to Separate analyte No Answer Erroneous Result False Hi False Lo 4 7 9 Wrong Material 3 3 3 5 5 5 60 105 135 Develops Color Menson & Associates, Inc AACC 2003 Edutrak 34
Process FMEA Identifies potential product-related process failure modes Assesses the potential customer effects of the failures Identifies the potential internal and external manufacturing or assembly process causes Identifies process variables on which to focus controls for reducing occurrence or increasing detection of the failure conditions Menson & Associates, Inc AACC 2003 Edutrak 35
Sources of Process Defects? Omitted processing Processing errors Errors setting up work pieces Missing parts Wrong parts Adjustment error Processing wrong work piece Mis-operation Equipment not set up properly Tools and fixtures improperly prepared Menson & Associates, Inc AACC 2003 Edutrak 36
FMEA Summary Powerful tool for summarizing: Important modes of failure Factors causing these failures Effects of these failures Risk prioritization Identifying plan to control and monitor Cataloging risk reduction activities Menson & Associates, Inc AACC 2003 Edutrak 37
HAZOP Haz ard and Op erability Study Bottom up analysis Deviations from design intentions Systematic brainstorming based on guide words Menson & Associates, Inc AACC 2003 Edutrak 38
HAZOP Guide Words No/Not More Less As well as Other than Menson & Associates, Inc AACC 2003 Edutrak 39
Hazop Model Design Statement Activity Material Destination Transfer Powder Hopper Menson & Associates, Inc AACC 2003 Edutrak 40
HAZOP Transfer Powder Hopper No Valve closed Tank empty Valve closed Line blocked Hopper full Pump broken More Pump fast Larger tank Inaccurate gage Other than Liquid Wrong powder Menson & Associates, Inc AACC 2003 Edutrak 41
HAZOP Plan Guide Deviation Causes Risk Action Who NO Powder flow Valve closed Low Interlock Line blocked Pump broken Med Med Operator training PM Menson & Associates, Inc AACC 2003 Edutrak 42
HACCP Risk Management System Biological Hazards Chemical Hazards Physical Hazards Requires Prerequisite Quality System Program Traditionally GMPs Menson & Associates, Inc AACC 2003 Edutrak 43
HACCP Steps 1. Conduct hazard analysis and identify preventive measures 2. Identify Critical Control Points. 3. Establish critical limits 4. Monitor each critical control point 5. Establish corrective action to be taken when deviation occurs 6. Establish verification procedures 7. Establish record-keeping system Menson & Associates, Inc AACC 2003 Edutrak 44
HACCP Decision Tree Preventive Measures Exist Modify Step Yes No Yes Step Eliminate or Reduce No Is control Necessary for Safety Excess Contamination Possible No Yes Yes Subsequent Steps eliminate or Reduce Yes No Critical Control Point No Stop Not a CCP Menson & Associates, Inc AACC 2003 Edutrak 45
HACCP Worksheet Firm Name: Product Description: Firm Address: Method of Storage and Distribution Intended Use and Consumer 1 Material/ processin g step 2 Identify potential hazards introduced, controlled or enhanced at this step(1) Biological 3 Are any potential safety hazards significant? (Y/N) 4 Justify your decisions for column 3. 5 What preventative measures can be applied to prevent the significant hazards? 6 Is this step a critical control point? (Y/N) Chemical Physical Menson & Associates, Inc AACC 2003 Edutrak 46
HACCP Plan Firm Name: Firm Address Product Description: Method of Storage and Distribution: Intended use and Consumer: (1) Critical Control Point (2) Significan t Hazards (3) Critical Limits for each Action Monitoring (8) Corrective Actions (9) Record s (10) Verification (4) (5) (6) (7) What How Frequency Who Menson & Associates, Inc AACC 2003 Edutrak 47
Contact Information Robert C. Menson, Phd Menson & Associates, Inc Quality Systems and Compliance For FDA Regulated Industries PO Box 480 Newport, RI 02840 Phone: 1.401.847.6320 Email: r.menson@alumni.duke.edu Menson & Associates, Inc AACC 2003 Edutrak 48
IVD Hazards, Patient Harm and Risk Controls Donald M. Powers, PhD AACC Annual Meeting Philadelphia, PA 22 July 2003 Menson & Associates, Inc AACC 2003 Edutrak 49