Summary These externally validated performance evaluations confirm that the results obtained with the cobas b 123 POC system for 17 commonly assessed critical care parameters are as true and accurate as those obtained from well-recognized and widely used analyzers, such as the cobas b 221 analyzer widely regarded as the gold standard BGE and the cobas 6 analyzer series. The is an attractive and proven solution for BGE at the point of care, designed for performance at low risk to the hospital. Performance evaluation Introduction This proof-of-performance white paper demonstrates that the cobas b 123 POC (point-of-care) system is as true and accurate in its measurement of 17 critical care parameters (see below) as well-established and widely used analyzers, such as the cobas b 221 analyzer, and the cobas 6 analyzer series. In order to demonstrate this, the document details the externally validated performance evaluations conducted with this POC system in order that potential users can have complete confidence that the results obtained for the critical care parameters assessed using this system are equivalent to those obtained with other well-recognised, widely-used analyzers. 17 parameters evaluated with the cobas b 123 POC system: ph, blood gas (BG: po2, pco2), electrolytes (ISE: Na, K, Cl, Ca), metabolites (Glu and Lac), COOX (hematocrit (Hct), total hemoglobin (thb), oxygen saturation (SO2), hemoglobin derivates (O2Hb, HHb, COHb, MetHb)) and bilirubin (total). Key conclusions The acceptance criteria for trueness and accuracy and the acceptance criteria repeatability and intermediate precision with all sites combined were fulfilled. These externally validated performance evaluations confirm that the results obtained with the for 17 commonly assessed critical care parameters are as true and accurate as those obtained from well-recognized and widely used analyzers, such as the cobas b 221 analyzer widely regarded as the gold standard BGE and the cobas 6 analyzer series. The is an attractive and proven solution for BGE at the point of care, designed for performance at low risk to the hospital. * For investigational use only in the U.S. CAUTION Investigational device limited by Federal (or United States) Law to investigational use The performance characteristics of this product have not been established. Not available in the U.S. pending FDA clearance. COBAS, COBAS B, COBAS C and LIFE NEEDS ANSWERS are trademarks of Roche. 211 Roche Roche Diagnostics Ltd. CH-6343 Rotkreuz Switzerland www.cobas.com
Overview of methods for external proof of performance evaluations performed for the cobas b 123 POC system The objective of the external performance evaluation studies was to investigate the performance of cobas b 123 POC system against widely-used and validated analyzers. Performance evaluation study: a. Method comparisons of the with the for ph, pco2, po2, Na, K, Cl, Ca, Hct, O2Hb, HHb, COHb, MetHb, thb and SO2, and compared with cobas 6 analyzer series for glucose and lactate. These were performed externally at six sites in Europe [see below] (2 residual samples per lot; 1 per site and lot). b. Determination of the intermediate precision of the QC results (daily quality control) over 21 days performed externally at six sites in Europe for three different lots at the same test sites [Note; this excluded bilirubin]. The six European evaluation sites were: Ludwigshafen, Hamburg, Graz, Zwolle, Göteborg and Cambridge. External performance evaluation study, bilirubin: a. Method comparisons of the for bilirubin compared with the cobas c 51 module (part of the cobas 6 analyzer series) performed externally at three US sites (see below). b. Determination of the intermediate precision of the QC results (daily quality control) over 21 days performed externally in three US sites.* The three US evaluation sites were: Galveston, TX, Louisville, KY and San Diego, CA. Additional information on the studies Heparinised arterial or venous whole blood samples were used throughout and fresh arterial, venous or capillary lithium heparin blood samples for the bilirubin evaluation. For determination of results within the cobas b 123 POC system, 15μL samples were required unless otherwise stated. Quality controls were performed on each measuring day of the separate study phases. Samples were measured with both of the used sensor cartridge lots in three different QC levels over 21 days in duplicate determinations (Intermediate precision; daily quality control). In situations where over 21 measuring days were needed, they were tested twice daily once only. A familiarisation phase was performed prior to the studies with one lot only to ensure that the study operators were suitably familiar with the handling and procedures of the new cobas b 123 POC analyser. Results for each of the 17 parameters assessed externally for the in external proof of performance evaluations The following pages summarise the main results obtained in the performance evaluations when each of the 17 parameters [see page 1] analyzed with the were compared against the reference analyzer [as stated within the results of each parameter]. Parameter: ph Reference analyzer: n = 691, slope =.99, intercept =.8, r =.99, median absolute bias =.9, accuracy 1 % 6.5 7.1.5.3.13.7 7.1 7.6.3.4.7.8 7.6 8.4.6.8.18 Table 1. Acceptance criteria for ph method comparison with blood samples outside the typical range ph 1 Potentiometry 8. 7.9 7.8 7.7 7.6 7.5 7.4 7.3 7.2 7.1 7. 7. 7.1 7.2 7.3 7.4 7.5 7.6 7.7 7.8 7.9 8. ph 1 Potentiometry Figure 1. Trueness overall and accuracy overall: ph Key point: The was shown to be equivalent to the cobas b 221 analyzer for assessment of ph for trueness overall and accuracy overall in this externally validated performance evaluation in 691 samples. Y =.991.1.5. -.5 -.1 X = +.78 md(95) =.15 7. 7.2 7.4 7.6 7.8 8. ph 1 Potentiometry N = 691 r =.9865 t =.8943
Blood gases Parameter: po2 (mmhg) Reference analyzer: n = 677, slope =.97, r = 1., median absolute bias = -3.1 mmhg, accuracy 98 % Parameter: pco2 (mmhg) Reference analyzer: n = 687, slope =.95, r =.99, median absolute bias = -.2 mmhg, accuracy 1 % 1 5 5 3.5 15 9.1 5 14 3.5 5 9.1 1.6 14 7 5 42.7 1.6 99.9 Table 2. Acceptance criteria for po2 (mmhg) method comparison with blood samples outside the typical range 1 25 4.75 2.5 12.25 5.5 25 7 2.5 2.9 5.5 6.7 7 15 2.9 23.9 6.7 4.9 Table 3. Acceptance criteria for pco2 (mmhg) method comparison with blood samples outside the typical range Y =.9864 X = - 1.8951 md(95) = - 3.1 N = 677 r =.9963 t =.9587 Y =.953 X = + 1.855 md(95) = 1.36 N = 687 r =.9933 t =.9162 po2 3 Amperometry 6 5 4 3 2 1 1 2 3 4 5 6 1. 8. 6. 4. 2.. -2. -4. -6. -8. -1. 1 2 3 4 5 6 pco2 4 Potentiometry 12 1 8 6 4 2 2 4 6 8 1 12 8. 4.. - -4. - -8. 2 4 6 8 1 12 po2 3 Amperometry po2 3 Amperometry pco2 4 Potentiometry pco2 4 Potentiometry Figure 2. Trueness overall and accuracy overall: po2 Figure 3. Trueness overall and accuracy overall: pco2 POC system and the for po2 was good in the 677 samples assessed in this externally validated performance evaluation, and the accuracy for po2 was 98 %. POC system and the for pco2 was good, and the accuracy for pco2 was 1 % in 687 samples assessed in this externally validated performance evaluation.
Electrolytes Parameter: sodium (mmol/l) Reference analyzer: n = 691, slope = 1.3, r =.94, mean absolute bias =.2 %, accuracy 1 % Parameter: potassium (mmol/l) Reference analyzer: n = 682, slope =.96, r =.99, mean absolute bias = -.3 mmol/l, accuracy 99.9 % 1 12 5 2.5 11 6.1 12 17 2.5 4.25 6.1 1.25 17 2 4.25 5 1.25 14 Table 4. Acceptance criteria for sodium (mmol/l) method comparison with blood samples outside the typical range 1 3.5.2 1.1.4 3 7.2.28.4.63 7 15.28 1.63 2.6 Table 5. Acceptance criteria for potassium (mmol/l) method comparison with blood samples outside the typical range Y = 1.34 X = - 4.568 md(95) = 2.365 N = 691 r =.9396 t =.76 Y =.956 X = +.146 md(95) = -.85 N = 682 r =.9947 t =.945 2 Correction factor used! 1. 19 18 9. 8. Na 6 Potentiometry 17 16 15 14 13 12 11 1 1 11 12 13 14 15 16 17 18 19 2 (Y X) / X [%] 8. % % 4. % %. % -% -4.% -% -8.% 1 11 12 13 14 15 16 17 18 19 2 K 7 Potentiometry 7. 5. 4. 3. 1... 1. 3. 4. 5. 7. 8. 9. 1. 1..6.2 -.2 -.6-1.. 1. 3. 4. 5. 7. 8. 9. 1. Na 6 Potentiometry Na 6 Potentiometry K 7 Potentiometry K 7 Potentiometry Figure 4. Trueness overall and accuracy overall: sodium Figure 5. Trueness overall and accuracy overall: potassium Key point: Overall, the method comparison of the vs. showed comparable results for Na, with 1 % of the values within the 95 percentile. Key point: Overall, the method comparison of the vs. showed a close correlation for K, with 1 % of the values within the 95 percentile.
Parameter: chloride (mmol/l) Reference analyzer: n = 657, slope = 1.15, r =.98, mean absolute bias = 1.17 %, accuracy 1 % Parameter: calcium (mmol/l) Reference analyzer: n = 689, slope =.95, r =.94, mean absolute bias = -.1 mmol/l, accuracy 1 % 7 95 4 4 9.14 8.75 95 12 4 4.8 8.75 1.8 12 15 4.8 6 1.8 13.8 Table 6. Acceptance criteria for chloride (mmol/l) method comparison with blood samples outside the typical range.1.5.15.1.25.2.5 1.5.1.9.2.24 1.5 2.5.9.25.24.51 Table 7. Acceptance criteria for calcium (mmol/l) method comparison with blood samples outside the typical range Y = 1.152 X = - 14.913 md(95) = 1.32 N = 657 r =.9832 t =.8821 Y =.948 X = +.48 md(95) =.53 N = 689 r =.9394 t =.7556 15 Correction factor used! 2.5 Correction factor used! Cl 9 Potentiometry 14 13 12 11 1 9 8 (Y X) / X [%] 15.% 1.% 5.%.% -5. % -1.% Ca 8 Potentiometry 1.5 1..5.3.2.1. -.1 -.2 7 7 8 9 1 11 12 13 14 15-15.% 7 8 9 1 11 12 13 14 15...5 1. 1.5 2.5 -.3..5 1. 1.5 2.5 Cl 9 Potentiometry Cl 9 Potentiometry Ca 8 Potentiometry Ca 8 Potentiometry Figure 6. Trueness overall and accuracy overall: chloride Figure 7. Trueness overall and accuracy overall: calcium Key point: Overall, the method comparison of the vs. showed a close correlation for Cl, with 1 % of the values within the 95 percentile. Key point: Overall, the method comparison of the vs. showed comparable results for Ca, with 1 % of the values within the 95 percentile.
Metabolites Parameter: glucose (mmol/l) Reference platform: cobas 6 analyzer series n = 686, slope =.98, r =.98, median relative bias = -2.4 %, accuracy = 99.7 % Parameter: lactate (mmol/l) Reference platform: cobas 6 analyzer series n = 59, slope = 1., r =.99, median relative bias =.6 %, accuracy = 1 % 1 4.5.5.8.9 4 25.5 2.5.9 5 25 3 2.5 3.27 5 6.87 Table 8. Acceptance criteria for glucose (mmol/l) method comparison with blood samples outside the typical range 1 2.5.5.5.8.8 2.5 12.5 1.2.8 2.4 12 2 1.2 2.4 2.4 5.2 Table 9. Acceptance criteria for lactate (mmol/l) method comparison with blood samples outside the typical range Y =.982 X = -.41 md(95) =.581 N = 686 r =.9794 t =.8698 Y = 1.33 X = -.2 md(95) =.6 N = 59 r =.9816 t =.8666 GLUC 18 Photometry 25 2 15 1 5 5 1 15 2 25 (Y X) / X [%] 3.% 2.% 1.%.% -1.% -2.% -3.% 5 1 15 2 25 LAC 3 Amperometry 2 18 16 14 12 1 8 6 4 2 2 4 6 8 1 12 14 16 18 2 4.. - -4. - 2 4 6 8 1 12 14 16 18 2 GLUC 18 Photometry cobas 6 analyzer series GLUC 18 Photometry cobas 6 analyzer series LAC 3 Photometry cobas 6 analyzer series LAC 3 Photometry cobas 6 analyzer series Figure 8. Trueness overall and accuracy overall: glucose Figure 9. Trueness overall and accuracy overall: lactate Key point: Overall, there was good correlation between the and the cobas 6 analyzer series for glucose in the 686 samples evaluated, with the correlation coefficient for all lots being.98 for glucose. The bias of -2.4 % for glucose on the was within the acceptable range. Key point: Overall, there was good correlation between the cobas 6 analyzer series and the for lactate in the 59 samples evaluated, with the correlation coefficient for all lots being 1. for lactate and no samples falling outside the 95 % accuracy limit.
Hematocrit Parameter: hematocrit (%) Reference analyzer: n = 691, slope.95, r =.99, mean absolute bias = -.1 %, accuracy = 99.7 % Total hemoglobin Parameter: total hemoglobin (thb) (g/dl) Reference analyzer: n = 682, slope.95, r =.98, mean absolute bias =.3 g/dl, accuracy = 99.3 % 1 25 6 3 14 7 25 6 3 3 7 7 6 75 3 6 7 14 Table 1. Acceptance criteria for hematocrit (%) method comparison with blood samples outside the typical range 4 1.5.5 1.2 1.1 1 2.5.5 1.1 1.1 2 25.5 1.2 1.1 2.2 Table 11. Acceptance criteria for total hemoglobin (thb) (g/dl) method comparison with blood samples outside the typical range Hct 5 Conductivity 6 5 4 3 2 1 1 2 3 4 5 6 Hct 5 Conductivity Figure 1. Trueness overall and accuracy overall: hematocrit Y =.949 1. - - -1. X = + 1.62 md(95) = 1.19 1 2 3 4 5 6 Hct 5 Conductivity N = 691 r =.9891 t =.9227 thb Coox 1 Spectro-photometrical 2 18 16 14 12 1 8 6 4 2 5 1 15 2 thb Coox 1 Spectro-photometrical Figure 11. Trueness overall and accuracy overall: total hemoglobin (thb) Y =.947 4. 3. 1.. -1. - -3. -4. X = +.252 md(95) =.444 N = 682 r =.9812 t =.8958 5 1 15 2 thb Coox 1 Spectro-photometrical POC system and the for hematocrit was good, and the accuracy for hematocrit was almost 1 %. POC system and the for thb was good, and the accuracy for thb was 99.3 %, with only few outliers.
Hemoglobin derivates Parameter: COHb (%) Reference analyzer: n = 682, slope 1., r =.97, mean absolute bias =. %, accuracy = 1 % Parameter: HHb (%) Reference analyzer: n = 668, slope 1.1, r =.99, mean absolute bias = -.6 %, accuracy = 98.8 % 2 1 3 2 4 2 7 3 7 4 8.4 Table 12. Acceptance criteria for COHb (%) method comparison with blood samples outside the typical range 7 2 2 5 5 Table 13. Acceptance criteria for HHb (%) method comparison with blood samples outside the typical range COHb 13 Spectro-photometrical 1. 9. 8. 7. 5. 4. 3. 1... 4. 8. 1. Y = 1. 2.5 1.5.5 -.5-1.5-2.5 X = +. md(95) =.212 N = 682 r =.9678 t =.7687. 4. 8. 1. HHb 16 Spectro-photometrical 7 6 5 4 3 2 1 1 2 3 4 5 6 7 Y = 1.14 1. - - -1. X = -.721 md(95) = 1.41 N = 668 r =.9975 t =.983 1 2 3 4 5 6 7 COHb 13 Spectro-photometrical COHb 13 Spectro-photometrical HHb 16 Spectro-photometrical HHb 16 Spectro-photometrical Figure 12. Trueness overall and accuracy overall: COHb Figure 13. Trueness overall and accuracy overall: HHb POC system and the for COHb was excellent. POC system and the for HHb was excellent, with a slight negative bias due to some outliers.
Parameter: MetHb (%) Reference analyzer: n = 682, slope 1., r =.67, mean absolute bias =.1 %, accuracy = 1 % Parameter: O2Hb (%) Reference analyzer: n = 666, slope 1., r =.99, mean absolute bias =.5 %, accuracy = 99.4 % 1 1 3 2 4 1 7 3 7 4 11.4 Table 14. Acceptance criteria for MetHb (%) method comparison with blood samples outside the typical range 3 1 3 3 6 6 Table 15. Acceptance criteria for O2Hb (%) method comparison with blood samples outside the typical range Y = 1. X = +.1 md(95) =.212 N = 682 r =.6665 t =.5692 Y = 1.7 X = -.165 md(95) = 1.425 N = 666 r =.9973 t =.918 MetHb 14 Spectro-photometrical 1. 9. 8. 7. 5. 4. 3. 1.. 1 2 3 4 5 6 7 8 9 1 MetHb 14 Spectro-photometrical 3. 1.. -1. - -3...5 1. 1.5 2.5 3. MetHb 14 Spectro-photometrical O2Hb 12 Spectro-photometrical 1 9 8 7 6 5 4 3 3 4 5 6 7 8 9 1 O2Hb 12 Spectro-photometrical 1. - - -1. 3 4 5 6 7 8 9 1 O2Hb 12 Spectro-photometrical Figure 14. Trueness overall and accuracy overall: MetHb Figure 15. Trueness overall and accuracy overall: O2Hb POC system and the for MetHb was good. The Pearson s coefficient is of limited value here. POC system and the for O2Hb was excellent, with good accuracy and very few outliers.
Oxygen saturation Parameter: SO2 (%) Reference analyzer: n = 666, slope 1.1, r =.99, mean absolute bias =.6 %, accuracy = 99.2 % Total bilirubin (neonatal) Parameter: total bilirubin (neonatal blood) (mg/dl) Reference analyzer: cobas c 51 module (part of cobas 6 analyzer series) n = 139, slope 1.26, r =.92, accuracy = 95.7 %, mean overall pooled bias = -.5, median overall pooled bias = -.8 3 1 2 2 6 6 Table 16. Acceptance criteria for SO2 (%) method comparison with blood samples outside the typical range 3 3 1.2 1.2 3 3 3 5 1.2 5 3 11 Table 17. Acceptance criteria for total bilirubin (neonatal blood) (mg/dl) method comparison with blood samples outside the typical range Y = 1.15 X = -.737 md(95) = 1.485 N = 666 r =.9973 t =.975 Y =.9816 X = +.2769 md(95) = -.112 N = 148 r =.9855 t =.8666 1 5 SO2 11 Specto-photometrical 9 8 7 6 5 4 3 3 4 5 6 7 8 9 1 1. - - -1. 3 4 5 6 7 8 9 1 Bili Photometry 45 4 35 3 25 2 15 1 5 5 1 15 2 25 3 35 4 45 5 15. 1. 5. -5. -1. -15. 5 1 15 2 25 3 35 4 45 5 SO2 11 Specto-photometrical SO2 11 Specto-photometrical Doumas et al. Doumas et al. Figure 16. Trueness overall and accuracy overall: SO2 Figure 17. Trueness overall and accuracy overall: pooled data from three US sites, bilirubin POC system and the for SO2 was excellent, with good accuracy and very few outliers. Key point: Generally, there was a good agreement between bilirubin concentrations measured between and cobas c 51 module (part of cobas 6 analyzer series). The slope of the regression lines were close to 1, and no significant systematic bias between the two test methods was observed