Evaluation of the accuracy of a computer aided diagnosis (CAD) system in breast ultrasound according to the radiologist's experience

Transcription

1 Evaluation of the accuracy of a computer aided diagnosis (CAD) system in breast ultrasound according to the radiologist's experience Poster No.: C-1921 Congress: ECR 2011 Type: Scientific Exhibit Authors: M. L. Chabi, R. Ardiles, I. Borget, G. Aboud, S. BOUSSOUAR JACQUET, V. S. Vilar, C. Dromain, C. Balleyguier ; Villejuif/ 2 3 FR, Paris/FR, Villejuif Cedex 05/FR Keywords: Breast, CAD, Ultrasound DOI: /ecr2011/C-1921 Any information contained in this pdf file is automatically generated from digital material submitted to EPOS by third parties in the form of scientific presentations. References to any names, marks, products, or services of third parties or hypertext links to thirdparty sites or information are provided solely as a convenience to you and do not in any way constitute or imply ECR's endorsement, sponsorship or recommendation of the third party, information, product or service. ECR is not responsible for the content of these pages and does not make any representations regarding the content or accuracy of material in this file. As per copyright regulations, any unauthorised use of the material or parts thereof as well as commercial reproduction or multiple distribution by any traditional or electronically based reproduction/publication method ist strictly prohibited. You agree to defend, indemnify, and hold ECR harmless from and against any and all claims, damages, costs, and expenses, including attorneys' fees, arising from or related to your use of these pages. Please note: Links to movies, ppt slideshows and any other multimedia files are not available in the pdf version of presentations. Page 1 of 14

2 Purpose To evaluate the performance of a Computer Aided Diagnosis (CAD) system in breast ultrasound to improve the characterization of breast lesions by junior and senior radiologists. To determine whether a breast ultrasound CAD system might be considered as a relevant teaching tool. Methods and Materials This retrospective study was conducted from November 2008 to march From institutional PACS database, 160 breast lesions visible in ultrasonography were included. Examinations were acquired either with a Sonograph Siemens S2000 or a Siemens Antares system. All lesions were previously biopsied and confirmed either by cytology and/or pathology. Among them, 83 lesions (21 cysts, 60 fibroadenoma, 2 papilloma) were benign and 77 (65 Canalar invasive carcinoma, 11 Lobular invasive carcinoma, 1 apocrin carcinoma) were malignant. All lesions were blindly reviewed by four radiologists with different level of expertise : A, B, C, D with respectively 20 years, 5 years, 12 and 4 month-experience in breast imaging. The four radiologists independently read each case, firstly without CAD and then after a complete first reading of all cases, with the help of an ultrasound CAD system : B-CAD, Version 2, medipattern, Canada. For each case, with and without CAD, evaluation criteria were : - Lesion type according to BI-RADS lexicon - Bi-RADS Category - Malignant probability measured on a 1 to 7 malignancy scale, independent from BiRADS Category Page 2 of 14

3 - Final assessment expressed in terms of probability of a benign or a malignant lesion For each radiologist, were calculated : - Sensitivity (Se) and Specificity (Sp) with and without CAD, based on Mac Nemar test - Percentage of conclusion discrepancies with and without CAD analysis Intrinsic sensitivity and specificity of CAD alone were also calculated Images for this section: Page 3 of 14

4 Fig. 1: Here is an example : - down : breast lesion ultrasound - up : lesion margins outlined by CAD - the right column gives : the automated analysis of breast lesion according to BIRADS features including shape, orientation, margin, lesion boundary, posterior features, echo pattern, calcification, surrounding tissue and then, final assessment of CAD according to BIRADS classification Page 4 of 14

5 Fig. 2: Here is another example Page 5 of 14

6 Results The CAD system detected all of the cancers : sensitivity was of 100%, with a specificity of 48% For the expert radiologist (A), the results show a very high sensitivity of the expert radiologist in detecting breast cancer, without impact of CAD. Sensitivity was 99% in both cases. (figure 1) However, for the expert radiologist (A), the use of CAD system increased false positive's rate : among 83 normal cases (A) classified as benign 58 without CAD and only 38 with CAD. Specificity decreased significantly with CAD from 70% to 46%. (figure 2) As regards intermediate radiologist (B) with 5 year experience, the use of CAD system significantly enhanced sensitivity from 87% to 96%. 7 additional cancer cases were diagnosed thanks to the CAD. (figure 3) As for the expert radiologist, specificity was significantly reduced with CAD, here from 80 to 58%. (figure 4) Regarding less experienced radiologists (C et D), same scheme. A small increase of sensitivity with CAD witch allowed to diagnose several additional cancers (5 for (C level) and 3 for (D level)). (figure 5) And few or no impact on specificity with CAD. (figure 6) Images for this section: Page 6 of 14

7 Fig. 1: Expert radiologist (A level) Page 7 of 14

8 Fig. 2: Expert radiologist (A level) Page 8 of 14

9 Fig. 3: Intermediate radiologist (B level) Page 9 of 14

10 Fig. 4: Intermediate radiologist (B level) Page 10 of 14

11 Fig. 5: LESS EXPERIENCED RADIOLOGIST Page 11 of 14

12 Fig. 6: LESS EXPERIENCED RADIOLOGIST Page 12 of 14

13 Conclusion Overall, the results of this study suggest that : - first, BCAD is a useful tool for cancer detection for intermediate and less experienced radiologists. Related to high sensitivity of CAD (Se 100%), allowing the diagnosis of several additional cancers - and second, BCAD facilitates the learning of breast ultrasound analysis and BIRADS lexicon for junior radiologists However, BCAD appears to have some limitations in this study : - It provides no real-time examination analysis : which might be difficult for everyday practice, because it's a separate computer. - It tends to increase the false-positive rate. This tendency must be taken into account in the analysis. Nevertheless, this fact is common and already known for other CAD systems - We must raise lesion selection bias : because all the selected lesions must had been biopsied. Each biopsy was justify because of the suspicious feature of the lesion or only related to the context, even for lesions classified BI-RADS 3 or 2. In conclusion : Breast ultrasound CAD system appears more useful for junior than senior radiologists. This is due to : - High sensitivity of CAD - its efficiency as a training method for an accurate breast ultrasound interpretation and then could be considered as a teaching tool However, CAD system users should never ignore the false positive rate when giving their conclusions. Maybe, news versions will probably improve this point. References Page 13 of 14

14 1- American College of Radiology. BI-RADS : ultrasound, 1 st ed. In : Breast imaging reporting and data system : BI-RADS atlas 4th ed. Reston, VA : American College of Razdiology Athanasiou A, Tardivon A, Ollivier L, Thiobault F, El Khoury C, Neuenschwander S. How to optimize breast ultrasound. Eur J Radiol 2009 Jan; 69 (1) : Balleyguier C, Kinkel K, Fermanian J, Malan S, Djen G, Taourel P, Helenon O. Computer-aided detection (CAD) in mammography: does it help the junior or the senior radiologist? Eur J Radiol Apr;54(1): Lee HJ, Kim EK, Kim MJ, Youk JH, Lee JY, Kang DR, Oh KK. Observer variability of Breast Imaging Reporting and Data System (BI-RADS) for breast ultrasound. Eur J Radiol Feb;65(2): Epub 2007 May Hong AS, Rosen EL, Soo MS, Baker JA. BI-RADS for sonography: positive and negative predictive values of sonographic features. AJR Am J Roentgenol Apr;184(4): Horsch K, Giger ML, Vyborny CJ, Venta LA. Performance of computer-aided diagnosis in the interpretation of lesions on breast sonography. Acad Radiol Mar;11(3): Singh S, Maxwell J, Baker JA, Nicholas JL, Lo JY. Computer-aided classification of breast masses: performance and interobserver variability of expert radiologists versus residents. Radiology Jan;258(1): Epub 2010 Oct 22 Personal Information Page 14 of 14