1 Handheld Radiofrequency Spectroscopy for Intraoperative Margin Assessment During Breast-Conserving Surgery Assessment Program Volume 28, No. 4 August 2013 Executive Summary Background Breast-conserving surgery as part of the treatment of localized breast cancer is optimally achieved by attaining margins around the surgical resection that are free from tumor cells. Failure to achieve clear margins will often require additional surgery to re-excise breast tissue. Currently, histologic examination of excised tissues after completion of surgery is the only method of definitively determining whether clear margins were achieved. Methods such as specimen imaging, frozen section pathology, or touch-print cytology are either not optimally accurate, require considerable time and resources, or are not commonly available. MarginProbe is a device based on the principles of dielectric spectroscopy that characterizes tissue that the device comes in contact with. Cancer cells and normal breast tissues produce different signals. A handheld probe is applied to a small area of the resected surgical specimen and analyzes the tissue as to whether it is likely malignant or benign. Using the MarginProbe device is intended to increase the probability that the surgeon will achieve clear margins in the initial operation, thus, avoiding the need for a second surgery to excise more breast tissue. Objective The purpose of this Assessment is to evaluate the evidence regarding the use of handheld radiofrequency spectroscopy for intraoperative margin assessment (i.e., MarginProbe ) in improving health outcomes for patients undergoing breast-conserving surgery for localized breast cancer. Search Strategy We searched MEDLINE via PubMed in July 2013 using the search term MarginProbe. The web site of the U.S. Food and Drug Administration (FDA) was searched for documents relating to the MarginProbe device. Selection Criteria Due to changes in the classification algorithm of the device, only one study, presented in FDA documents, represents current performance and met selection criteria. BlueCross BlueShield Association An Association of Independent Blue Cross and Blue Shield Plans Main Results One clinical trial enrolled 596 patients randomized to either using the MarginProbe device with standard intraoperative assessment of margins or standard intraoperative assessment alone. The trial was designed with an endpoint called complete surgical resection, in which all positive margins on final pathologic examination in the final main lumpectomy specimen should have been additionally excised or noted if not excised. This outcome appears to be biased against the control arm of the study, and may not be fully clinically relevant because there was no requirement that the NOTICE OF PURPOSE: TEC Assessments are scientific opinions, provided solely for informational purposes. TEC Assessments should not be construed to suggest that the Blue Cross Blue Shield Association, Kaiser Permanente Medical Care Program or the TEC Program recommends, advocates, requires, encourages, or discourages any particular treatment, procedure, or service; any particular course of treatment, procedure, or service; or the payment or non-payment of the technology or technologies evaluated Blue Cross and Blue Shield Association. Reproduction without prior authorization is prohibited. 1
2 additional excision produce a clear surgical margin. This outcome was achieved in 71.8% of the MarginProbe arm and 22.4% of the control arm and was statistically significant. However, the trial design had several deficiencies which preclude full understanding of the results. How control patients were clinically evaluated for margin status is not completely described. The sequence of shavings either performed before randomization, in response to MarginProbe readings, or in response to clinical evaluation is not known, and thus, it is less than clear what can be attributed to MarginProbe assessment. The amount of tissue removed in the MarginProbe arm was greater than the control arm. After initial surgery, 31% of MarginProbe subjects had positive margins, and 42% of control subjects had positive margins. The re-excision rate was 23.8% in the MarginProbe group and 28.5% in the control group, a difference that was not statistically significant. The diagnostic characteristics of the MarginProbe device in detecting cancer in the main lumpectomy specimen, using pathologic examination as the reference standard and the surgical margin as the unit of analysis, showed that MarginProbe was 75.3% sensitive and 46.4% specific. Patient outcomes beyond the re-excision rate were not reported. Author s Conclusions and Comment The clinical trial of MarginProbe did not show a statistically significant difference in the only outcome having direct clinical relevance there was a 5% absolute difference in the re-excision rate. A lower re-excision rate would constitute a health benefit if the local recurrence rate is equivalent or less, an outcome not evaluated in this study. Therefore, the benefit of MarginProbe is uncertain due to the trial being insufficient to demonstrate improvement in the net health outcome. Use of MarginProbe results in more tissue being removed during the initial surgery, which is of uncertain health significance. The study shows that the diagnostic characteristics of MarginProbe in assessing presence of cancer at specimen margins is moderately sensitive and less than 50% specific. Positive tests have a 24% probability of being true positive, and 25% of positive margins will be missed by the device. The clinical trial assessed MarginProbe against a single comparative strategy that included gross inspection and radiologic imaging of the specimen. Based on the available evidence, the Blue Cross and Blue Shield Association Medical Advisory Panel (MAP) made the following judgments about whether handheld radiofrequency spectroscopy for intraoperative margin assessment during breast-conserving surgery (i.e., MarginProbe ) meets the Blue Cross and Blue Shield Association Technology Evaluation Center (TEC) criteria. 1. The technology must have final approval from the appropriate governmental regulatory bodies. In January 2013, MarginProbe received premarket application (PMA) approval from the U.S. Food and Drug Administration (FDA) as an adjunctive tool for identification of cancerous tissue at the margins of the main lumpectomy specimen. 2. The scientific evidence must permit conclusions concerning the effect of the technology on health outcomes. The evidence does not permit conclusions concerning the effect of the technology on health outcomes. There is one randomized trial comparing use of MarginProbe to a standard-of-care assessment for surgical margins with its current classification algorithm. The principal outcome of the study was biased against the standard-of-care arm. There was an absolute reduction of 5% in re-excision rates in the MarginProbe arm, but it was not statistically significant. A greater amount of breast tissue was removed in the MarginProbe arm. No further outcomes were assessed beyond the re-excision rate. The diagnostic performance characteristics of the MarginProbe system showed moderate sensitivity of approximately 75% and poor specificity of 46% Blue Cross and Blue Shield Association. Reproduction without prior authorization is prohibited.
3 Handheld Radiofrequency Spectroscopy for Intraoperative Margin Assessment 3. The technology must improve the net health outcome. It has not been demonstrated that the net health outcome was improved. The re-excision rates were neither clinically meaningful nor statistically different between MarginProbe and standardof-care groups. No further outcomes beyond re-excision rates were evaluated in the trial. 4. The technology must be as beneficial as any established alternatives. MarginProbe has not been assessed in comparison to other techniques of intraoperative margin assessment such as frozen section and touch-print cytology. 5. The improvement must be attainable outside the investigational settings. Lacking evidence for improved net health outcome, whether they can be attained outside the investigational setting cannot be determined. Based on the above, handheld radiofrequency spectroscopy for intraoperative margin assessment during breast-conserving surgery (i.e., MarginProbe ) does not meet the TEC criteria. Contents Assessment Objective 4 Background 4 Methods 6 Formulation of the Assessment 7 Review of Evidence 7 Discussion 12 Summary of Application of the 13 Technology Evaluation Criteria References 15 Published in cooperation with Kaiser Foundation Health Plan and Southern California Permanente Medical Group. TEC Staff Contributors Author David H. Mark, M.D., M.P.H.; TEC Executive Director Naomi Aronson, Ph.D.; TEC Director, Technology Assessments Mark D. Grant, M.D., M.P.H.; Director, Clinical Science Services Kathleen M. Ziegler, Pharm.D.; Research/Editorial Staff Claudia J. Bonnell, B.S.N., M.L.S.; Kimberly L. Hines, M.S. Blue Cross and Blue Shield Association Medical Advisory Panel Trent T. Haywood, M.D., J.D. Chairman, Senior Vice President, Clinical Affairs/Medical Director, Blue Cross and Blue Shield Association; Steven N. Goodman, M.D., M.H.S., Ph.D. Scientific Advisor, Dean for Clinical and Translational Research, Stanford University School of Medicine, Professor, Departments of Medicine, Health Research and Policy; Mark A. Hlatky, M.D. Scientific Advisor, Professor of Health Research and Policy and of Medicine (Cardiovascular Medicine), Stanford University School of Medicine. Panel Members Peter C. Albertsen, M.D., Professor, Chief of Urology, and Residency Program Director, University of Connecticut Health Center; Sarah T. Corley, M.D., F.A.C.P., Chief Medical Officer, NexGen Healthcare Information Systems, Inc. American College of Physicians Appointee; Helen Darling, M.A., President, National Business Group on Health; Josef E. Fischer, M.D., F.A.C.S., William V. McDermott Professor of Surgery, Harvard Medical School American College of Surgeons Appointee; I. Craig Henderson, M.D., Adjunct Professor of Medicine, University of California, San Francisco; Jo Carol Hiatt, M.D., M.B.A., F.A.C.S., Chair, Inter-Regional New Technology Committee, Kaiser Permanente; Saira A. Jan, M.S., Pharm.D., Associate Clinical Professor, Ernest Mario School of Pharmacy, Rutgers, The State University of New Jersey, Residency Director and Director of Clinical Programs Pharmacy Management, Horizon Blue Cross and Blue Shield of New Jersey; Thomas Kowalski, R.Ph., Clinical Pharmacy Director, Blue Cross Blue Shield of Massachusetts; Lawrence Hong Lee, M.D., M.B.A., F.A.C.P., Vice President and Executive Medical Director for Quality and Provider Relations, Blue Cross and Blue Shield of Minnesota; Bernard Lo, M.D., Professor of Medicine and Director, Program in Medical Ethics, University of California, San Francisco; Randall E. Marcus, M.D., Charles H. Herndon Professor and Chairman, Department of Orthopaedic Surgery, Case Western Reserve University School of Medicine; Barbara J. McNeil, M.D., Ph.D., Ridley Watts Professor and Head of Health Care Policy, Harvard Medical School, Professor of Radiology, Brigham and Women s Hospital; William R. Phillips, M.D., M.P.H., Clinical Professor of Family Medicine, University of Washington American Academy of Family Physicians Appointee; Richard Rainey, M.D., Medical Director, Regence BlueShield of Idaho; Rita F. Redberg, M.D., M.Sc., F.A.C.C., Professor of Medicine and Director, Women s Cardiovascular Services, University of California San Francisco; Maren T. Scheuner, M.D., M.P.H., F.A.C.M.G., Chief, Medical Genetics, VA Greater Los Angeles Healthcare System; Associate Clinical Professor, Department of Medicine, David Geffen School of Medicine at UCLA, Affiliate Natural Scientist, RAND Corporation; J. Sanford Schwartz, M.D., F.A.C.P., Leon Hess Professor of Medicine and Health Management & Economics, School of Medicine and The Wharton School, University of Pennsylvania. CONFIDENTIAL: This document contains proprietary information that is intended solely for Blue Cross and Blue Shield Plans and other subscribers to the TEC Program. The contents of this document are not to be provided in any manner to any other parties without the express written consent of the Blue Cross and Blue Shield Association Blue Cross and Blue Shield Association. Reproduction without prior authorization is prohibited. 3
4 Assessment Objective Breast-conserving surgery as part of the treatment of localized breast cancer is optimally achieved by attaining margins around the surgical resection that are free from tumor cells. Failure to achieve clear margins will often require additional surgery to re-excise breast tissue. Handheld radiofrequency spectroscopy for intraoperative margin assessment (i.e., MarginProbe ) is a device that can detect tumor cells at the surface of the main lumpectomy specimen. It is intended for use in addition to standard intraoperative assessment to guide additional resection of tissue during breast-conserving surgery. The purpose of this Assessment is to evaluate the evidence regarding the use of MarginProbe in improving health outcomes for patients undergoing breast-conserving surgery for localized breast cancer. Background Margin Status and Outcomes of Breast-Conserving Surgery Breast-conserving surgery with radiation is considered equivalent to mastectomy in producing equivalent long-term survival for stage I or II breast cancer. Achieving clear surgical margins has been considered an important aspect of adequate breast-conserving surgery. Several studies have demonstrated an association between margin status and local recurrence rate. A meta-analysis by Houssami et al. (2010) showed an association (odds ratio 2.4 [95% CI: 1.9 to 3.0]) between local recurrence and margin status classified as positive or negative. Most studies of margin status and clinical outcome assess local recurrence rate as the outcome. At least one study reported an association between margin status and overall survival, but there was confounding caused by earlier stage of disease in patients with clear margins (Obedian and Haffty 2000). Beyond the generally agreed-upon prognostic significance of margins that are overtly positive, there is considerable controversy regarding the extent to which negative surgical margins are associated with better outcomes. According to the National Comprehensive Cancer Network (NCCN) guidelines on ductal carcinoma in situ, margins greater than 10 mm are widely considered as negative, but may be considered by some as excessive due to less than optimal cosmetic result (NCCN 2013). Margins less than 1 mm are considered inadequate, but margins less than 1 mm at boundaries with the skin or chest wall do not mandate re-excision. For infiltrating carcinoma, re-excision for positive margins is recommended, but in patients without an intraductal component and only a focally positive margin, it is acceptable to treat the patient with a radiation boost dose. Studies of hospital level and surgeon-specific re-excision rates show substantial variability. In a study by McCahill et al. (2012), re-excision rates were highest for patients with positive margins (86%), and declined as the margin increased. For patients with negative margins, re-excision rates varied between 0% and 70% among surgeons. However, assessment of margin status even after final pathologic examination has some degree of uncertainty or error. In a study by Kotwall et al. (2007), the margins of excisional biopsies (procedures not meant to be therapeutic breast-conserving procedures) were compared with the subsequent breast-conserving surgical findings. In patients with clear margins of various distance in their excisional biopsies, 8% of subjects with clear margins less than 1 mm, 15% of subjects with margins 1-2 mm, and 4% of subjects with margins greater than 2 mm had residual cancer in their breast-conserving surgery specimens. Thus, clear margins indicate a low (4% to 15%), but not zero, probability of residual cancer. Despite uncertainties in the literature, evidence would appear to support the standard practice of re-excising breast tissue in a second operation for patients who have positive margins in resected breast tissue. An American Society of Breast Surgeons position statement on breast cancer lumpectomy margins states that margins 1 mm or greater do not require reexcision, less than 1 mm or focally involved margins should be considered on a case-bycase basis, and that positive ink-positive margins require re-excision unless there are extenuating circumstances (American Society of Breast Surgeons 2013). Whether achieving margins greater than 2 mm has an effect in improving local recurrence or long-term outcome is uncertain. Intraoperative Assessment of Margins The reference standard for margin status is currently only achieved by pathologic examination of excised tissue performed after surgery Blue Cross and Blue Shield Association. Reproduction without prior authorization is prohibited.
5 Handheld Radiofrequency Spectroscopy for Intraoperative Margin Assessment Various techniques and methods have been evaluated to assist the surgeon in achieving clear margins during the initial operation. Some techniques do not actually assess the surgical margin, but attempt to more accurately identify the location of the tumor to the surgeon, thus increasing the probability of a clean margin by marking a better target for resection. Intraoperative ultrasound and cryoprobe-assisted localization are techniques to make the tumor more visible or more palpable to the surgeon (Pleijhuis et al. 2009). Intraoperative specimen radiography involves imaging the lumpectomy specimen and comparing the image of the specimen to the preoperative mammogram. If microcalcifications occur close to the edges of the specimen, the surgeon may decide to shave the associated cavity edges to create a larger margin from the calcifications. Frozen section analysis involves an immediate histological examination of excised tissue during the operation. It has been demonstrated that frozen section analysis results in additional tissue excision during the operation and results in low re-excision rates after definitive pathologic examination (Pliujhuis et al. 2009). However, it adds an average of 30 minutes to operating time, and its use is not routine. Intraoperative touch-preparation cytology is another technique in which cells at the surgical margin are directly examined. Studies report good diagnostic performance characteristics for identifying tumor at surgical margins, but it appears that it is also not in common use. MarginProbe Device The MarginProbe device is based on the principles of dielectric spectroscopy to characterize tissue that the device comes in contact with. A handheld probe with a measurement area of 7 mm generates radiofrequency electromagnetic fields at MHz that penetrate a few millimeters deep into tissues touching the probe. The signals reflected back to the device are collected and analyzed by the system. The resonance frequency and amplitude of the reflection change significantly with the change in tissue properties. When tissue of samples of pure cancer tissue and pure benign tissue are applied to the device, the frequency and amplitude of the signals produce a fairly pure separation of signal. In early developmental studies in which tissue samples in which known homogenous tissue samples were directly within the 7 mm footprint of the device, sensitivity and specificity of the device were high. When the measurement site contained at least 75% cancer, sensitivity was 100% and specificity was 87%. However, margins of actual lumpectomy specimens are not pure tissue samples. The signals from various tissues of known pathology have been subjected to analysis to produce a classification algorithm to optimize sensitivity and specificity for detecting tumor cells. The classification algorithm, however, does not allow a perfect distinction between malignant and benign tissue. The classification algorithm has been revised several times during the development of the product. It appears that only the study described in the evidence review section of this Assessment uses the latest classification algorithm; all prior studies used different classification algorithms and may not reflect the performance of the algorithm used in the pivotal study. For the user of the device, the classification algorithm produces a binary positive or negative result for a single test of a 7-mm diameter surface of the lumpectomy specimen. Each surface of the lumpectomy should be tested a minimum of 5 times and up to whatever maximum necessary to cover the entire surface of the particular margin. A margin is considered positive if one or more readings indicates positive with the device. Early Clinical Study of MarginProbe An early published study of clinical outcomes in a randomized trial of MarginProbe will be reported in this section of the Assessment because according to FDA documents, the classification algorithm used in this study is no longer used, and thus this study may not reflect current performance characteristics of the device. The study is similar in structure to the FDA pivotal study reviewed in the evidence section of this Assessment. In a study by Allweis et al. (2008), 300 patients undergoing breast-conserving surgery were randomized to either MarginProbe assessment plus standard margin assessment or standard margin assessment alone. Standard margin assessment was not specified in the study, but could include specimen imaging and microscopic pathological assessments. The principal outcome of the study was the intraoperative surgical resection (ISR) rate, which was defined as the proportion of patients in whom the main specimen surgical margins were positive who had all the 2013 Blue Cross and Blue Shield Association. Reproduction without prior authorization is prohibited. 5
6 margins surgically re-excised. Although reexcising the positive margin might produce a clean margin, it was not necessary in order to count it. In the study, 60% of patients with positive margins had all the positive margins re-excised in the MarginProbe group, whereas 41% of subjects with positive margins in the control group had all the positive margins re-excised (p=0.044). This translated ultimately to lower but not statistically significantly different reoperation rates (12.6% versus 18.6%, p=0.098). In the subgroup of patients with nonpalpable lesions, results were similar, and the reoperation rates were lower and statistically significant in the MarginProbe group (9.8% versus 20.9%). Excised tissue volume was slightly larger but not statistically significant in the MarginProbe group. The study is consistent with a possible benefit of MarginProbe in reducing the re-excision rate, but its results may not reflect current performance characteristics of the MarginProbe system. FDA Status. In January 2013, MarginProbe received premarket application (PMA) approval from the FDA. The Dune MarginProbe System is an adjunctive diagnostic tool for identification of cancerous tissue at the margins ( 1mm) of the main ex-vivo lumpectomy specimen following primary excision and is indicated for intraoperative use, in conjunction with standard methods (such as intraoperative imaging and palpation) in patients undergoing breast lumpectomy surgery for previously diagnosed breast cancer. The FDA labeling states that the Dune MarginProbe System should not be used (i.e., is contraindicated): n To replace standard tissue histopathology assessment n On ex-vivo lumpectomy specimens that have been exposed to saline, ultrasound gel or local anesthetic solutions. n On in-vivo tissue (i.e., it should not be used within the lumpectomy cavity) n On tissues other than breast tissue (i.e. it should not be used on sentinel lymph nodes) n Closer than 1.5 mm to a fine-needle localization guidewire Other warnings in the FDA labeling include warnings against its use in several types of patients, including those with known multicentric disease, bilateral disease, prior radiation, prior surgery, and others. Adverse effects listed included an extension in operating time, removal of healthy breast tissue, infection, bleeding, and local tissue damage. Methods Search Methods We searched MEDLINE via PubMed in July 2013 using the search term MarginProbe. The web site of the U.S. Food and Drug Administration (FDA) was searched for documents relating to the MarginProbe device. Study Selection There are relatively few published studies evaluating the MarginProbe device. In reviewing the documents concerning the trial evaluated by the FDA that led to approval, it is apparent that some of the earlier published studies have been used to refine the classification algorithms used by the system to determine test positivity. Only the FDA study reports the results of the MarginProbe device in its apparently most recent device and classification algorithm version. Thus, the only study selected for detailed review is the study described as the pivotal study in FDA documents (FDA 2012). Data Abstraction, Calculations, Quality Assessment Data were abstracted from various FDA documents by a single reviewer. Standard quality assessment tools were difficult to apply to this study, as it involved both a diagnostic device and subsequent intervention, and due to the nature of the device its use could not be blinded. Accordingly, we note possible biases and study limitations as necessary. Medical Advisory Panel Review This Assessment was initially reviewed by the Blue Cross and Blue Shield Association Medical Advisory Panel (MAP) on June 6, In order to maintain the timeliness of the scientific information in this Assessment, literature searches were performed subsequent to the Panel s review (see Search Methods ). An additional published article was found in the update search (Rivera et al. 2012). This study is the same as that reported as the pivotal trial in the FDA documents used in this Assessment and provides duplicate results Blue Cross and Blue Shield Association. Reproduction without prior authorization is prohibited.
7 Handheld Radiofrequency Spectroscopy for Intraoperative Margin Assessment Formulation of the Assessment Patient Indications MarginProbe is indicated for adjunctive evaluation of the main lumpectomy specimen in order to guide additional excision of breast tissue. In addition to the specific types of patients in the FDA label warnings that should be excluded from use, it should be noted that the inclusion criteria of the pivotal trial included only women with nonpalpable lesions. Results from the trial may not be generalizable to women with palpable lesions. It should also be noted that since MarginProbe evaluates tissue only in the main lumpectomy specimen, certain techniques of breast-conserving surgery contraindicate its use. Where the standard practice or planned procedure is to shave additional tissue beyond the main lumpectomy specimen at all margins, MarginProbe results do not reflect status of tissue directly adjacent to that remaining in the patient. Thus, MarginProbe is not indicated in cases where mandatory cavity shaving is carried out. Technologies to be Evaluated and Compared MarginProbe was used in a relatively specific manner and in relation to a comparative strategy in breast-conserving surgery in the study reviewed in this assessment. MarginProbe was used in patients after initial removal of the main lumpectomy specimen AND additional shavings judged as initially necessary. MarginProbe was then used to evaluate the main lumpectomy specimen, and the surgeon responded to MarginProbe results AND also to additional standard margin assessments including specimen radiography. Intraoperative histologic techniques such as frozen section or touch-print cytology were not used. This MarginProbe strategy was compared to a control strategy in which the main specimen plus additional shavings were initially performed, followed by additional shavings as indicated by additional standard margin assessments including specimen radiography. As in the MarginProbe strategy, frozen section or touch-print cytology were not used in the control arm. Health Outcomes n Breast-cancer-specific survival n Cancer recurrence n Re-excision For breast cancer, the most important health outcome is overall survival from the disease. MarginProbe is not intended to directly improve the surgical cure rates of cancer. Rather, it is intended to assist the initial surgical procedure so that equivalent or better outcomes can be achieved with less morbidity due to re-excision procedures. Thus, the principal outcome associated with use of the MarginProbe device is the re-excision rate. However, a lower re-excision rate can only be considered a positive health outcome if it is demonstrated or assumed that the local recurrence rate is equivalent or lower. A lower reexcision rate that ultimately results in higher local recurrence rates would simply indicate inadequate surgery. Lacking direct observation of these local recurrence rates, it is necessary that the patients undergo surgical procedures performed with at least an equivalent standard, and that they have a similar prognosis at the end of the initial sequence of care. The final pathologic margin status of the patient after initial breast-conserving surgery, although not a health outcome itself, is probably the major determinant of whether the patient will require re-excision, and might be considered a reasonable surrogate for a health outcome. When used as intended, MarginProbe is likely to result in additional breast tissue excision. Any adverse effects of MarginProbe would relate to the additional excision, which would include any adverse effects of additional operating time, cosmetic appearance of the breast, infection, and additional trauma to the breast and surrounding tissues. When not used as intended, for example, as the sole means of assessing surgical margins, it is possible that MarginProbe may produce worse outcomes for final surgical margins and/or re-excision rates. Specific Assessment Question Does use of MarginProbe as an adjunctive method to determining presence of cancer at the margins of lumpectomy specimens, improve health outcomes when compared to standard techniques of intraoperative margin assessment? Review of Evidence The evidence for this Assessment consists of the pivotal clinical trial evaluated in FDA documents which led to approval by the FDA in January 2013 (FDA 2012). Clinical studies prior 2013 Blue Cross and Blue Shield Association. Reproduction without prior authorization is prohibited. 7
8 to this trial were used to refine classification algorithms of the MarginProbe system; thus these earlier studies may not reflect current performance. This study reports the most relevant patient outcomes available for evaluating MarginProbe with the largest number of patients, including a large proportion of U.S. patients. In addition to clinical outcomes, the study allows assessments of diagnostic test performance of MarginProbe which will help inform judgments of its utility. FDA documents such as this summary of safety and effectiveness, in addition to showing results of analyses that are in general identical to those generated by the sponsors of the device, often include additional analysis requested by the FDA, analyses performed by FDA statisticians, and commentary indicating FDA analysts agreement or disagreement with any aspects of the sponsors analysis or claims. Pivotal Study Design and Outcomes The pivotal study compared surgical processes and short-term outcomes in patients in whom MarginProbe was used versus patients in whom MarginProbe was not used. The comparative strategy did not include intraoperative histologic techniques, but radiographic imaging of the main resection specimen was performed in addition to inspection of the resection specimen. A study flow chart is shown in Figure 1. 1 Patients enrolled in the trial first underwent excision of the main specimen. Additional shavings from the lumpectomy cavity could be performed before randomization to MarginProbe or standard-of-care (SOC) groups. Patients randomized to the MarginProbe arm at this point then had their main lumpectomy specimens tested with MarginProbe. Surgeons were then required to re-excise margins testing positive on MarginProbe or to document the positive test and note why a re-excision did not occur. A common reason was that the margin was close to skin, fascial layer, or chest wall. Patients in both study arms then underwent specimen imaging and other intraoperative assessment, after which additional shavings could be taken. The main specimen and all shavings were evaluated with complete histological assessment after surgery. Final margin status of the main specimen, as well as the patient, could be determined based on the histologic findings of the main specimen and all additional shavings. The performance of re-excision was based on the judgment of the treating surgeon. Figure 1. MarginProbe Pivotal Study Flow Chart Shaving option Shaving option Path report No repeat surgery Patient enrollment Main specimen excision Randomize SOC Arm Device use SOC+Device Arm Specimen imaging Shaving pathology + Main specimen pathology Repeat surgery Shaving option Shaving option Shaving option Blue Cross and Blue Shield Association. Reproduction without prior authorization is prohibited.
9 Handheld Radiofrequency Spectroscopy for Intraoperative Margin Assessment Several study outcomes were specified. Some of them are quite complex and require detailed description and comment. Table 1 shows the endpoints described as co-primary endpoints and confirmatory secondary endpoints. Other additional endpoints shown in the FDA Executive Summary will be defined and shown as necessary. Several issues regarding the design and outcomes of the study should be pointed out. Patients were randomized after initial removal of the main specimen and removal of additional tissue ( shavings ) on the judgment of the treating surgeon. If shavings were taken at this point, the surface of the main lumpectomy specimen no longer represents the interface of the specimen and tissue remaining in the patient at the margin where shavings were taken. This point is not addressed in the document. Several issues regarding these shavings taken before randomization are not fully discussed and raise some questions. It is also important to note that in the analysis of the study, although all shavings were evaluated histologically to assess the ultimate margin status of the patient, no information is available regarding whether shavings were performed pre-randomization, subsequent to testing with MarginProbe, or subsequent to standard margin assessment. The outcome of complete surgical resection (CSR) requires further discussion. Positive margins on final pathology were counted as CSR if they were excised or addressed, but there was no requirement that the additional shavings result in a clear margin. Thus, the re-excision performed in response to a true positive MarginProbe test or standard assessment, if it does not result in a clear margin, may not be clinically relevant. Noting but not re-excising a positive margin also is not clinically relevant. The part of the definition that counts addressing a positive test but not resecting (due to anatomic considerations) as a positive outcome for the MarginProbe arm appears to be biased against the control arm of the study. Information from the documents regarding how clinical assessment was carried out in the control group appears to be incomplete. It is unclear how diagnostic characteristics for the control group were determined. There is nothing in the document that states how a positive test is declared in the control arm. It appears that the only countable response in the control arm is a re-excision since there is no formal test to respond to; thus, this aspect of the outcome is biased. It is also unclear whether pre-randomization shavings are counted as being appropriate responses to a positive main specimen margin in either study arm. It is stated that a positive MarginProbe test required the surgeon to reexcise or address a positive test, but unclear if a pre-randomization shaving qualified as a valid re-excision or whether further excision was required. Table 1. Definition of Pivotal Trial Endpoints Co-primary Endpoints Complete surgical resection (CSR) Non-randomness Test Normalized Total Tissue Volume Definition All positive margins on the main specimen on final pathology re-excised or addressed (noted but not re-excised) Statistical test only applied to MarginProbe group to evaluate whether test positivity is not random Total tissue removed (main specimen, all shavings, plus additional procedures) as a % of baseline breast size Confirmatory Secondary End Points Total Tissue Volume Positive margin rate Re-lumpectomy procedure rate Total tissue removed (main specimen, all shavings, plus additional procedures) Percent of patients with at least one positive margin after initial surgical procedure ( using final pathology analysis of main specimen plus all shavings) Number of repeated ipsilateral breast conserving surgical procedures for each patient. Increase by 1 if there are additional subsequent lumpectomy procedures 2013 Blue Cross and Blue Shield Association. Reproduction without prior authorization is prohibited. 9
10 The study protocol offers 3 opportunities for shaving additional tissue in the MarginProbe arm and 2 opportunities for shaving in the control arm. The additional option to perform shavings may be in part responsible for more thorough excision of cancer tissue in the initial surgery. A non-randomness test was utilized to address this concern. 2 The principal outcome of CSR was criticized by the FDA because there was no accounting for false-positive MarginProbe readings. In patients with specimens who had no positive margins but had positive MarginProbe tests, additional healthy tissue would be resected. False-positive tests at different margins than some true-positive tests would result in unnecessary additional resection in addition to necessary additional excisions. The comparison of total quantity of tissue removed from each study group should at least, in part, address some of these concerns. Characteristics of Patients Enrolled in the Pivotal Study The pivotal study was a multicenter (21 sites) randomized study involving 596 patients assigned equally to the two arms of the study. Patients enrolled in the study met criteria mentioned in the FDA labeling, but also all had nonpalpable lesions which required imageguided localization. Some patient demographic and baseline breast cancer characteristics up to the point of randomization of the two groups are shown in Table 2. The characteristics of the patients and excised lesions appear to be generally similar. We could not find data on what proportion of patients had shavings taken before randomization. Outcomes of Pivotal Study Several outcomes were reported in the FDA Summary of Safety and Effectiveness relevant ones reported in Table 3. It is important to note that some results are reported using percentages or rates based on the subset of patients that had positive margins in the main specimen, and some results using all patients in the study. Although MarginProbe only has possible benefits in patients with positive margins in their main specimens, the negative consequences of false positives and subsequent unnecessary additional excisions can occur to all patients. In most cases, comparing rates using a denominator of all patients is the appropriate analysis. Of 298 subjects in the MarginProbe arm, 163 (55%) had positive margins by final pathology in the main specimen. Of 298 subjects in the control arm, 147 (49%) had positive margins by final pathology in the main specimen. This is probably a result that correlates with the slightly lower mean weight of specimens in the MarginProbe arm, a chance result since patients were randomized after the main specimen was excised. For the principal outcome of complete surgical resection, MarginProbe showed a rate of 71.8% versus 22.4% for controls, with positive margin subjects as the denominator, which is a Table 2. Baseline Patient and Breast Tumor Characteristics Characteristic MarginProbe, n=298 Control, n=298 Mean age % White % Black Mean BMI Preoperative Diagnosis (not all types listed) % Ductal carcinoma in situ % Mixed % Palpable lesion in excised specimen Mean weight of excised specimen (g) The non-randomness test described as [e]valuat[ing] the association between the margin level device output (positive or negative margin) and histology results (positive or negative) of main specimen obtained from an analysis accounting for correlated results within patients (either generalized linear mixed or estimating equations models) Blue Cross and Blue Shield Association. Reproduction without prior authorization is prohibited.
11 Handheld Radiofrequency Spectroscopy for Intraoperative Margin Assessment Table 3. Selected Results of Pivotal Trial Endpoint/ (denominator) ALL or PSS MarginProbe Control p-value or CI Complete Surgical Resection/PSS 71.8% 22.4% < Non-randomness Test/ALL Statistically significant detection of margins Not applicable Normalized Total Tissue Volume/ALL 15.1% 12.5% Meets noninferiority criteria (not specified) Total Tissue Volume/ALL 93cc 85cc Meets noninferiority criteria, 25cm Positive margin rate/all 31% 42% Re-excision procedure rate/pss 33.7% 46.9% 0.04 Re-excision procedure rate including mastectomy/all 23.8% 28.5% 0.32 ALL: all subjects in trial, 298 in each group PSS: positive specimen subjects, subset in each group with positive margins in main specimen on final pathology, n=163 in MarginProbe group, 147 in control group large magnitude of difference and statistically significant. Due to the previously mentioned bias involving positive margins noted but not resected, this outcome is biased. Greater volume of tissue resected on both a relative and absolute scale were greater in the MarginProbe group, but the data analysis presented only reports conclusions of a noninferiority analysis. The noninferiority margin for the normalized total tissue volume was not specified. More clinically relevant outcomes include the proportion of patients with positive margins on final pathology after surgery, which was 31% for the MarginProbe group and 42% in the control group (p=0.0082). It should be noted that some patients with positive margins in the MarginProbe group arise from subjects that did not have positive margins in their main specimen. However, due to false-positive MarginProbe readings, additional shavings were undertaken in which cancer tissue was found at the margin. Without these additional shavings taken in response to MarginProbe assessment, these patients would have been considered to have a clear margin. This occurrence reflects the uncertainty of final pathology in trying to ascertain whether all cancer tissue has been removed. It complicates the comparison of outcomes between the two groups because a measure usually considered a poor outcome such as a positive margin, in this case, is not due to inadequate surgery but inadvertent discovery of residual cancer due to falsepositive MarginProbe readings. Re-excision rates using all patients as the denominator showed about a 5% absolute reduction in the MarginProbe group, which was not statistically significant. The decision to re-operate was based on judgment of the surgeon based on patient and tumor characteristics and the totality of pathologic findings. It cannot be determined what factors led to the decision not to re-excise when the final pathologic determination showed positive margins. Limited analysis examining results in subgroups was reported. Most of the patients were from Israel, and subjects from Israel tended to show better results, in terms of the primary outcome, re-excision rates, and diagnostic performance, than subjects from the United States. No results were reported comparing results for different types of cancer, such has ductal carcinoma in situ and invasive cancers. Diagnostic Performance of MarginProbe on Main Specimens Although not a clinical outcome, the pivotal trial allows calculation of the diagnostic characteristics of MarginProbe in the ability to detect cancer at the margin of the main lumpectomy specimen. Since all margins of each specimen 2013 Blue Cross and Blue Shield Association. Reproduction without prior authorization is prohibited. 11
12 in the MarginProbe group required testing, and all margins of the main specimen were examined by histopathology, sensitivity and specificity and predictive values can be calculated. The sensitivity and specificity calculations represent the results of the test as it is to be used on a margin, with at least 5 readings over the total surface of the margin, and counting the test as positive if one or more readings are positive. Out of 1,788 margins with final histopathology, MarginProbe readings were valid or not missing in 1,750. Three-hundred twenty-seven margins were positive, and MarginProbe was positive in 246 for a sensitivity of 75.2%. Out of 1,423 negative margins, MarginProbe was negative in 660 for a specificity of 46.4%. These performance characteristics showing moderate sensitivity and poor specificity are consistent with better than random capability of the device in detecting positive margins, and was probably the statistical basis for the test being meeting a declared outcome of non-randomness. Given the 19% (327/1,750) prevalence of positive margins, the positive predictive value of a positive MarginProbe test for a margin is 24%. In another analysis (apparently performed or requested by FDA) in which the location of the positive margin was ignored, and the test was considered positive if any margin tested positive, MarginProbe was 96.3% sensitive but 8.9% specific. Although this test performance characteristic is less clinically relevant, what it does indicate is that in this study, the low specificity indicates that MarginProbe was positive for at least one margin in almost every patient in the study, even though the prevalence of at least one positive margin was 52%. The FDA Summary reports sensitivity and specificity values calculated by the device sponsor for the control arm of the study, which show apparently worse performance of standard margin assessment. However, it could not be determined how these diagnostic characteristics were calculated, as there was no specific designation during the study in which the control arm was considered to be undergoing a diagnostic test. It appears that only the behavior of taking additional shavings was recorded, and the timing of these shavings (pre-randomization or after clinical assessment) was not recorded. Other Adverse Effects Adverse effects between the study arms did not reveal any notable differences between treatment arms that could possibly be related to use of MarginProbe. Postoperative wound infections and breast abscess were rare or zero in both groups, and no other events could be plausibly related to MarginProbe. Discussion Although the MarginProbe device detects cancer cells in lumpectomy specimens statistically better than random, its performance characteristics show moderate sensitivity of 75% and poor specificity of 46% when evaluated on a per-margin basis. When used as an adjunct to standard margin assessment that uses specimen imaging, the clinical trial showed a statistically significantly higher proportion of patients with clear margins after initial surgery. However, this did not translate to a significantly different rate of patients undergoing second surgeries, although the absolute difference was 5%. It is not certain what factors lead the surgeons to not attempt a reoperation in all cases in which subjects did not have clear margins. In addition to not being statistically significant, it is not known whether the lower second surgery rate will result in equivalent or better long-term outcome rates such as the local recurrence rate. As mentioned previously, a lower re-excision rate is only a health benefit if it achieved without a higher local recurrence rate. Use of MarginProbe results in overall more tissue removed in surgery. The impact of this on cosmetic outcome and patient satisfaction is uncertain, and difficult to weigh against the potential for lower second surgery rates. The device sponsor designed their trial in with a specific outcome called Complete Surgical Resection, which effectively counts up truepositive MarginProbe readings, without considering whether an adequate or even any excision results from the true-positive test, and does not consider any consequences of false-positive tests. In this study, this outcome appears to be biased against the control arm of the study. Information in the study regarding clinical assessment and calculation of diagnostic performance in the control group was not completely reported Blue Cross and Blue Shield Association. Reproduction without prior authorization is prohibited.
13 Handheld Radiofrequency Spectroscopy for Intraoperative Margin Assessment It is important to reiterate that this clinical trial evaluated MarginProbe specifically as an adjunct to standard assessment of margins that did not include any intraoperative histologic techniques and that radiologic imaging and gross inspection of the main specimen was employed in addition to MarginProbe. It is unknown how MarginProbe would compare to other techniques of intraoperative margin assessment, nor how would it work if used without at least the standard techniques used in the pivotal study. Given the moderate sensitivity of the device of 75%, it is possible that outcomes might be worse without other methods. Surgeons should not abandon other methods of margin assessment when using MarginProbe. According to the labeling information, use of MarginProbe is contraindicated in some common situations. It should not be used when specimens have been exposed to ultrasound gel, saline, or local anesthetic solutions, or cryoassisted localization has been performed. It is uncertain how the device performs when other prior procedures, such as radiotracer injections or core needle biopsies, have been done. Future Research Future studies evaluating techniques for improving the performance of breast-conserving surgery by improving the rate at which margin-free resections are achieved and lowering the re-excision rate should aim at ultimately demonstrating effectiveness with rigorous randomized clinical trials. The pivotal trial of MarginProbe had critical flaws which prevented inferring effectiveness. The principal outcome of the study was designated in order to highlight true positive tests, regardless of the clinical relevance of those tests, and to disregard false-positive tests. Future studies of other devices or techniques should correct these flaws by careful consideration of the sequence of events and attention to bias. There should be sufficient statistical power to evaluate clinically important differences in relevant clinical outcomes such as the re-excision rate and the local recurrence rate. A standard protocol should be developed for all study arms, providing guidance on how surgeons should respond to the tests. Although demonstrating effectiveness in terms of cancer survival or metastatic rates is probably unreasonable, it might be possible to determine equivalency or superiority of local recurrence rates of different techniques. Summary of Application of the Technology Evaluation Criteria Based on the available evidence, the Blue Cross and Blue Shield Association Medical Advisory Panel (MAP) made the following judgments about whether handheld radiofrequency spectroscopy for intraoperative margin assessment during breast-conserving surgery (i.e., MarginProbe ) meets the Blue Cross and Blue Shield Association Technology Evaluation Center (TEC) criteria. 1. The technology must have final approval from the appropriate governmental regulatory bodies. In January 2013, MarginProbe received premarket application (PMA) approval from the U.S. Food and Drug Administration (FDA) as an adjunctive tool for identification of cancerous tissue at the margins of the main lumpectomy specimen. 2. The scientific evidence must permit conclusions concerning the effect of the technology on health outcomes. The evidence does not permit conclusions concerning the effect of the technology on health outcomes. There is one randomized trial comparing use of MarginProbe to a standardof-care assessment for surgical margins with its current classification algorithm. The principal outcome of the study was biased against the standard-of-care arm. There was an absolute reduction of 5% in re-excision rates in the MarginProbe arm, but it was not statistically significant. A greater amount of breast tissue was removed in the MarginProbe arm. No further outcomes were assessed beyond the re-excision rate. The diagnostic performance characteristics of the MarginProbe system showed moderate sensitivity of approximately 75% and poor specificity of 46%. 3. The technology must improve the net health outcome. It has not been demonstrated that the net health outcome was improved. The re-excision rates were neither clinically meaningful nor statistically different between MarginProbe and standard-of-care groups. No further outcomes beyond re-excision rates were evaluated in the trial Blue Cross and Blue Shield Association. Reproduction without prior authorization is prohibited. 13
14 4. The technology must be as beneficial as any established alternatives. MarginProbe has not been assessed in comparison to other techniques of intraoperative margin assessment such as frozen section and touch-print cytology. 5. The improvement must be attainable outside the investigational settings. Lacking evidence for improved net health outcome, whether they can be attained outside the investigational setting cannot be determined. Based on the above, handheld radiofrequency spectroscopy for intraoperative margin assessment during breast-conserving surgery (i.e., MarginProbe ) does not meet the TEC criteria. NOTICE OF PURPOSE: TEC Assessments are scientific opinions, provided solely for informational purposes. TEC Assessments should not be construed to suggest that the Blue Cross Blue Shield Association, Kaiser Permanente Medical Care Program or the TEC Program recommends, advocates, requires, encourages, or discourages any particular treatment, procedure, or service; any particular course of treatment, procedure, or service; or the payment or non-payment of the technology or technologies evaluated. CONFIDENTIAL: This document contains proprietary information that is intended solely for Blue Cross and Blue Shield Plans and other subscribers to the TEC Program. The contents of this document are not to be provided in any manner to any other parties without the express written consent of the Blue Cross and Blue Shield Association Blue Cross and Blue Shield Association. Reproduction without prior authorization is prohibited.
15 Handheld Radiofrequency Spectroscopy for Intraoperative Margin Assessment References Allweis TM, Kaufman Z, Lelcuk S et al. (2008). A prospective, randomized, controlled, multicenter study of a real-time, intraoperative probe for positive margin detection in breast-conserving surgery. Am J Surg, 196: American Society of Breast Surgeons. (2013). Position statement on breast cancer lumpectomy margins. Accessed at https://www.breastsurgeons.org/statements/ PDF_Statements/Lumpectomy_Margins.pdf. Balch GC, Mithani SK, Simpson JF et al. (2005). Accuracy of intraoperative gross examination of surgical margin status in women undergoing partial mastectomy for breast malignancy. Am Surg, 71:22-7. Houssami N, Macaskill P, Marinovich ML et al. (2010). Meta-analysis of the impact of surgical margins on local recurrence in women with early-stage invasive breast cancer treated with breast-conserving therapy. Eur J Cancer, 46: Kotwall C, Ranson M, Stiles A et al. (2007). Relationship between initial margin status for invasive breast cancer and residual carcinoma after re-excision. Am Surg, 73: McCahill LE, Single RM, Aiello Bowles EJ et al. (2012). Variability in reexcision following breast conservation surgery. JAMA, 307: National Comprehensive Cancer Network Clinical Practice Guidelines in Oncology (NCCN Guidelines). (2013). Breast Cancer. Version Available at Obedian E, Haffty BG. (2000). Negative margin status improves local control in conservatively managed breast cancer patients. Cancer J Sci Am, 6: Pleijhuis RG, Graafland M, de Vries J et al. (2009). Obtaining adequate surgical margins in breast-conserving therapy for patients with early-stage breast cancer: current modalities and future directions. Ann Surg Oncol, 16: Rivera RJ, Holmes DR, Tafra L. (2012). Analysis of the Impact of Intraoperative Margin Assessment with Adjunctive Use of MarginProbe versus Standard of Care on Tissue Volume Removed. Int J Surg Oncol, Vol. 2012: Epub 2012 Dec 26. U.S. Food and Drug Administration (FDA). (2012). Summary of safety and effectiveness data: MarginProbe System. Accessed at docs/pdf11/p110014b.pdf Blue Cross and Blue Shield Association. Reproduction without prior authorization is prohibited. 15
16 Technology Evaluation Center Blue Cross and Blue Shield Association 225 North Michigan Avenue Chicago, Illinois Registered marks of the Blue Cross and Blue Shield Association, an Association of Independent Blue Cross and Blue Shield Plans Registered trademark of Kaiser Permanente 2013 Blue Cross and Blue Shield Association. Reproduction without prior authorization is prohibited.