PEER REVIEW HISTORY BMJ Open publishes all reviews undertaken for accepted manuscripts. Reviewers are asked to complete a checklist review form (http://bmjopen.bmj.com/site/about/resources/checklist.pdf) and are provided with free text boxes to elaborate on their assessment. These free text comments are reproduced below. ARTICLE DETAILS TITLE (PROVISIONAL) AUTHORS A protocol for developing, disseminating, and implementing a core outcome set for endometriosis Hirsch, Martin; Duffy, James; Barker, Claire; Hummelshoj, Lone; Johnson, Neil; Mol, Ben; Khan, Khalid; Farquhar, Cynthia VERSION 1 - REVIEW Professor Andrew Horne University of Edinburgh UK Chair of ESHRE Special Interest Group Endometriosis/Endometrium Lead of James Lind Alliance Priority Setting Partnership for Endometriosis Member of NICE Endometriosis Guideline Group and of ESHRE Endometriosis Guideline Group Medical Advisor to Endometriosis UK I receive grant funding from the National Institute for Health Research and Wellbeing of Women for pelvic pain and endometriosis research 18-Sep-2016 This manuscript describes the protocol for an important but challenging project to develop a core outcome set for endometriosis. The protocol is well-written and clearly presented. However, I have a few comments regarding the proposed project. 1. There is no acknowledgement that it is possible that none of the previous endometriosis studies used optimum outcome measures. Therefore the authors' strategy of performing a systematic review of 'endometriosis studies and involving key stakeholders in the endometriosis field' misses the opportunity to get key input from experts in other fields. This is clearly particularly important for pain but may well also be relevant for other areas eg psychology, medical co-morbidities, etc. 2. I think the authors should consider acknowledging the recommendations from: (A) Vincent et al. (Fertil Steril. 2010;93(1):62-7) on 'Pain scoring in endometriosis: entry criteria and outcome measures for clinical trials'. These recommendations may not have been the result of a formal consensus process, but the authors of this manuscript did try to assess the above issue by bringing together experts from different pain disciplines. (B) Dworkin et al. (Pain. 2005 113(1-2):9-19) on 'Core outcome measures for chronic pain clinical trials: IMMPACT recommendations'. The pain community (IMMPACT now ACTTION)
have done a huge amount of work on standardising and optimising trial design in chronic pain conditions in general, which has not been acknowledged in this paper. Whilst pain journals do not explicitly state that IMMPACT needs to be followed, most reviewers would criticise a pain trial not incorporating their recommendations. 3. I note that the authors are working with a consortium of 'Women's Health' journals. Given that endometriosis is a 'chronic pain condition', I think that it would be helpful if the team were able to engage with journals from other disciplines e.g. Pain, etc. to ensure take up of the outcomes in all fields. I am concerned that that working with Women's Health Journals alone could prevent collaboration rather than enhance it. 4. It would be helpful if the authors could specify the make-up of the steering group e.g. does this include pain medicine, psychology, etc? How has this group been selected? 5. I have concerns that such recommendations in future would be assumed to be sufficient, and thus future studies may collect less relevant data because they think the researchers are meeting appropriate standards by using this core set. Nowhere is a strategy mentioned to mitigate this risk. C. Matthew Peterson MD Division of Reproductive Endocrinology and Infertility - Utah Center for Reproductive Medicine Department of Obstetrics and Gynecology University of Utah Health Sciences Center 50 North Medical Drive Salt Lake City, Utah 84132 26-Sep-2016 The authors have described an ambitious and comprehensive plan to involve key stakeholders in endometriosis research, including patients, with the goal of developing a set of consistent measures to be collected in trials. The methods to arrive at consensus are valid and do not restrict innovative evaluations and measurements. Journal editors and national funding sources appear to be on board with the process. The rationale, methods and goals are well described and fully endorsed by this reviewer. I have no criticisms but only kudos for a long overdue standardization of definitions and measurements that will greatly facilitate future research and outcomes reporting. The articulation of these goals and processes is well done. VERSION 1 AUTHOR RESPONSE Comment 1. There is no acknowledgement that it is possible that none of the previous endometriosis studies used optimum outcome measures. Therefore, the authors' strategy of performing a systematic review of 'endometriosis studies and involving key stakeholders in the endometriosis field' misses the opportunity to get key input from experts in other fields. This is clearly particularly important for pain but may well also be relevant for other areas eg psychology, medical co-morbidities, etc.
Response: We agree previous endometriosis studies may not have reported optimum outcome measures The manuscript currently states a scoping review will identify potential instruments. We will follow the recently published, 13th September 2016, recommendations from the Core Outcome Measures in Effectiveness Trials (COMET) initiative and COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) initiative. Prinsen C, et al. How to select outcome measurement instruments for outcomes included in a Core Outcome Set a practical guideline. Trials 2016 17:449. Prinsen and colleagues recommends using three sources of information to identifying existing outcome measures: (1) systematic reviews, (2) literature searches, and (3) other sources, considered as optional. We have updated the manuscript accordingly and we have additionally included the clarifying statement This approach will ensure, for example, that all core outcomes will still be included in the highly unlikely event that all previous endometriosis studies failed to include a particular core outcome measure. Page 8 Line 235-242 Comment 2: I think the authors should consider acknowledging the recommendations from: Vincent et al. (Fertil Steril. 2010;93(1):62-7) on 'Pain scoring in endometriosis: entry criteria and outcome measures for clinical trials'. These recommendations may not have been the result of a formal consensus process, but the authors of this manuscript did try to assess the above issue by bringing together experts from different pain disciplines. (B) Dworkin et al. (Pain. 2005 113(1-2):9-19) on 'Core outcome measures for chronic pain clinical trials: IMMPACT recommendations'. The pain community (IMMPACT now ACTTION) have done a huge amount of work on standardising and optimising trial design in chronic pain conditions in general, which has not been acknowledged in this paper. Whilst pain journals do not explicitly state that IMMPACT needs to be followed, most reviewers would criticise a pain trial not incorporating their recommendations. Response 2: We have outlined the Art and Science of Endometriosis meeting and the IMMPACT recommendations within the introduction. Page 4-5 Line 139-162
Comment 3: I note that the authors are working with a consortium of 'Women's Health' journals. Given that endometriosis is a 'chronic pain condition', I think that it would be helpful if the team were able to engage with journals from other disciplines e.g. Pain, etc. to ensure take up of the outcomes in all fields. I am concerned that that working with Women's Health Journals alone could prevent collaboration rather than enhance it. Response 3: Recognizing that the current inconsistency in outcome reporting is a serious hindrance to progress in our specialty, seventy-eight editors of Women s Health journals have formed a consortium to support the development, dissemination, and implementation of core outcome sets. The CROWN initiative are actively engaging with relevant journals from other disciplines, including pain journals, to join the consortium. We will invite journal editors from other relevant disciplines to participate within the consensus process. Page 4 Line 122-129 Comment 4: It would be helpful if the authors could specify the make-up of the steering group e.g. does this include pain medicine, psychology, etc? How has this group been selected? Response 4: An international steering group, including healthcare professionals, researchers, and women with endometriosis, has been formed to guide the development of this core outcome set. The steering group has been established to make decisions regarding the study s methods, for example, determining the scope of the core outcome set and selecting appripaite consensus methods. The selection of core outcomes will be determining by a formal consensus method, i.e. modified Delphi method. All stakeholders will be invited to participate including gynaecologists, chronic pain experts, health psychologists, family physicians, researchers, and women with endometriosis. Anyone, anywhere is welcome to suggest an outcome to be entered into the consensus process and participate in the prioritization of outcomes. Page 5 Line 142-144
Line 146-152 Line 174-176 Comment 5: I have concerns that such recommendations in future would be assumed to be sufficient, and thus future studies may collect less relevant data because they think the researchers are meeting appropriate standards by using this core set. Nowhere is a strategy mentioned to mitigate this risk. Response 5: The development and use of a core outcome set does not enforce harmony at the expense of innovation. The existence or use of a core outcome set does not imply that outcomes in an endometriosis trial should be restricted. Rather, there is an expectation that the core outcomes will be collected and reported, making it easier for the results of trials to be compared, contrasted and combined as appropriate; while researchers continue to explore other outcomes as well. Page 4 Line 111-120 Comment 6: The authors have described an ambitious and comprehensive plan to involve key stakeholders in endometriosis research, including patients, with the goal of developing a set of consistent measures to be collected in trials. The methods to arrive at consensus are valid and do not restrict innovative evaluations and measurements. Journal editors and national funding sources appear to be on board with the process. The rationale, methods and goals are well described and fully endorsed by this reviewer. I have no criticisms but only kudos for a long overdue standardization of definitions and measurements that will greatly facilitate future research and outcomes reporting. The articulation of these goals and processes is well done. Response 6: VERSION 2 REVIEW Andrew Horne University of Edinburgh UK 21-Oct-2016 I am happy with the authors' responses to my comments.