Title:Examining the implementation of NICE Guidance: cross-sectional survey of the use of NICE interventional procedures guidance by NHS Trusts

Transcription

1 Author's response to reviews Title:Examining the implementation of NICE Guidance: cross-sectional survey of the use of NICE interventional procedures guidance by NHS Trusts Authors: Karin Lowson Michelle Jenks Alexandra Filby Louise Carr Bruce Campbell John Powell Version:3Date:12 June 2015 Author's response to reviews: see over

2 York Health Economics Consortium, University of York, York, YO10 5DD Nick Sevdalis Associate Editor Implementation Science Dear Prof. Sevdalis, We thank the reviewer for the comments and suggestions. Please find our responses in the table (overleaf). The page numbers refer to those in the revised manuscript. Yours Sincerely, Karin Lowson, Senior Associate, York Health Economics Consortium, University of York 1

3 Comment Response Para Pg no. Abstract I think it would be better for the sentence This study was therefore designed to understand to read to examine or to investigate Changed to examine 1 2 A cross-sectional survey was developed and distributed should have Changed to a cross-sectional survey questionnaire 2 2 the word questionnaire added I m not sure the following sentence is needed in the abstract: The response rate to the survey was 75% with 135 completed surveys received. Additionally, policies from 25% of acute NHS hospitals were identified and analysed. 2 Thank you for this comment. We considered removing this from the abstract but on reflection we believe that the response rate, and total number of responses, is valuable information and we would prefer to retain this. 3 2 However, resource implications should this read resource limitations? Changed to limitations 4 2 I think final sentence would be improved by moving the subject of sentence to the beginning: Guidance implementation may be facilitated through encouragement of shared learning by organisations such as NICE and open knowledge transfer between organisations. Introduction First sentence has a dangling modifier change to Various organisations exist worldwide that produce evidence-based guidance and recommendations about health care. Paragraph 2 If you are going to say (such as reminders only) I think you should also provide an example of strategies with multiple components. Either provide an example of multiple and single components or neither. *** It seems a bit facile to say future research in this area is needed. It would be better to say (something like) much more systematic research is required in future to build a stronger and more coherent evidence base (in your own words!) and explain why. Sentence amended as suggested 4 2 Sentence re-worded as suggested 1 4 Thank you for this comment. On balance we think the neither option is preferable and such as reminders only has therefore been deleted from the text In this sentence we were reporting the conclusion of a previous systematic review. We agree that there is room for improvement. The text has now been revised and made more precise and changed to: However, the review also concluded that the evidence base in this area is still limited and well-constructed empirical research looking at various implementation strategies is needed It might be best not to make a backwards time reference such as a decade ago just state the year in the text (not in parentheses) Amended as suggested 3 4 a combination of system, organisational and individual problems I think the word factors would be better than problems (problems seems a bit Amended as suggested 3 4 value-laden) You say Weng and colleagues but Sheldon et al. could you make these types of references consistent in style? Yes we have revised this for consistency. All and colleagues have been changed to et al The sentence These include a lack of management support leading to Qualifier deleted as suggested 1 5

4 poorer implementation compared with high level support being present [7, 8] does not need or benefit from the qualifier compared with high level support being present The word that is not needed in and that sufficient resources and staff engagement Not sure why there are some in-text dates such as Michie et al. (2011) *** The addition of the penultimate paragraph of the introduction about Michie/BCW feels descriptive and tacked-on (not focused and integrated). If the theory is going to be introduced I think there needs to be slightly more information about why it is relevant to the title/subject of the paper. The following sentence is too long and complex Given the limited evidence base around the barriers and enablers in the successful implementation of guidance, a self-administered cross- sectional survey of all acute NHS hospitals in the UK was conducted to evaluate current awareness and use of NICE Interventional Procedures guidance, and to identify barriers and enablers to the successful implementation of the guidance. I think it should be simplified and split into 2 or more shorter sentences. Method *** A draft questionnaire was developed iteratively through a review of previous studies in this area and a process of survey validation. What kind of review? What papers were reviewed? *** The survey was piloted in a subset of non-experts (20 participants). a bit more info on who these people were, how they were selected, and how the piloting was conducted might be helpful. That deleted as suggested 1 5 In-text dates removed as suggested As the reviewer has suggested we have now added text to explain the relevance of the theory here. The following text has been added to this paragraph: The behaviour change wheel considers how and why people change behaviour in professional practice. Given that the aim of implementing guidance is to change behaviour, using the wheel as a theoretical framework can help to evaluate how and when behaviour change interventions (such as guidance) are successfully introduced. As suggested, this has now been split into two sentences: Given the limited evidence base around the barriers and enablers in the successful implementation of guidance, a self-administered cross-sectional survey of all acute NHS hospitals in the UK was conducted. The survey aimed to evaluate current awareness and use of NICE Interventional Procedures guidance, and to identify barriers and enablers to the successful implementation of the guidance It was a targeted literature review. This explanation has been added to the text. The studies used were most of those referred to in the introduction and discussion; references have been added to the text. The survey was piloted on a convenience sample. This extra information has been added to the text, as suggested by the reviewer: The survey was piloted in a convenience sample (20 participants) who work at the University of York who have knowledge of the NHS

5 Thank you. As requested extra information has been added to explain which authors undertook the documentary analysis and the measures taken to ensure reliability and validity. The following text has been added (new text in italics): *** I would still like to see more information about the documentary analysis. Which of the authors did it? What assurances are there that the analysis was reliable and valid (was the analysis checked by anyone?) The online survey was distributed and hosted by a third party which third party? SurveyMonkey? SNAP? In order to maximise response rates I think it would be better to say in order to get as many responses as possible rather than to maximize response rate(s) It might be better to say Qualitative and quantitative analyses were combined rather than Both quantitative and qualitative analysis of the surveys was combined *** Please clarify what is meant by chi-squared tests to examine differences between proportions *** I don t think it s appropriate to say The checklist was relatively simple, as the survey methodology did not concern the majority of elements addressed in, for example, the 32-item COREQ checklist [16]. I mentioned COREQ in my previous review as an example of what elements might be included in a methods section you shouldn t really cite it as an to support why you are not including some elements of the checklist. This justification of your method needs to be reworked. The survey was supplemented and informed by a documentary analysis of NHS hospitals policies on implementing NICE guidance (specifically IP guidance where possible) carried out by two members of the research team (MJ and AF). Searches were undertaken initially of websites ending in nhs.uk and then more widely to identify hospital policies. A data extraction table was designed and populated with information from each policy and themes and characteristics were detected. Double data extraction was carried out by both reviewers (MJ and AF) and any discrepancies were resolved through discussion. Added: (Qa Research Ltd) specialising in public sector research This has been amended as suggested 3 6 This has been amended as suggested 4 6/7 We have clarified the text as requested to: a chi-square test to examine whether there were differences between types of respondent. We are making a slightly different point here, in that we are explaining why we used our own checklist.(we appreciate that COREQ could also be used to structure a methods section). Text amended: To ensure the processes were rigorous a review checklist was developed and completed [15]. The checklist was developed as the survey methodology did not concern the majority of elements addressed in many 4 6/7 4 6/7 4

6 yielded few data on many cases in many cases? The final sentence of the method is long, slightly confusing and ends with a dangling modifier. This should be corrected: Following the analyses, survey respondents were provided with a summary of results from the analyses and with reported examples of good practice in the implementation of NICE IP guidance, gathered as a result of the survey. Results The part about what the chi-square test does ( to test the assumption that there was no difference in the frequency of responses received by teaching hospitals and non-teaching hospitals ) could be stated in simpler/less statistical language Figure 2 should come before the text The content analysis identified five broad themes of generic good practice, described as follows. This text should probably also be under the heading enablers and before engagement *** Under engagement (and anywhere else in the results this occurs), I advise not to mention what previous studies have shown. That is a discussion point. The results section should only detail what was learned from the current research, and this should be discussed in light of the research literature in the discussion section. Under resources it might be best not to start with elsewhere as this reference implies some other reference, which isn t included in this section. Elsewhere from what? Discussion The first paragraph is one sentence long. I think this should be moved down to become the first sentence in the limitations paragraph and listed as a strength. Then go on to list the limitations. *** echoed in the behaviour change wheel (BCW) proposed by Michie et al.[12]. Clinician engagement stimulates behaviour change in other staff through persuasion.. Please spell this out in a bit more detail. How are they echoed in the BCW? Exactly which parts? What evidence do you have for clinician engagement persuading others to change behaviour? Is that your research or someone else s? Clarify and provide references as appropriate. published qualitative research checklists. Yes this could be phrased better. I has been changed to from. It now reads few data from many cases. Meaning there were lots of responses but few data per response. Amended to: Following the analyses, survey respondents were provided with a summary of results gathered as a result of the survey and with reported examples of good practice in the implementation of NICE IP guidance Amended to the sentence below: to test if there was a difference in the number of responses received by teaching hospitals and nonteaching hospitals 4 6/7 4 6/7 3 8 Amended as suggested 5 8 Deleted from the engagement paragraph and the rest of the results section has been checked (no other occurrences were identified). Elsewhere deleted as suggested 3 10 Moved as suggested 4 14 Text has been amended to make it clearer where we are relating the text back to the BCW (new text in italics): Clinician engagement stimulates behaviour change in other staff. It is plausible to assume this may be a method of persuasion (as outlined in the BCW). Having clear processes and resources available enables the guidance to be implemented. Clear organisational

7 The current study suggests that there has been some improvement over the past decade.. The current study doesn t really suggest this it would be more accurate to say compared with the findings of previous studies, the findings of the current study suggest x. *** It seems to me that the first 7 paragraphs of the discussion are essentially summarizing the narrative of the results and comparing these findings to the existing research literature. I think this is a good and appropriate thing to do, but I think this should be done more clearly and succinctly. i.e. What is the overall picture of the main findings (1 paragraph); next, a few paragraphs that go into more detail about the specific findings listed in paragraph 1, and make comparisons to other relevant research where appropriate. structures, including committees, is defined in the BCW by environmental restructuring and through coercion. For example, when there is a non-response to new guidance, this is often escalated to committee level. Finally, sharing information is a form of education (as outlined in the BCW) which increases understanding within and between hospitals and hospital departments. Thank you. This has been amended to: Compared with the findings of previous studies, the findings of the current study suggest that there has been some improvement in the processes associated with guidance implementation and monitoring We have added the following first paragraph (below) and removed the first part of paragraph two. However, the authors feel that the rest of the discussion does as the reviewer has suggested (go into more detail about specific findings and make comparisons to research where appropriate), therefore, the subsequent paragraphs have not been amended. This study identified the processes used by NHS acute hospitals to implement and monitor guidance. The main findings of this research identified key barriers and enablers to guidance implementation in NHS acute hospitals. These are discussed in more detail in the succeeding discussion. These findings can be used by both NHS acute hospitals and organisations like NICE, which produce guidance in order to make the process of implementing guidance easier The final paragraph ( Although the survey focused on NICE IP guidance, the findings are relevant to the implementation of other types of guidance. Indeed, it was evident in the responses received that respondents were often discussing guidance implementation generally, in particular with respect to barriers and enablers for implementation. ) is short and I don t understand how it stands alone. Can this be appended onto the previous paragraph? Additional file 2 See my earlier comment about COREQ. I don t think this is needed, and any important elements of this should be recorded in the main text of As suggested this has now been added to the previous paragraph 4 14/15 Additional file removed from submission N/A N/A 6

8 the paper 7