PEER REVIEW HISTORY ARTICLE DETAILS TITLE (PROVISIONAL)

Transcription

1 PEER REVIEW HISTORY BMJ Open publishes all reviews undertaken for accepted manuscripts. Reviewers are asked to complete a checklist review form ( and are provided with free text boxes to elaborate on their assessment. These free text comments are reproduced below. ARTICLE DETAILS TITLE (PROVISIONAL) AUTHORS Using probabilistic record linkage methods to identify Australian Indigenous women on the Queensland Pap Smear Register: the National Indigenous Cervical Screening Project Whop, Lisa; Diaz, Abbey; Baade, Peter; Garvey, Gail; Cunningham, Joan; Brotherton, Julia; Canfell, Karen; Valery, Patricia; O'Connell, Dianne; Taylor, Catherine; Moore, Suzanne; Condon, John VERSION 1 - REVIEW REVIEWER REVIEW RETURNED Sergio Miranda Freire Universidade do Estado do Rio de Janeiro, Brazil 07-Sep-2015 GENERAL COMMENTS The paper is well written, clear, with a sound methodology and presents a thorough discussion of its methods and results. There are some points that need to be better explained. Title: the title of the paper is a bit misleading because it may be interpreted that the authors are comparing record linkage methods which is not the case. I suggest that the authors change the title to something like "Using probabilist record linkage to identify...". Abstract: I suggest that the sentence "There was a range of 0.08 % (2.00%-2.08%) when implementing..." in the results section be rephrased to something like "The proportion of Indigenous women ranged from 2.00% to 2.08% when using different algorithms to define Indigenous status." Strenghts and Limitations of the study: either the authors explain what "robust" means in the phrase "robust record linkage methodology" or remove this adjective from the text. Methods: 1) In lines (page 7), the authors state that "Given the number of records in each of the QHAPDC and the PSR, the QRLG was not able to conduct a probabilistic matching of the two complete datasets." Please elaborate more on this. I agree that the strategy of using the two extracts of the QHAPDC is adequate and more efficient for the purposes of the study but I wonder why the QRLG was not able to conduct a probabilistic linkage of the two complete datasets based on their number of records. 2) In lines 20-22, the authors state that they used 11 as the threshold value to classify pairs of records as matches. However, in the discussion section (lines 38-40, page 13), the threshold is 12, which is also the minimum value used in figure 2. Please clarify it. In relation to figure 2, the authors presented a bar chart for the number of potential pairs for each integer value of the combined probabilistic weighting score above of equal to 12. Since the

2 combined weighting score is a continuous variable, the authors must explain how they created those bar charts. Does the plot means, for example, that all pairs or records whose combined score was equal or greater than 13 and lower than 14 were grouped under the 13 label? If that is the case, it must be stated in the text. 3) I answered "No" to the question "Are the methods described sufficiently to allow the study to be repeated" in the review checklist. This answer is not due to any weakness in the study methods' section, but to the fact that in order to replicate the study would require the access to the identified datasets, which would be a hard if not impossible task due to ethical reasons. Figures: The legend in Figures 1 and 3 is missing. Some typos: 1) adoloescent (page 15, line 52) 2) interst (page 17, line 36) 3) probabalistic in Fig. 2 (in all three charts) REVIEWER REVIEW RETURNED Orrin Myers University of New Mexico United States of America 22-Oct-2015 GENERAL COMMENTS Overall the manuscript is well-written, however there is little evaluation of the feasibility and reliability of the linkage that the title implies. Abstract says 88% accuracy for ascertainment of indigenous status but Page 7 Line 53 says 87% based on Ref 15. This reference also found important variation in ascertainment accuracy by remoteness. Assuming 87% is the Indigenous detection rate of hospital records for true Indigenous women, how would this affect linkage reliability? Figure 2 shows counts not proportions. Unclear what this figure adds to paper. Suggest deleting or moving to supplement with brief summary of linkage quality perhaps as part of the results. Approvals Figure 3 seems like it could be deleted and handled in the text. Indigenous Status Algorithms Page 12 Line 52. Authors state that proportions of Indigenous status classifications were similar over age and remoteness categories. Inspection of Fig 4 and Fig 5 support this, but a statistical test for each would be useful to confirm it. Without some context, like a population measure of % Indigenous women by age or remoteness category, it s difficult to assess whether there could be bias. There may not be much opportunity for the algorithms to be different. What proportion of women hand only one record and thus have no opportunity for variation in classification? The discussion is far too long for such limited data and even more limited evaluation. Sections relating to benchmarks and policy changes seem beyond the scope of the paper. A useful addition would be to discuss the implications of the variation in Indigenous ascertainment rate of 72% in major cities to >=95% in outer regional to very remote locations (ref 15).

3 VERSION 1 AUTHOR RESPONSE Reviewer 1 Comments: 1. The paper is well written, clear, with a sound methodology and presents a thorough discussion of its methods and results. There are some points that need to be better explained. Thank you for your feedback we believe after addressing your comments and those of reviewer 2 the overall clarity of the manuscript has been improved. 2. Title: the title of the paper is a bit misleading because it may be interpreted that the authors are comparing record linkage methods which is not the case. I suggest that the authors change the title to something like "Using probabilist record linkage to identify..." Thank you, we have revised the title to now read Using probabilistic record linkage methods to identify Australian Indigenous women on the Queensland Pap Smear Register: the National Indigenous Cervical Screening Project (page 1, Title). 3. Abstract: I suggest that the sentence "There was a range of 0.08 % (2.00%-2.08%) when implementing..." in the results section be rephrased to something like "The proportion of Indigenous women ranged from 2.00% to 2.08% when using different algorithms to define Indigenous status." We have amended this sentence in the Abstract to now read "The proportion of Indigenous women ranged from 2.00% to 2.08% when using different algorithms to define Indigenous status" as suggested (page 3, paragraph 3 under Results). 4. Strengths and Limitations of the study: either the authors explain what "robust" means in the phrase "robust record linkage methodology" or remove this adjective from the text. As suggested, we have removed robust and the sentence now reads This study provides evidence that record linkage methodology can be used to identify Indigenous Australian women on Pap Smear Registers (PSRs). (Page 4, paragraph 1, under Strengths and Limitations of the study) Methods: 5. In lines (page 7), the authors state that "Given the number of records in each of the QHAPDC and the PSR, the QRLG was not able to conduct a probabilistic matching of the two complete datasets." Please elaborate more on this. I agree that the strategy of using the two extracts of the QHAPDC is adequate and more efficient for the purposes of the study but I wonder why the QRLG was not able to conduct a probabilistic linkage of the two complete datasets based on their number of records. At the time of our data request there was a change in the office of Queensland Government which resulted in considerable restructure to departments and the linkage group were largely under resourced. Linking the entire hospital data set would have meant extracting and linking several million women as opposed to only linking records for Indigenous women; a much smaller proportion of hospital records and the group we are interested in. We have expanded on our reference to this on page 10, paragraph 3, under approvals which now reads Changes in the State government following a general election in March 2012 reduced the QRLG s resources, which halted progress for several months and prevented linkage of the entire QHAPDC dataset. Consequently the linkage method was revised to use the two extracts. 6. In lines 20-22, the authors state that they used 11 as the threshold value to classify pairs of records as matches. However, in the discussion section (lines 38-40, page 13), the threshold is 12, which is

4 also the minimum value used in figure 2. Please clarify it. Our apologies the threshold was set at 12 not at 11, those with 11 or less were not reviewed and those with a weighting of 12 or over were reviewed. We have amended this sentence to now read Those with a weighting 12 and above were reviewed. (page 7, paragraph 1) and have made this clearer throughout the manuscript (page 11, paragraph 2, under linkage quality). 7. In relation to figure 2, the authors presented a bar chart for the number of potential pairs for each integer value of the combined probabilistic weighting score above of equal to 12. Since the combined weighting score is a continuous variable, the authors must explain how they created those bar charts. Does the plot means, for example, that all pairs or records whose combined score was equal or greater than 13 and lower than 14 were grouped under the 13 label? If that is the case, it must be stated in the text. The linkage output provided the number of potential pairs of women within each probabilistic weighting score category. The score was given in whole numbers and treated as ordinal. Note, that this Figure has now been moved to a supplementary file as suggested by Reviewer 2 (see our response to comment 14) 8. I answered "No" to the question "Are the methods described sufficiently to allow the study to be repeated" in the review checklist. This answer is not due to any weakness in the study methods' section, but to the fact that in order to replicate the study would require the access to the identified datasets, which would be a hard if not impossible task due to ethical reasons. Thank you for your clarification we note that whilst obtaining the linked datasets was challenging and lengthy in process it is possible particularly if processes in obtaining the data are more streamlined. 9. Figures: The legend in Figures 1 and 3 is missing. Thank you for pointing this out we have now added a legend to the bottom of Figure 1. We have deleted Figure 3 as suggested by Reviewer 2 see our response to comment Some typos: 1) adoloescent (page 15, line 52), we have corrected this typo. 2) interst (page 17, line 36), we have corrected this typo. 3) probabalistic in Fig. 2 (in all three charts) we have amended this typo on all three occasions within this figure. Reviewer: Overall the manuscript is well-written, however there is little evaluation of the feasibility and reliability of the linkage that the title implies. Thank you, we have amended the title, see our response to comment Abstract says 88% accuracy for ascertainment of indigenous status but Page 7 Line 53 says 87% based on Ref 15. We were unable to find the 88% within the abstract that the Reviewer has mentioned. To clarify, on page 5 paragraph 2, we do make reference to 88% which is the national agreement between hospital

5 records and self-report of Indigenous status which does differ to the Queensland accuracy reported at 87%. 13. This reference also found important variation in ascertainment accuracy by remoteness. Assuming 87% is the Indigenous detection rate of hospital records for true Indigenous women, how would this affect linkage reliability? Indigenous status is not used as a linkage variable therefore the variation in accuracy by remoteness of Indigenous status within QHAPDC would not affect the linkage reliability. Indigenous identification in QHAPDC is lower in Major Cities (72%) so we have not identified all Indigenous women living in Major Cities (please also refer to our response to comment 19). 14. Figure 2 shows counts not proportions. Unclear what this figure adds to paper. Suggest deleting or moving to supplement with brief summary of linkage quality perhaps as part of the results. Thank you for pointing out this error, we have amended this sentence to now read Possible matches were either accepted or rejected after clerical review, and the total number of possible matches accepted as true matches or rejected matches were calculated at each probability score. (page 8, paragraph 3, under Assessment of the Linkage Quality). We have also decided to change Figure 2 to a supplementary figure as suggested by the reviewer we feel the current description on page 11 under Linkage of PSR, QCR and QHAPDC is sufficient. Instead we have added a sentence to the end of the paragraph referring the reader to Supplementary material The number of potential matches rejected or accepted at each weighting score for each part of the linkage is detailed in Supplementary Figure Approvals Figure 3 seems like it could be deleted and handled in the text. We agree and have deleted references to Figure 3 throughout the manuscript. 16. Indigenous Status Algorithms Page 12 Line 52. Authors state that proportions of Indigenous status classifications were similar over age and remoteness categories. Inspection of Fig 4 and Fig 5 support this, but a statistical test for each would be useful to confirm it. Without some context, like a population measure of % Indigenous women by age or remoteness category, it s difficult to assess whether there could be bias. Thank you, the purpose of these figures is to show the difference in proportions across the use of each algorithm within our cohort. As statistical tests are sensitive to sample size, the large number of women in this cohort means that p-values can be misleading and therefore we have decided to not include them. However, we have included the proportion of Indigenous women in the estimated resident population by age-group and remoteness category as suggested by the reviewer (see new Figures 2 and 3 which have been renumbered due to deleting other figures at the request of the reviewers). Note, that the estimated resident population provides context about the proportion of Indigenous women in those categories but, our study, will only reflect the proportion of Indigenous women who have a record on the Pap Smear Register and linked to a hospital admission which was recorded as Indigenous. We have added in two sentences regarding this change For context, the proportion of Indigenous women within the estimated resident population (ERP) was also reported for five-year age groups and remoteness categories. The proportion of women within our cohort identified as Indigenous was expected to be lower than the ERP regardless of algorithm use given that screening rates are not 100%. (page 10 paragraph 1)

6 17. There may not be much opportunity for the algorithms to be different. What proportion of women had only one record and thus have no opportunity for variation in classification? 19% of Indigenous identified women from extract 1 only had one hospital admission (5729/30132). To check if there was a bias due to single admission we excluded women from the Ever Indigenous QHAPDC extract who had less than three hospital admissions within the study period. The proportion of women on the Pap Smear Register who were identified as Indigenous (who had 3 or more hospital admissions) was similar across the four algorithms (0.08% absolute difference). 18. The discussion is far too long for such limited data and even more limited evaluation. Sections relating to benchmarks and policy changes seem beyond the scope of the paper. We agree that the length of the discussion is too long. We also believe the extent to which we refer to policy changes and benchmarks made possible by this linkage needs to be reduced. We have deleted from paragraph 2 page 16 to the end of paragraph 1 page 17. We have revised this section to only be concerned with the reporting structure of the new cervical screening program in Australia. This now reads In 2014, after recommendation by the Medical Services Advisory Committee (MSAC), the Australian Government announced a renewed cervical screening program (known as the Renewal ) will be implemented by May The current implementation stage is concerned with, among other things, implementing a national data collection and register system. As a component of Renewal implementation the aim is to establish a national cervical screening register (real or virtual). Despite the difficulties in collecting information on Indigenous status at an individual level, we recommend that the work program for the national screening register considers this important issue, which will ultimately facilitate better delivery of care to Indigenous women. (page 17, paragraph 2) 19. A useful addition would be to discuss the implications of the variation in Indigenous ascertainment rate of 72% in major cities to >=95% in outer regional to very remote locations (ref 15). We agree with the reviewer and have now added in some information regarding the variation of Indigenous status accuracy by remoteness. The known, reasonably high, accuracy of the Indigenous identifier contained in the QHAPDC (87% accuracy) was a major advantage of this study The accuracy in the QHAPDC, however, also varied by remoteness areas. The accuracy of the Indigenous identifier improved with increasing remoteness where major cities reported 72% (95% CI 62%-80%) accuracy and Remote/Very Remote reported 100% (95% CI 88%-100%).15 This means up to 13% of Indigenous women in our cohort overall or up to 28% Indigenous women in Major Cities may have been incorrectly identified as non-indigenous or of unknown Indigenous status.15 While we are unable to quantify the exact extent of misclassification bias in our study, sensitivity analyses using correction factors devised by the AIHW, will be performed for certain outcome measures to account for potential under-identification of Indigenous women for both overall Queensland estimates and by remoteness.15 ( page 13 paragraph 3) We have also taken this opportunity to further clarify some points in the manuscript as follows: We have made some changes to page 14 regarding the number of Indigenous women who would have been hospitalised in the study time frame and available for linkage. This was an oversight on our original submission and we have subsequently added a small paragraph. This additional paragraph also meant it was better to rearrange two paragraphs. See changes on page 14. We have removed two references and added another, and subsequently this has changed the number of the references. We have changed the references both within the manuscript and in the reference list.

7 We have made some slight changes and additions to the acknowledgments section, page 18 paragraph 3. VERSION 2 REVIEW REVIEWER REVIEW RETURNED Sergio Miranda Freire Universidade do Estado do Rio de janeiro, Brazil 06-Dec-2015 GENERAL COMMENTS The authors answered my questions. Below are some minor issues. 1) Some corrections in the text: Page 3: Line 18: women, aged 20-69, who had ever identified as Indigenous Suggestion: women, aged 20-69, who had ever been identified as Indigenous.. line 24:, and the proportion of potential pairs that required clerical review. I suggest to keep to the former text: as well as the proportion of potential pairs that required clerical review. This makes it clear that this proportion is reported. When I read the modified text, I expected something after the sentence. Lines 31 e 32: the proportion of women identifying as Indigenous was calculating... Replace by the proportion of women identified as Indigenous was calculated 2) Typos: Page 15, line 13: Probabalistic Page 15, line 28: exlude and hysterectmoy Page 15, line 40: extact and plausable VERSION 2 AUTHOR RESPONSE 1. Page 3: Line 18: women, aged 20-69, who had ever identified as Indigenous Suggestion: women, aged 20-69, who had ever been identified as Indigenous.. We have amended this as suggested 1. Page 3: Line line 24:, and the proportion of potential pairs that required clerical review. I suggest to keep to the former text: as well as the proportion of potential pairs that required clerical review. This makes it clear that this proportion is reported. When I read the modified text, I expected something after the sentence. We have amended this as suggested Page 3, line Lines 31 e 32: the proportion of women identifying as Indigenous was calculating... Replace by the proportion of women identified as Indigenous was calculated We have amended this as suggested, page 3, line31-32.

8 4. Typos: Page 15, line 13: Probabalistic We have corrected this to Probabilistic, page 1, line 6 5. Page 15, line 28: exlude and hysterectomy We have corrected this to exclude and hysterectomy, page 15, line Page 15, line 40: extact and plausable We have corrected this to extract and plausible, page VERSION 3 - REVIEW REVIEWER REVIEW RETURNED Sergio Miranda Freire Universidade do Estado do Rio de Janeiro - Brazil 21-Dec-2015 GENERAL COMMENTS Congratulations