PEER REVIEW HISTORY BMJ Open publishes all reviews undertaken for accepted manuscripts. Reviewers are asked to complete a checklist review form (http://bmjopen.bmj.com/site/about/resources/checklist.pdf) and are provided with free text boxes to elaborate on their assessment. These free text comments are reproduced below. ARTICLE DETAILS TITLE (PROVISIONAL) AUTHORS Developing and validating a risk prediction model for acute care based on frailty syndromes Soong, John; Poots, Alan; Scott, Stuart; Donald, Kelvin; Bell, Derek VERSION 1 - REVIEW REVIEWER REVIEW RETURNED Gordon, Adam University of Nottingham, Division of Rehabilitation and Ageing 05-Jun-2015 GENERAL COMMENTS This paper describes a project to develop a frailty index which is predictive of adverse outcomes in older patients presenting to acute care. It uses the prevalence or absence of frailty syndromes as computed from numerous ICD-10 codes included in natively generated HES data and considers whether these were predictive of outcome measures (length of stay, mortality and functional dependency determined through the proxy measure of institutionaltype as stratified into a tiered hierarchy) collected from the same HES database. The rationale for the study is well described. The clinical justification is evident. The analytical and statistical methods are reported in comprehensive detail. The limitations of the study that have been recognised by the authors are transparently described, as are the strengths of their approach. I have, however, a number of misgivings which I think the authors should be encouraged to address through a series of revisions ahead of publication. These are as follows: - The comprehensive and detailed reporting of methods results in a report which is densely written and loaded with statistical jargon. As an academic, well versed in the frailty literature and statistically literate, I found myself having to read and re-read several segments of the report to clarify what had been done and why. The core readership of the BMJ Open, which comprises doctors without advanced statistical training or topic expertise in frailty will, I suspect, struggle to an even greater extent. The authors should work with clinicians within their institution who are removed from the project to de-jargonise the report and make it accessible to a general medical readership. I think this will require extensive editing. - Related to the above point, the report is peppered with column headings from the HES database which are not explained, either in the text or in an appendix glossary (I had to, for example, google "admimeth" to find out it was a column header from HES). These abbreviations should either be explained in the text or removed in
full. - The assumption that institutional type is a reasonable proxy for functional dependency is subject to some considerable debate and possibly flawed. We know, for example, from cohort and epidemiological studies that the overlap in terms of functional dependency between residential and nursing homes (care homes with and without nursing) is very considerable. Further there is marked heterogeneity between the thresholds for transfer into and out of homes in the residential care sector. Thus for any two given patients with a step increase in frailty, one may continue to live in their residential home, whilst another is moved onto nursing care. The authors need to consider the correlation, or otherwise, between institutional setting and functional status if they are to mount a robust defence of this as an outcome measure (or they need a frank discussion of this in their limitations setting). - Allied to the above, the rationale for the division of care settings into a five tiered hierarchy is not clear to me. Is this based upon how these data are encoded within HES? Is this empirical, if so what is the justification? Is it based upon an established hierarchy used in other studies? Some of the distinctions drawn in the hierarchy would be viewed with some degree of uncertainty by readers who work in and with the long term care sector. Why are NHS care homes a hierarchy level above privately run care homes? NHS care homes are vanishingly rare. Most NHS-funded long-term care placements are hosted within the private long-term care sector. Was the actual distinction here between those patients receiving NHS long-term care funding and those who were funded from the social care budget and privately? If so, the terminology requires some tidying up. The authors might also want to consider the variability and heterogeneity with which criteria for NHS long-term funding are applied between geographical settings (which loops back to the potential pitfalls in using venue of care as a proxy for dependency as outlined above). - Further related to the above point - the term "nursing home" has a number of different connotations for international readers, as does "care home" (see recent publication by Orrell et al, which coincidentally, got the definition of a UK nursing home wrong(!) but at least serves to demonstrate the considerable heterogeneity in terminology between countries). Other terms used in the report, like "Part 3 residential accommodation" mean nothing outside of England and Wales. The authors need to unpackage these terms for international readers if their report it going to be comprehensible to people working outside of England and Wales. It will take a bit of work for the authors to address these concerns and I think, therefore, that they amount to major revisions. But there remains here a very valuable, high quality piece of academic work that deserves to be reported to the broadest readership. If they can address my concerns I'd be happy to provide further peer-review input to see if we can get this into BMJ Open. REVIEWER REVIEW RETURNED Parker, S 08-Jun-2015 GENERAL COMMENTS INTRODUCTION The introduction could demonstrate a more coherent grasp of the
relevant concepts related to ageing and frailty. In paragraph 1 the authors assert boldly, that the population living to a greater age is associated with increased co-morbidity and functional dependence and poorer quality of life. The latter assertion of poorer quality of life is supported by an incomplete reference to a DEFRA document, which I believe says the following in discussion of wellbeing and age Age also appeared strongly related to satisfaction with certain aspects of life. Whilst satisfaction with certain aspects of life tends to increase with age feeling part of a community, future financial security, satisfaction with other aspects decreases achieving goals, the ability to influence one s life and health. The phrase poorer quality of life does not reflect the content of the document quoted except by an abstraction to emphasise the negative, and does not reflect the broader literature evidence on the impacts of ageing and age related disease on the way in which individuals manage their expectations, and as a consequence their perceptions of their quality of life. In my view this paragraph should be revised to either reflect the content of the quoted document in a more balanced way, or otherwise re-worded to remove the over-simplification about the impact of ageing on quality of life. The 2nd and third paragraphs of the introduction try to set the scene for the development and importance of the frailty concept, but they are not yet very persuasive. In my view they could be re-worked to advantage, to make a more coherent argument for the identification and measurement of frailty in the acute clinical setting and arguing why it s identification and measurement in the clinical setting / population may be of importance for the organisation and delivery of (particularly acute) health care Some detail related to this: Paragraph 2 is about assessing frailty. It conflates the notion that there is controversy about defining frailty (because there are two competing conceptual models) with the idea that there there are at least two approaches to measuring frailty (derived from the two conceptual models) and the clinical intuition that measuring the presence and magnitude of frailty of an individual may be a useful component of a clinical assessment (because it may help to predict clinical need, or clinical risk). Paragraph 3 then discusses the Geriatric giants and equates them to frailty syndromes. The Geriatric Giants is a term generally attributed (as the authors do) to Bernard Isaacs, and was essentially a mnemonic for the common clinical presentations of illness in old age. The mnemonic developed into the 4 Is Incontinence, Immobility, Instability and Intellectual impairment, does not usually include pressure sores. The development of a concept of Geriatric Clinical Syndromes is usually attributed to Sharon Inouye, and is an empirical derivation. The paragraph then brings in Kenneth Rockwoods concept of frailty associated clinical syndromes as a final common pathway representing complex system failure and the loss of high level functions such as balance, continence and cognition. Nowhere in either of these introductory paragraphs (nor anywhere else in the manuscript) is the seminal work of Linda Fried referred to drectly, and yet it is a key component of much of the controversy about the definition and measurement of frailty that is alluded to. Accordingly the connections between concepts of frailty, associated clinical syndromes, their association with key outcomes of illness in old age and hence their potential importance in the prediction of need, risk or outcomes in acute medical care are not well made.
Finally in the introduction an overarching hypothesis and 2 research aims are presented. The research aims are coherent and accurately reflected in the text, but to this reader the hypothesis being explored frailty syndromes are a valid measure of frailty appears to be a tautology. METHODS AND RESULTS The methods appear to be clearly described and the results are presented in a comprehensive set of tables and figures. DISCUSSION In parts the discussion is strong in that it makes the case for the strengths and weaknesses fot he approach and dataset used and sets the work in a coherent context. It appears to this reader that the key context is risk stratification for older people in acute care and that the concept of frailty is being used as a potential dimension, or determinant of acute care risk. It would be helpful if the key concepts could be more clearly delineated in the discussion: For example the authors refer to frailty scales and frailty scores and appear to use the terms interchangeably. The relationships between these two concepts and the literature data on frailty scores (such as the Rockwood frailty index, a deficit based model, or the use of the Fried criteria a physiology based model), and existing risk stratification tools are not clearly distinguished. VERSION 1 AUTHOR RESPONSE Reviewer Name: Adam Gordon; Institution and Country University of Nottingham, UK This paper describes a project to develop a frailty index which is predictive of adverse outcomes in older patients presenting to acute care. It uses the prevalence or absence of frailty syndromes as computed from numerous ICD-10 codes included in natively generated HES data and considers whether these were predictive of outcome measures (length of stay, mortality and functional dependency determined through the proxy measure of institutional-type as stratified into a tiered hierarchy) collected from the same HES database. The rationale for the study is well described. The clinical justification is evident. The analytical and statistical methods are reported in comprehensive detail. The limitations of the study that have been recognised by the authors are transparently described, as are the strengths of their approach. >>>>>>Thank you for representing our paper well The comprehensive and detailed reporting of methods results in a report which is densely written and loaded with statistical jargon. As an academic, well versed in the frailty literature and statistically literate, I found myself having to read and re-read several segments of the report to clarify what had been done and why. The core readership of the BMJ Open, which comprises doctors without advanced statistical training or topic expertise in frailty will, I suspect, struggle to an even greater extent. The authors should work with clinicians within their institution who are removed from the project to de-jargonise the report and make it accessible to a general medical readership. I think this will require extensive editing. Related to the above point, the report is peppered with column headings from the HES database which are not explained, either in the text or in an appendix glossary (I had to, for example, google "admimeth" to find out it was a column header from HES). These abbreviations should either be explained in the text or removed in full. >>>>>>>Thank you so much for your insightful comments. We have amended the text in main document to ensure clarity. The assumption that institutional type is a reasonable proxy for functional dependency is subject to some considerable debate and possibly flawed. We know, for example, from cohort and
epidemiological studies that the overlap in terms of functional dependency between residential and nursing homes (care homes with and without nursing) is very considerable. Further there is marked heterogeneity between the thresholds for transfer into and out of homes in the residential care sector. Thus for any two given patients with a step increase in frailty, one may continue to live in their residential home, whilst another is moved onto nursing care. The authors need to consider the correlation, or otherwise, between institutional setting and functional status if they are to mount a robust defence of this as an outcome measure (or they need a frank discussion of this in their limitations setting). Allied to the above, the rationale for the division of care settings into a five tiered hierarchy is not clear to me. Is this based upon how these data are encoded within HES? Is this empirical, if so what is the justification? Is it based upon an established hierarchy used in other studies? Some of the distinctions drawn in the hierarchy would be viewed with some degree of uncertainty by readers who work in and with the long term care sector. Why are NHS care homes a hierarchy level above privately run care homes? NHS care homes are vanishingly rare. Most NHS-funded long-term care placements are hosted within the private long-term care sector. Was the actual distinction here between those patients receiving NHS long-term care funding and those who were funded from the social care budget and privately? If so, the terminology requires some tidying up. The authors might also want to consider the variability and heterogeneity with which criteria for NHS long-term funding are applied between geographical settings (which loops back to the potential pitfalls in using venue of care as a proxy for dependency as outlined above). Further related to the above point - the term "nursing home" has a number of different connotations for international readers, as does "care home" (see recent publication by Orrell et al, which coincidentally, got the definition of a UK nursing home wrong(!) but at least serves to demonstrate the considerable heterogeneity in terminology between countries). Other terms used in the report, like "Part 3 residential accommodation" mean nothing outside of England and Wales. The authors need to unpackage these terms for international readers if their report it going to be comprehensible to people working outside of England and Wales. >>>>>>Thank you very much for your very useful comments. We have amended main document to reflect these concerns, within methodology and the limitations sections. Additionally, we have clarified definitions of care homes within main text as well. It will take a bit of work for the authors to address these concerns and I think, therefore, that they amount to major revisions. But there remains here a very valuable, high quality piece of academic work that deserves to be reported to the broadest readership. If they can address my concerns I'd be happy to provide further peer-review input to see if we can get this into BMJ Open. >>>>>Thank you, we hope to have addressed these points, and that you will consider us at peerreview Reviewer Name: Stuart Parker; Institution and Country Newcastle University, UK An interesting paper, generally well conducted and described. >>>>>Thank you for your kind comments The introduction could demonstrate a more coherent grasp of the relevant concepts related to ageing and frailty. In paragraph 1 the authors assert boldly, that the population living to a greater age is associated with increased co-morbidity and functional dependence and poorer quality of life. The latter assertion of poorer quality of life is supported by an incomplete reference to a DEFRA document, which I believe says the following in discussion of wellbeing and age Age also appeared strongly related to satisfaction with certain aspects of life. Whilst satisfaction with certain aspects of life tends to increase with age feeling part of a community, future financial security, satisfaction with other aspects decreases achieving goals, the ability to influence one s life and health. The phrase poorer quality of life does not reflect the content of the document quoted except by an abstraction to emphasise the negative, and does not reflect the broader literature evidence on the impacts of ageing and age related disease on the way in which individuals manage their expectations, and as a consequence their perceptions of their quality of life. In my view this paragraph should be revised to either reflect the content of the quoted document in a more balanced way, or otherwise re-worded to
remove the over-simplification about the impact of ageing on quality of life. >>>>>>>>Thank you for your insightful comments, we have amended main text to reflect a better representation of the experience of frailty within the context of older persons The 2nd and third paragraphs of the introduction try to set the scene for the development and importance of the frailty concept, but they are not yet very persuasive. In my view they could be reworked to advantage, to make a more coherent argument for the identification and measurement of frailty in the acute clinical setting and arguing why it s identification and measurement in the clinical setting / population may be of importance for the organisation and delivery of (particularly acute) health care Some detail related to this: Paragraph 2 is about assessing frailty. It conflates the notion that there is controversy about defining frailty (because there are two competing conceptual models) with the idea that there are at least two approaches to measuring frailty (derived from the two conceptual models) and the clinical intuition that measuring the presence and magnitude of frailty of an individual may be a useful component of a clinical assessment (because it may help to predict clinical need, or clinical risk). Paragraph 3 then discusses the Geriatric giants and equates them to frailty syndromes. The Geriatric Giants is a term generally attributed (as the authors do) to Bernard Isaacs, and was essentially a mnemonic for the common clinical presentations of illness in old age. The mnemonic developed into the 4 Is Incontinence, Immobility, Instability and Intellectual impairment, does not usually include pressure sores. The development of a concept of Geriatric Clinical Syndromes is usually attributed to Sharon Inouye, and is an empirical derivation. The paragraph then brings in Kenneth Rockwood s concept of frailty associated clinical syndromes as a final common pathway representing complex system failure and the loss of high level functions such as balance, continence and cognition. Nowhere in either of these introductory paragraphs (nor anywhere else in the manuscript) is the seminal work of Linda Fried referred to directly, and yet it is a key component of much of the controversy about the definition and measurement of frailty that is alluded to. Accordingly the connections between concepts of frailty, associated clinical syndromes, their association with key outcomes of illness in old age and hence their potential importance in the prediction of need, risk or outcomes in acute medical care are not well made. >>>>>Thank you for your clear and helpful comments. We have amended main text and references to reflect a more clear picture of the trajectory of research into measuring frailty. Finally in the introduction an overarching hypothesis and 2 research aims are presented. The research aims are coherent and accurately reflected in the text, but to this reader the hypothesis being explored frailty syndromes are a valid measure of frailty appears to be a tautology. >>>>>>Thank you so much for your comments. We have amended main text to reflect a clearer objective that examines frailty syndromes within the context of risk stratification in older persons in the acute care setting The methods appear to be clearly described and the results are presented in a comprehensive set of tables and figures. In parts the discussion is strong in that it makes the case for the strengths and weaknesses for the approach and dataset used and sets the work in a coherent context. >>>>Thank you for representing our paper well. It appears to this reader that the key context is risk stratification for older people in acute care and that the concept of frailty is being used as a potential dimension, or determinant of acute care risk. It would be helpful if the key concepts could be more clearly delineated in the discussion:for example the authors refer to frailty scales and frailty scores and appear to use the terms interchangeably. The relationships between these two concepts and the literature data on frailty scores (such as the Rockwood frailty index, a deficit based model, or the use of the Fried criteria a physiology based model), and existing risk stratification tools are not clearly distinguished. >>>>Thank you for very useful comments. We have included a better description of this within the
introduction section, and amended main body of discussion section to reflect this as well. Thank you, we hope to have addressed these points, and that you will consider us at peer-review