CHSD. Refining Continence Measurement Tools. Centre for Health Service Development

Transcription

1 CHSD Centre for Health Service Development Refining Continence Measurement Tools RefiningContinenceMeasurementToolsRefiningContinenceMeasurementT oolsrefiningcontinencemeasurementtoolsrefiningcontinencemeasurem enttoolsrefiningcontinencemeasurementtoolsrefiningcontinencemeasu rementtoolsrefiningcontinencemeasurementtoolsrefiningcontinenceme asurementtoolsrefiningcontinencemeasurementtoolsrefiningcontinenc emeasurementtoolsrefiningcontinencemeasurementtoolsrefiningcontin encemeasurementtoolsrefiningcontinencemeasurementtoolsrefiningco ntinencemeasurementtoolsrefiningcontinencemeasurementtoolsrefinin gcontinencemeasurementtoolsrefiningcontinencemeasurementtoolsref iningcontinencemeasurementtoolsrefiningcontinencemeasurementtools RefiningContinenceMeasurementToolsRefiningContinenceMeasurementT oolsrefiningcontinencemeasurementtoolsrefiningcontinencemeasurem enttoolsrefiningcontinencemeasurementtoolsrefiningcontinencemeasu rementtoolsrefiningcontinencemeasurementtoolsrefiningcontinenceme asurementtoolsrefiningcontinencemeasurementtoolsrefiningcontinenc emeasurementtoolsrefiningcontinencemeasurementtoolsrefiningcontin encemeasurementtoolsrefiningcontinencemeasurementtoolsrefiningco ntinencemeasurementtoolsrefiningcontinencemeasurementtoolsrefinin UNIVERSITY OF WOLLONGONG Centre for Health Service Development July, 26

2 Jan Sansoni Nick Marosszeky Emily Sansoni and Graeme Hawthorne Department of Psychiatry The University of Melbourne Suggested citation Sansoni J, Marosszeky N, Sansoni E and Hawthorne G (26) Refining Continence Measurement Tools (Final Report). Centre for Health Service Development, University of Wollongong and the Department of Psychiatry, The University of Melbourne.

3 Table of Contents 1 EXECUTIVE SUMMARY Aims Methods Results Urinary Incontinence Scales Faecal Incontinence Scales Conclusions and Recommendations INTRODUCTION Introduction and Background The Continence Outcome Measurement Suite Project The Measuring Incontinence in Australia Project Description of Continence Measures Included in SAHOS Short Form of the UDI The Incontinence Severity Index (ISI) Wexner Faecal Continence Grading Scale (Wexner FCGS) 1 3 INCONTINENCE MEASURES: FINDINGS FROM RECENT REPORTS Measures of Urinary Incontinence Measures of Faecal Incontinence RATIONALE OF STUDY Aims Overview of Methods DATA ANALYSIS: CLASSICAL APPROACH Item Properties Item Endorsement and Discrimination Item-total Correlations and Internal Consistency Reliability Exploratory Factor Analysis Relationships with Other Health Variables Norm Tables Summary Revised Urinary Incontinence Scale (RUIS) Items Revised Faecal Incontinence Scale (RFIS) Items 35 6 DATA ANALYSIS: ITEM RESPONSE THEORY Introduction Methods

4 6.3 Item Response Theory Basics Urinary Continence Measurement Descriptive Properties of Urinary Incontinence Items Analysis of Urinary Continence Measurement Constructing Improved Measurement Some Validation Tests of the UCA Faecal Continence Measurement Descriptive Properties of Faecal Incontinence Items Analysis of Faecal Continence Measurement Constructing Improved Measurement Some Validation Tests of the FCA Norm Tables CONCLUSIONS AND RECOMMENDATIONS Context Conclusions concerning urinary incontinence measures Classical Test Theory Analysis Modern Test Theory Analysis Relationships with Other Health Variables Conclusions concerning faecal incontinence measures Classical Test Theory Analysis Modern Test Theory Analysis Recommendations REFERENCES 58 APPENDIX 1: THE INSTRUMENTS USED AND THE REVISED INSTRUMENTS 63 Instruments included in 24 SAHOS Urinary Distress Inventory Incontinence Severity Index 63 Wexner Faecal Continence Grading Scale 64 Other Faecal Items (included in 24 SAHOS) 64 Revised Measures for Urinary and Faecal Incontinence Revised Urinary Incontinence Scale 66 Urinary Continence Assessment 66 Revised Faecal Incontinence Scale 67 Faecal Continence Assessment 67 APPENDIX 2: SUPPLEMENTARY FIGURES FOR SECTION 5 68 Additional Figures Demonstrating the Level of Non-endorsement for each Incontinence Item Additional Figures Showing the Distribution of Response for Each Incontinence Item APPENDIX 3: NORM TABLES 95

5 List of Tables Table 1 The Wexner Faecal Incontinence Grading System... 1 Table 2 Total number of subjects according to sex and age group in the SAHOS Data set (N = 2924)* Table 3 Labels, questions and abbreviations for the urinary incontinence items in the SAHOS data set Table 4 Labels, questions and abbreviations for the faecal incontinence items in the SAHOS data set Table 5 The item properties of the urinary incontinency items for males Table 6 The item properties of the urinary incontinency items for females Table 7 The item properties of the faecal incontinency items for males... 2 Table 8 The item properties of the faecal incontinency items for females... 2 Table 9 The item properties of the three incontinence measures (ISI, UDI-6 and Wexner FCGS) for males 21 Table 1 The item properties of the three incontinence measures (ISI, UDI-6 and Wexner FCGS) for females Table 11 Corrected item - total correlations and Cronbach s alpha if the item was deleted for each item of the UDI Table 12 Corrected item - total correlations and Cronbach s alpha if the item was deleted for each item of the Wexner Scale (X4, X5, X6, X8, X1) Table 13 The correlation matrix between the ISI, UDI-6 and Wexner FCGS according to the Pearson correlation coefficient (parametric) [n = 2911] Table 14 Correlation matrix for the 8 urinary incontinence items Table 15 Correlation matrix for the 1 faecal incontinence items... 3 Table 16 Rotated Factor Matrix for the urinary incontinence items W1 W Table 17 Rotated Factor Matrix for the faecal incontinence items X1 X Table 18 Pearson correlations between the ISI, UDI-6 and Wexner FCGS with other measures of health. Correlations.2 or greater are shown Table 19 Urinary continence item frequency distribution and PCA factor loading (item pooled) Table 2 IRT analysis of the UDI and ISI Table 21 Initial IRT analysis of all urinary incontinence items probability thresholds (items pooled)... 4 Table 22 IRT analysis of items psychometric properties (items pooled) Table 23 IRT analysis of items uniform DIF, by gender and age group (items pooled) Table 24 IRT analysis of the UCS measure, derived from the UDI and ISI pooled items Table 25 Preliminary comparison of the UCA with the UDI and ISI Table 26 Suggested cutpoints for interpreting the UCA Table 27 Discrimination of the UCA by stress incontinence and urination frequency Table 28 Faecal continence item frequency distribution and PCA factor loading (items pooled) Table 29 IRT analysis of the Wexner and other items Table 3 IRT analysis of the revised Wexner, excluding #3 and after recoding other items Table 31 IRT analysis of the Faecal Continence Assessment (FCA) scale... 5 Table 32 Suggested cutpoints for interpretation of the FCA... 5 Table 33 Preliminary comparison of the FCA with the Wexner Table 34 Discrimination of the FCA Table 35 Norm tables for males according to age group, using the SAHOS weighted sample Table 36 Norm tables for females according to age group, using the SAHOS weighted sample... 97

6 List of Figures Figure 1 The level of non-endorsement for males and females on Item W1 (Frequent Urination)...22 Figure 2 The level of non-endorsement for males and females on Item W5 (Emptying Bladder)...23 Figure 3 The level of non-endorsement for males and females on Item X4 (Leak Solid)...24 Figure 4 The level of non-endorsement for males and females on Item X6 (Leak Gas)...24 Figure 5 The distribution of responses for females on item W4 (Leak small amount)...25 Figure 6 The distribution of responses for males on item W4 (Leak small amount)...25 Figure 7 The distribution of responses for females on item X4 (Leak solid)...26 Figure 8 The distribution of response for females on item X3 (urgency)...26 Figure 9 Item characteristic curve (ICC) for #1, UDI, frequent urination...37 Figure 1 Probability curves for ISI, #1, How often is there urinary leakage...4 Figure 11 DIF for the UDI, #3, Urine leakage related to activity, by gender...42 Figure 12 Faecal continence #7, number of bowel movements, disordered thresholds...46 Figure 13 Percentage of respondents who answered Not at all for item W1 (UDI-6, Item 1)...68 Figure 14 Percentage of respondents who answered Not at all for item W2 (UDI-6, Item 2)...68 Figure 15 Percentage of respondents that answered Not at all for item W3 (UDI-6, Item 3)...69 Figure 16 Percentage of respondents who answered Not at all for item W4 (UDI-6, Item 4)...69 Figure 17 Percentage of respondents who answered Not at all for item W5 (UDI-6, Item 5)...7 Figure 18 Percentage of respondents who answered Not at all for item W6 (UDI-6, Item 6)...7 Figure 19 Percentage of respondents who answered Never for item W7 (ISI, Item 1)...71 Figure 2 Percentage of respondents who answered None for item W8 (ISI, Item 2)...71 Figure 21 Percentage of respondents who answered Normal for item X Figure 22 Percentage of respondents who answered Never for item X Figure 23 Percentage of respondents who answered Never for item X4 (Wexner, Item 1)...73 Figure 24 Percentage of respondents who answered Never for item X5 (Wexner, Item 2)...73 Figure 25 Percentage of respondents who answered Never for item X6 (Wexner, Item 3)...74 Figure 26 Percentage of respondents who answered Never for item X Figure 27 Percentage of respondents who answered Never for item X8 (Wexner, Item 4)...75 Figure 28 Percentage of respondents who answered Never for item X Figure 29 Percentage of respondents who answered Never for item X1 (Wexner, Item 5)...76 Figure 3 Percentage of males that selected each response option in item W1 for each age group...77 Figure 31 Percentage of females that selected each response option in item W1 for each age group...77 Figure 32 Percentage of males that selected each response option in item W2 for each age group...78 Figure 33 Percentage of females that selected each response option in item W2 for each age group...78 Figure 34 Percentage of males that selected each response option in item W3 for each age group...79 Figure 35 Percentage of females that selected each response option in item W3 for each age group...79 Figure 36 Percentage of males that selected each response option in item W4 for each age group...8 Figure 37 Percentage of females that selected each response option in item W4 for each age group...8 Figure 38 Percentage of males that selected each response option in item W5 for each age group...81 Figure 39 Percentage of females that selected each response option in item W5 for each age group...81 Figure 4 Percentage of males that selected each response option in item W6 for each age group...82

7 Figure 41 Percentage of females that selected each response option in item W6 for each age group Figure 42 Percentage of males that selected each response option in item W7 for each age group Figure 43 Percentage of females that selected each response option in item W7 for each age group Figure 44 Percentage of males that selected each response option in item W8 for each age group Figure 45 Percentage of females that selected each response option in item W8 for each age group Figure 46 Percentage of males that selected each response option in item X1 for each age group Figure 47 Percentage of females that selected each response option in item X1 for each age group Figure 48 Percentage of males that selected each response option in item X2 for each age group Figure 49 Percentage of females that selected each response option in item X2 for each age group Figure 5 Percentage of males that selected each response option in item X3 for each age group Figure 51 Percentage of females that selected each response option in item X3 for each age group Figure 52 Percentage of males that selected each response option in item X4 for each age group Figure 53 Percentage of females that selected each response option in item X4 for each age group Figure 54 Percentage of males that selected each response option in item X5 for each age group Figure 55 Percentage of females that selected each response option in item X5 for each age group Figure 56 Percentage of males that selected each response option in item X6 for each age group... 9 Figure 57 Percentage of females that selected each response option in item X6 for each age group... 9 Figure 58 Percentage of males that selected each response option in item X7 for each age group Figure 59 Percentage of females that selected each response option in item X7 for each age group Figure 6 Percentage of males that selected each response option in item X8 for each age group Figure 61 Percentage of females that selected each response option in item X8 for each age group Figure 62 Percentage of males that selected each response option in item X9 for each age group Figure 63 Percentage of females that selected each response option in item X9 for each age group Figure 64 Percentage of males that selected each response option in item X1 for each age group Figure 65 Percentage of females that selected each response option in item X1 for each age group... 94

8 List of Abbreviations 15D AIHW AQoL COMS CTT DIF EQ5D FCA ICC IIQ IRT ISI MCS MTT PCA PCS PSI RFIS RUIS S.E. mean SAHOS SD SF-36 UCA UDI UDI-6 Wexner FCGS 15D Measure of Health Related Quality of Life Australian Institute of Health and Welfare Assessment of Quality of Life Continence Outcome Measurement Suite Project classical test theory differential item functioning EuroQol, European Quality of Life Measure Faecal Continence Assessment item characteristic curve Incontinence Impact Questionnaire Item Response Theory Incontinence Severity Index Mental Component Summary score of the SF-36 modern test theory Principal Component Analysis Physical Component Summary score of the SF-36 Person Separation Index Revised Faecal Incontinence Scale Revised Urinary Incontinence Scale standard error of the mean South Australian Health Omnibus Survey standard deviation Short Form 36 Item Health Survey (Medical Outcomes Study) Urinary Continence Assessment Urogenital Distress Inventory Urogenital Distress Inventory 6 item, Wexner Faecal Continence Grading Scale

9 1 Executive summary 1.1 Aims A Continence Outcome Measurement Suite Project (COMS) was commissioned by the Australian Government Department of Health and Ageing, National Continence Management Strategy Research Program with the goal of recommending a suite of continence outcome measures to be used by clinicians and researchers in Australia. This project was finalised in early 26 (Thomas et al., 26). Recommendations from this report led to a related project Measuring Incontinence in Australia (Hawthorne, 26). Measuring Incontinence in Australia (26) assessed a number of the recommended measures (Urogenital Distress Inventory 6, Incontinence Severity Index, and the Wexner Faecal Continence Grading Scale) by including them in the autumn 24 South Australian Health Omnibus Survey (Harrison Health Research, 24), which was a community population survey. The Hawthorne (26) study provided Australian prevalence estimates for both faecal and urinary incontinence based on this community survey data. For urinary incontinence, the results suggested that the preferred urinary incontinence measure was the Incontinence Severity Index (ISI). It was found to possess superior measurement properties in comparison with the Urogenital Distress Inventory (UDI-6). Because the UDI-6 measures the impact of urinary incontinence on peoples lives rather than incontinence per se, and may contain items that may be endorsed by those without urinary incontinence, the UDI-6 may overstate incontinence prevalence and the impact of this on peoples lives (defined as their health status and their quality of life). Given its poor psychometric properties, there was a prima facie case for major revision of the UDI-6. Although the ISI was the preferred measure, because it violated the assumptions of classic psychometric theory relating to scale stability as it contains only two items, further research into its properties was also recommended. A main purpose of the current study was to undertake further analysis of the SAHOS dataset to refine these urinary incontinence measures to provide a better instrument for the assessment of urinary incontinence in Australia. Regarding the measurement of faecal incontinence, as Hawthorne (26) indicated the current definition of faecal incontinence by the International Continence Society excludes flatus, yet this is included in the Wexner Faecal Continence Grading System (Wexner FCGS). In addition to this definitional inconsistency, the evidence from Hawthorne (26) and AIHW (26) suggested that the inclusion of the flatus item led to overestimates of faecal incontinence prevalence. It was recommended that further work on the Wexner FCGS be undertaken to remove the flatus item and to improve the measurement properties of this scale. This study examines all three measures for the purpose of refining the measurement of both faecal and urinary incontinence. 1.2 Methods The first approach taken was to examine the items and scales using classical test theory (CTT) (Streiner and Norman, 23). The following steps were undertaken and for these analyses unweighted survey data from 24 SAHOS (Harrison Health Research, 24) were utilized: 1. Explore basic item properties such as means, medians and distributions 2. Item analysis examine item endorsement and discrimination 3. Item analysis examine item total correlations and internal consistency reliability Refining Continence Measurement Tools Page 1

10 4. Content Validity relate the items back to their construct(s) using exploratory factor analysis 5. Select items for refined measures and compare with Item Response Theory (IRT) approaches 6. Examine the relationship of the items and the refined scales with other constructs, namely health utilization, disability and self-reported health status The report on these analyses is to be found in Section 5. The second approach to refining incontinence measures was to use modern test theory (MTT), specifically IRT. In this phase the steps undertaken were: 1. Examine instruments from a logical perspective (e.g. differentiate between items measuring incontinence vs. impact of incontinence) 2. Use IRT analyses to provide more in depth analysis of item data and response options to construct more refined incontinence measures 3. Undertake structural equation modelling to examine and test models that best fit the data 4. Compare findings with the CTT taken above to finalise the refined measures The report on these analyses is to be found in Section Results Urinary Incontinence Scales Sections 5 and 6 provide a discussion of the steps undertaken to derive a refined measure for urinary incontinence. As Hawthorne (26) had earlier indicated, a number of items within the UDI- 6 gained endorsement from conditions that may not be directly associated with urinary leakage or incontinence. An analysis of the factor structure of all urinary incontinence items that were included in the survey indicated that a 2 factor structure resulted (explaining 67 of the variance). This would appear to support this contention. The first factor may best be described as a general urinary incontinence factor whereas the second factor contained higher loadings from UDI-6 items such as frequency of urination (UDI- Item 1), pain or discomfort in the lower abdominal or genital region (UDI-Item 6) and difficulty with emptying your bladder (UDI-Item 5). This factor could be considered to be other urological symptoms. The loadings of these items were lower than for all other items on the general urinary incontinence factor (refer to Table 16). Item total correlations for these items, within the UDI-6 scale were also lower than for the other items (urinary leakage due to urge (UDI-Item 2), urinary leakage due to stress (UDI-Item 3), and small amounts of urine leakage (UDI-Item 4)). For these reasons it is thought a better instrument may result from combining the ISI items (frequency and amount of urinary leakage) with the three items from the UDI-6 that load highly on the general urinary incontinence factor (refer to Table 11 and Table 16). The analyses using CTT methods would suggest a urinary incontinence scale (labelled the Revised Urinary Incontinence Scale (RUIS)) comprised of the following items: 1. Urine leakage related to the feeling of urgency? (UDI-6) 2. Urine leakage related to physical activity, coughing or sneezing? (UDI-6) 3. Small amounts of urine leakage (drops)? (UDI-6) Page 2 Refining Continence Measurement Tools

11 4. How often do you experience urine leakage? (ISI) 5. How much urine do you lose each time? (ISI) Modern Test Theory offers three methods of assessing the psychometric properties of items and scales: (a) the ability to examine non-parametric items response scale performance, thus making sure that the response levels within an item are discriminating as expected (item response theory (IRT), threshold analysis), (b) the opportunity to observe the relationship between item response and respondent characteristics to assess whether known groups differ in their interpretation of an item (IRT, differential item functioning (DIF)), and (c) the capacity to assess the impact on a scale of non-normally distributed items through adding or removing an item. The analysis of the urinary incontinence items using modern test theory (MTT) approaches (Item Response Theory) derived a shorter 3 item scale for urinary incontinence (items 1, 3, and 5 above) than the CTT approach. Item 2 from the UDI (concerning stress incontinence) did not fit the best model due to differential item functioning as it was found that males and females interpret this item quite differently. The response categories for Item 1 from the ISI were also found to be problematic as the response thresholds were disordered, meaning that there was not a graded relationship between incontinence status and endorsed response levels. The resulting 3-item measure, labelled the Urinary Continence Assessment (UCA) scale, is included below: 1. Urine leakage related to the feeling of urgency? 2. Small amounts of urine leakage (drops)? 3. How much urine do you lose each time? These suggested scales, with the proposed response categories, can be found in Appendix 1. It is important to note that these revised scales, in conjunction with the original scales, need to be assessed in clinical field trials as part of their validation. As was the case with the Hawthorne (26) study it was found that there were significant correlations between the urinary and faecal incontinence scales (UDI-6; ISI, Wexner) and relevant domains within the measures of health status / health related quality of life that were included in the survey (SF-36, EQ-15D, AQOL; refer Table 18). This indicates that urinary incontinence is associated with a lessening of one s self-rated health status and Hawthorne (26) has also indicated that health status declines with increasing severity of urinary incontinence. The association of the revised scales (RUIS and UCA, RFIS and FCA) with these variables was further explored and they produced very similar correlations in magnitude and direction to those presented in Table Faecal Incontinence Scales Sections 5 and 6 provide a discussion of the steps undertaken to derive a refined measure for faecal incontinence. The Wexner FCGS and a number of additional items concerning soiling, bowel pattern, frequency and urge were also included in the 24 SAHOS (Harrison Health Research, 24) given the concerns with the Wexner FCGS raised by Thomas et al. (26). The factor analysis of the faecal incontinence items indicated that a 3 factor structure resulted explaining 61 of the variance (Table 17). The items that load highly on the first factor are mainly items concerning soiling / wearing a pad, leakage and the effect of leakage on lifestyle. This factor may be considered to be a general faecal incontinence factor, as all items are concerned with leakage and soiling. The items that loaded highly on the second factor were the flatus leakage item from the Wexner FCGS, a question about type of bowel pattern (e.g. Item X1:normal, constipation, diarrhoea, alternating) and a question concerning faecal urge (Item X3: Do you experience an urgent need to have a bowel movement that makes you rush to the toilet?). These Refining Continence Measurement Tools Page 3

12 items appear to be tapping other bowel symptoms that do not appear to be related to faecal incontinence / leakage per se. The only item that loaded on the third factor is frequency of bowel motions (Item X2) and this item has extremely low loadings on the other two factors. It appears to be unrelated to faecal leakage or soiling. An analysis of the item total correlations for the Wexner FCGS indicates that the item concerning flatus has a low corrected item-total correlation and that Cronbach s alpha would be improved if this item were removed from the scale (refer to Table 12). Both the Hawthorne (26) and AIHW (26) reports recommended this item should be removed from the Wexner FCGS as it confounds prevalence estimates and is outside the current ICS definition of faecal incontinence (Norton et al., 22;25). The factor analysis results discussed above would also suggest a refined faecal incontinence scale should not include this item. Although clinicians (Moore et al., 26) had suggested the inclusion of a faecal urgency item this item (Do you experience an urgent need to have a bowel movement that makes you rush to the toilet?) only had a low loading on the general faecal incontinence factor and loaded highly on the second other bowel symptoms factor. Its endorsement pattern was quite different from the other faecal incontinence items (refer to Figure 8) with far fewer respondents endorsing never in relation to this item. This might suggest that bowel urgency is a relatively more common symptom in the community and thus may be endorsed by those not experiencing faecal incontinence. For these reasons the faecal urgency item is not considered for inclusion in the refined scale. However, the item Do you leak stool if you don t get to the toilet in time? loads highly on the general faecal incontinence factor and does contain an urge component. The item concerning type of bowel pattern loaded most highly on the second factor (other bowel symptoms) and has a low loading on the general faecal incontinence factor. Similarly, the question concerning frequency of bowel movements loaded poorly on the general factor and thus these items were not included in the revised scale. An issue concerning item redundancy might be considered by comparing the following items: Do you need to wear a pad to protect your underwear from stool? (Wexner FCGS) Does stool leak so that you have to change your underwear? The pad question from the Wexner FCGS has previously been criticized by Vaizey (1999) as it may relate more to patient fastidiousness rather than faecal incontinence / soiling per se. Given these considerations and its similarity to the soiling item above, in loading on the general faecal incontinence factor (.78 vs..71), it is suggested this item be excluded from the Revised Faecal Incontinence Scale. The proposed scale that results from CTT analysis is the Revised Faecal Incontinence Scale (RFIS) and it contains the following items: 1. Do you leak, have accidents or lose control with solid stool? (Wexner FCGS) 2. Do you leak, have accidents or lose control with liquid stool? (Wexner FCGS) 3. Do you leak stool if you don t get to the toilet in time? 4. Does stool leak so that you have to change your underwear? 5. Does bowel or stool leakage cause you to alter your lifestyle? (Wexner FCGS) However, it is noted that for Item 5 of the Wexner FCGS (and item 5 above) that although it loads highly on the general faecal incontinence factor there may be logical grounds for considering its Page 4 Refining Continence Measurement Tools

13 exclusion as it is measuring a consequence of incontinence rather than a symptom of incontinence per se. It is unconventional to include both the symptom and its consequence in the one scale as this in effect can represent a form of double counting (Thomas et al., 26). There were similarities between the CTT and MTT analyses concerning a number of items that should be excluded from a faecal incontinence scale. These included the items of flatus leakage (Wexner) and items concerning the number of bowel movements and bowel pattern which loaded poorly of the general incontinence factor and were also found to be problematic in the IRT analyses. Analyses using Modern Test Theory approaches (IRT) came to a 4 item solution for faecal incontinence which contains items 1-4 above. IRT is used to find the model with the best fit to the data within the minimum number of items and it is a process commonly used to shorten scales. Item 5 from the RFIS above is not included in this solution as the other items fitted the model better as can be seen in Table 29. The Faecal Continence Assessment (FCA) scale resulting from Modern Test Theory Analyses is provided below. 1. Do you leak, have accidents or lose control with solid stool? (Wexner) 2. Do you leak, have accidents or lose control with liquid stool? (Wexner) 3. Do you leak stool if you don t get to the toilet in time? 4. Does stool leak so that you have to change your underwear? An IRT analysis of a modified Wexner Scale (where the flatus item is excluded) also indicated this scale performed well. Table 33 shows a head-to-head comparison of the FCA with the formal Wexner and the modified Wexner without the flatus question. There was a modest Spearman correlation between the Wexner and the FCA, and a good correlation with the modified Wexner. With respect to discrimination between known groups, the FCA appears to be less sensitive to gender differences and more sensitive to age differences. Regarding classification of cases, the FCA and the modified Wexner produce remarkably consistent results. It is suggested that the response categories for the Wexner FCGS are retained for the revised scales. The revised faecal incontinence scales can be viewed in Appendix 1. It is important to note that the revised scales, in conjunction with the original scales, need to be assessed in clinical field trials as part of their validation. 1.4 Conclusions and Recommendations From the analysis of the urinary and faecal incontinences items and scales included in the 24 SAHOS this study has developed some revised scales for urinary and faecal incontinence (RUIS, RFIS, UCA and FCA) which appeared to improve the screening assessment of incontinence when compared with the original measures. However, one of the limitations of using community survey data is that as the data is collected in face to face interviews the data are at the level of subjective reports of incontinence symptoms rather than confirmed diagnoses. This implies that a community survey will also exclude those currently placed in institutional settings (e.g. nursing homes). These considerations mean that in a community survey there will be a limited range of responses to incontinence items particularly those pertaining to more severe levels of symptoms. It will thus be necessary to trial the refined measures in a range of clinical settings in follow-up field trials. Refining Continence Measurement Tools Page 5

14 It is suggested that the incontinence items included in the 24 SAHOS are administered in a clinical field trial to assess whether the findings from these analyses are replicated amongst clinical samples. This would also enable the datasets to be merged for these items which would permit a more comprehensive analysis and allow for a more definitive conclusion concerning the revised measures. This will also assist in indicating whether the refined measures have superior psychometric properties as would be anticipated from this study. Given some issues raised by the IRT analysis concerning item 1 from the ISI (the frequency of urinary leakage item) and the UDI item 2 (concerning stress incontinence) it is thought that some additional items concerning stress incontinence and the frequency of urinary leakage could be included in the clinical dataset. It is thought that to gain acceptability by clinicians, items covering these domains may need to be included. The three item UCA, however, may be a better instrument to use in future prevalence studies than the ISI. Additional questions which have been developed recently could also be included in the proposed clinical field trials. These include patient-rated global assessments of treatment benefit, satisfaction and willingness to continue treatment (Pleil et al., 25) and Patient Global Impression of improvement and severity for incontinence (Yalcin and Bump, 23). A current study by Hawthorne, Sansoni, Hayes and Marosszeky on patient satisfaction measures may also include recommendations concerning the inclusion of patient satisfaction items. It would then be desirable if field trials could be conducted using the recommended measures across a broader range of field settings specialist continence clinics, general practice and community care settings and in residential care settings. It is also noted that as the 24 SAHOS was a community survey there is limited Australian prevalence data available for continence conditions for those in aged care residential settings and thus it would be particularly useful to pilot test the revised measures in these settings. Page 6 Refining Continence Measurement Tools

15 2 Introduction 2.1 Introduction and Background The Continence Outcome Measurement Suite Project A Continence Outcome Measurement Suite Project (COMS) was commissioned by the Australian Government Department of Health and Ageing, National Continence Management Strategy Research Program with the goal of recommending a suite of continence outcome measures to be used by clinicians and researchers in Australia. This project was finalised in early 26 although recommendations from the earlier draft of this report were critical in the development of a related project Measuring Incontinence in Australia (26) on which this study is based. Measuring Incontinence in Australia (26) assessed a number of the recommended measures from the COMS report by including them in the autumn 24 South Australian Health Omnibus Survey (Harrison Health Research, 24) - a community population survey. The COMS project (Thomas et al., 26) undertook a review of all commonly used instruments to assess faecal and urinary incontinence, and related instruments that assessed the health status, health related quality of life and functional measures of patients experiencing incontinence. With regard to the Urinary Incontinence Symptom Measures category, the King s Health Questionnaire, the Urogenital Distress Inventory (UDI) Short Form, the Urogenital Distress Inventory (UDI) Long Form, the Incontinence Severity Index, the Bristol Female Lower Urinary Tract Symptom assessment, the American Urological Association Symptom Index, the International Continence Society Male assessment, and the International Continence Society Male short form assessment, were assessed as suitable. Given their high ratings on a range of measurement criteria (reliability, validity, applicability and practicability) the first two measures, the King s Health Questionnaire and the UDI (in either form) were the recommended tools in this category and the ISI (a 2-item incontinence severity index) was recommended for use in primary care and public health settings (Thomas et al., 26). With regard to measures of faecal incontinence Thomas et al. (26) found there was little in psychometric terms to separate the various measures of Pescatori et al., (1992), Wexner Faecal Continence Grading System (Jorge and Wexner, 1993), American Medical Systems and St Marks Faecal Incontinence Measures (Vaizey et al., 1999). However, the Pescatori Index stood out from the others in the Vaizey et al. (1999) study in terms of it having significantly lower test-retest reliability. Because of the widespread use of the Wexner Faecal Continence Grading System (Wexner FCGS) and its acceptability amongst clinicians its interim adoption was recommended pending the broadening of the evidence base as to the psychometric properties concerning these tools (Thomas et al., 26). It was noted, however, that the Wexner FCGS did not address the issue of faecal urgency. It was thought the inclusion of the Wexner FCGS along with some further items addressing faecal urgency, bowel pattern and soiling in the Autumn 24 South Australian Health Omnibus Survey (Harrison Health Research, 24) may enable some refinement or revision of the Wexner FCGS (Thomas et al., 26) The Measuring Incontinence in Australia Project As a follow up to the COMS project and as part of the NCMS the Australian Government Department of Health and Ageing funded a special version of the South Australian Health Omnibus Survey (SAHOS) in 24 which included a number of recommended tools from the COMS report. In this report Measuring Incontinence in Australia Hawthorne (26) indicated this study was designed to report on four important incontinence issues: (a) it provided current prevalence estimates of incontinence in the Australian general community, (b) it provided psychometric insights into those incontinence assessment instruments recommended in the Refining Continence Measurement Tools Page 7

16 Thomas et al. (26) report, (c) it reported Australian population norms for the leading five utility instruments and the impact of incontinence on respondents' lives, and (d) it provided Australian population norms and Australian-derived weights for the SF-36 Version 2. The 24 survey involved interviews with sampled households throughout South Australia. The total number of participants interviewed was 315, giving a within scope response rate of 72. The obtained data were weighted by Australian Bureau of Statistics population estimates to achieve representativeness. Full details concerning sampling procedures and the instruments and items included in the survey can be found in Hawthorne (26) and in Harrison Health Research (24). Urinary incontinence was measured by the Incontinence Severity Index (ISI) (Sandvik et al., 1993; Sandvik et al., 1995, Sandvik et al., 2) and the Urogenital Distress Inventory Short Form (UDI-6) (Shumaker et al., 1994; Uebersax et al., 1995). Faecal incontinence was assessed by the Wexner Faecal Continence Grading Scale (Jorge and Wexner, 1993). Given some concerns about the coverage of the Wexner FCGS (Thomas et al., 26) five additional questions were included measuring bowel pattern, the number of weekly bowel movements, bowel movement urge and soiling. 2.2 Description of Continence Measures Included in SAHOS Short Form of the UDI The Urogenital Distress Inventory (UDI) and the Incontinence Impact Questionnaire (IIQ) were developed by Shumaker, Wyman, Uebersax, McLeish & Fantl (1994) to assess the impact of urinary incontinence symptoms upon quality of life for women. The original forms of the IIQ and the UDI had 3 and 19 items respectively but work by Uebersax, Wyman, Shumaker, McLeish and Fantl (1995) created a 7 item version of the IIQ and a 6 item version of the UDI. The UDI assesses symptoms of incontinence whereas the IIQ focuses on the impact of these symptoms on everyday life. For the latter reason the UDI-6 was included in the community survey. The UDI-6 items ask the respondent Do you experience and, if so, how much are you bothered by the following list of symptoms: Frequent urination Urine leakage related to the feeling of urgency Urine leakage related to physical activity, coughing or sneezing Small amounts of urine leakage (drops) Difficulty emptying your bladder Pain or discomfort in the lower abdominal or genital area The response scale is: Not at all Slightly Moderately Greatly Most of the measurement properties of the UDI-6 have been assessed with women. Test-retest reliability (Long Form) was assessed by (Hagen et al., 22); validity has been demonstrated by items predictive of urodynamic findings in women (Lemack and Zimmern, 1999); correlation with patient self-reported symptom complaints (Harvey et al., 21); and sensitivity to change (Harvey et al., 21). Clinical data is provided by Lukacz et al. (24) and Harvey et al. (21), with normative data for both men and women provided by Svatek et al. (25). Recently, Rodriguez and Raz (23) have added an item on nocturia and an additional urgency item to create the UDI-8. Lubeck et al. (1999) and Brown et al. (1999) created a 9 item Urge-UDI Page 8 Refining Continence Measurement Tools

17 from the Long Form UDI, which includes 3 items from the UDI-6. However, the UDI-6 is more commonly used in the field. Although the short form of the UDI includes items on type of incontinence (urge/stress) the short form only addresses the frequency of a small amount of urine loss. It does not include the item from the UDI (long form) concerning the frequency of a large amount of urine loss. For this reason it may not be as good a measure of severity (frequency x amount) as other instruments. Given this it was thought useful to also include the ISI in the community survey as another measure of symptom severity The Incontinence Severity Index (ISI) This index was developed by Sandvik et al. (1993) and has been used in Norwegian epidemiological surveys of health problems (Sandvik et al., 1993; Hannerstad et al., 2). Testretest reliability (kappa) at 3-days for the Index was.78 and test-retest reliability for each question was.69 and.83 which was also satisfactory (Hanley et al., 21). Although developed for epidemiological surveys slightly modified versions of ISI have been used in the evaluation of treatment programs in general practice (Seim et al., 1996; Holtedahl et al., 1998; Melville et al., 23). It correlates well with physician assessment (Melville et al., 23) and other incontinence instruments (Melville et al., 23; 25b; Klovning et al., 24; Murphy et al., 24). In the original 1993 study, concurrent validity with a 48-hour pad weighing test was found to be r =.59. In a later study, Sandvik, Seim, Vanvik and Hunskaar (2) reported a similar concurrent validity of.54 again with a 48-hour pad weighing test. The moderate correlation of the Index with objective measurement parameters is a strong feature of this tool, as many other instruments show lower validation correlations (Thomas et al., 26). The Incontinence Severity Index was originally intended for use with women, and has been validated with women, but there are no reasons why it cannot be used with men as can be seen below the items do not appear to be gender specific. Its inclusion in the SAHOS 24 survey provided some opportunity to obtain data for males for this Index. It provides a useful short form diagnostic severity measure for urinary incontinence, although clearly the ISI does not measure either urgency or the type of urinary incontinence. As Hawthorne (26) indicated the Incontinence Severity Index (ISI) originally consisted of two items, one with 4 response levels and the other with two response levels (Sandvik et al., 1993). In 2 the instrument developers altered the second item s response scales from 2 to 3 levels, known as the four-level severity index (Sandvik et al., 2). The four-level severity index is reported here whereas the earlier version of this instrument is reviewed by Thomas et al. (26) in the COMS report. The instrument has the following structure: How often do you experience urine leakage? Never = Less than once a month = 1 A few times a month = 2 A few times a week = 3 Every day and/or night = 4 How much urine do you lose each time? None = Drops = 1 Small splashes = 2 More = 3 Severity index = (points for frequency) x (points for amount) Refining Continence Measurement Tools Page 9

18 Scoring is through multiplication of endorsed response levels giving a score range from to 12. Higher scores denote more severe urinary incontinence (Hanley et al., 21). Sandvik et al. (2) recommended that when using the four-level severity index the interpretations were scores 1-2 a slight problem, 3-6 moderate, 7-9 severe and 1-12 very severe. The standard scoring system did not discriminate between those with no incontinence symptoms and those with slight symptoms. The ISI was therefore modified through inclusion of a never category; thus the categories became Never / Less than once a month / 1-several times a month / 1-several times a week / Every day and / night which has added an extra category,, describing those with no symptoms. This procedure is that recommended by Sandvik et al. (2). In the Hawthorne (26) study where classification was needed, ISI scores were recoded into (score range) None () / Slight (1 to 2) / Moderate (3 to 6) / Severe (7 to 9) / Very Severe (1 to 12) levels Wexner Faecal Continence Grading Scale (Wexner FCGS)1 The Jorge and Wexner faecal continence grading scale was developed to provide clinicians with a means of assessing faecal incontinence severity (Jorge and Wexner, 1993). The Wexner scale requires assessment on leakage / accidental faeces for solid, liquid, and gas, the need to wear a pad and alterations to lifestyle. Vaizey et al. (1999) report the test-retest reliability of the Wexner score as r =.75 which falls within an acceptable range (Streiner and Norman, 23). Table 1 The Wexner Faecal Incontinence Grading System Type of incontinence Frequency Never Rarely < 1/month Sometimes < 1/week 1/month Usually < 1/day 1/week Solid Liquid Gas Requires pad Lifestyle Always 1/day Q1. In the past four weeks: do you leak, have accidents or lose control with solid stool? Q2. In the past four weeks: do you leak, have accidents or lose control with liquid stool? Q3. In the past four weeks: do you leak, have accidents or lose control with gas (flatus or wind)? Q4. In the past four weeks: do you need to wear a pad to protect your underwear from stool? Q5. In the past four weeks: does bowel or stool leakage cause you to alter your lifestyle? Each item is assessed on a Guttman scale (Never / Rarely / Sometimes / Usually / Always). Scores are determined by a simple summation of endorsements. The range is from to 2 and the higher the score the worse the faecal incontinence. Scores can also be classified into categorical levels (faecal incontinence Never / Rarely (1 episode in past month) / Sometimes (2-4 episodes) / Weekly (>1 week - <1 day episodes) / Daily (1 or more daily episodes)). The construct validity of the Wexner FCGS is supported by significant correlations with a disease specific measure of psychosocial functioning (Deutekom et al., 25b); and correlation with items from the Euro-Qol 5D - usual activities, anxiety / depression, and mobility (Deutekom et al., 25a); patient s subjective feelings of relief after physiotherapy treatment (Deutekom et al., 25b); and health related quality of life the Gastrointestinal Quality of Life Index (GIQLI) and MOS SF-2 - (Rothbarth et al., 21). Recently, Deutekom et al. (25b) found that change scores on the Wexner correlate.94 with change scores on the Vaizey Scale (Deutekom et al., 25b). The Wexner also correlates well with clinical assessment (.78) and the clinical 1 Wexner has started calling this the Cleveland Clinic Florida Fecal Incontinence Score. Since this measure is not widely known by this name it is not used here. Page 1 Refining Continence Measurement Tools

19 assessment of improvement (.87) (Vaizey et al., 1999); as well as clinical variables using anal manometry (Deutekom et al., 25b; Nazir et al., 22). It is also sensitive to change (Oliveria et al., 1996; Devesa et al., 22; Deutekom et al., 25b; Vaizey et al., 1999). The discriminative validity of the Wexner score is supported by its ability to discriminate between patients who rated their health much better after physiotherapy treatment than those who rated their situation worse or equal or better (Deutekom et al., 25b). Oliveria et al. (1996) found a good correlation between Wexner scores for those patients with successful sphincter repair surgery based on patient, clinical and physiological evaluation after surgery. Significant improvement was found for those patients with a successful repair, while no improvement was found for those patients whose repair had failed. Clinical Data is provided by papers from Oliveria et al. (1996); Rothbarth et al. (21), Kairaluoma et al. (24), Vaizey et al. (1999) and Nazir et al. (22). Normative Data for the Wexner FCGS in women is provided by Melville et al. (25a), with faecal incontinence defined as loss of liquid or solid stool occurring at least monthly. However, Thomas et al. (26) and Hawthorne (26) have both commented that an obvious difficulty with the Wexner FCGS is that it is unconventional to sum symptoms and symptom effects; a procedure that gives rise to double counting. In the Wexner FCGS, stool leakage and its consequence on lifestyle are both counted. Vaizey et al. (1999) has also criticized the Wexner FCGS for the pad wearing question, arguing this was a measure of patient fastidiousness or urinary comorbidity. There are also no items included in the scale that address the issue of urge incontinence (Deutokom et al., 25a; Hawthorne, 26; Thomas et al., 26; Vaizey et al., 1999). Another issue with the Wexner FCGS is the inclusion of an item on flatus which is equally weighted with other items in the scale. Thus a person who endorses daily flatus and no other item will receive the same score as another who endorses only daily incontinence for liquid stool which might be considered a more severe condition. Thus its measurement of severity may be confounded by flatus. This problem associated with the Wexner FCGS, and indeed a number of other faecal incontinence scales, has been commented on by both Hawthorne (26) and AIHW (26). It should be noted that the ICS definition of faecal incontinence specifically excludes flatus (Norton et al., 22; 25). The impact of the inclusion of flatus items in faecal incontinence scales is that estimates of prevalence will be inflated (AIHW, 26). Indeed the AIHW report notes that of the 199 persons in the SAHOS 24 that report problems with any faecal incontinence symptoms that the majority of the 82 / 199 are only reporting flatus. Similar findings for clinical samples were noted by Boreham et al. (25). Following a recommendation from Thomas et al. (26) Hawthorne (26) included both the Wexner FCGS and additional items on bowel patterns, faecal urgency and soiling in the autumn 24 SAHOS as it was thought that following psychometric analysis of these items it may be possible to further refine the Wexner FCGS to address these issues. Refining Continence Measurement Tools Page 11

20 3 Incontinence Measures: Findings from Recent Reports 3.1 Measures of Urinary Incontinence Hawthorne (26) in Measuring Incontinence in Australia reported that in his analysis of the 24 SAHOS (Harrison Health Research, 24) the reliability of the UDI-6 was Cronbach α =.78. Two items, the last two questions How much are you bothered by pain or discomfort in the lower abdominal or genital area? and Do you have difficulty emptying your bladder? did not fit well with the other items (the item-total correlations were r =.31 and r =.37 respectively). Deletion of these items would have improved the Cronbach α to.81 and the explained variance from 49 to 66 (Hawthorne, 26). The reliability of the ISI was Cronbach α =.89 (Hawthorne, 26). The relationship between the two items of the ISI was examined using kappa (κ) and found to be.74 indicating good agreement between the two items (Landis and Koch, 1977). Hawthorne (26) report there was perfect agreement between the two items for 9 of all respondents. Thus it appears the ISI gained its high level reliability through replication; the r s =.96 and the proportion of explained variance was 92 (Hawthorne, 26). Although this analysis suggested the ISI has excellent measurement properties that it consists of just two items suggests that it violates classical test theory which postulates that at least 3 items are needed for stable measurement interpretation (Hawthorne, 26; Tabachnick and Fidell, 21). Regarding the relationship between the two estimates of urinary incontinence (the ISI and UDI-6) this was examined after conversion to McCall s T-scores to compensate for the different scale ranges used by the two measures (Hawthorne, 26). The correlation was r S =.75 (n = 35, p <.1). This suggested that the ISI and UDI-6 are measuring similar and related aspects of incontinence. However, Hawthorne (26) indicated that the UDI-6 was more sensitive for those with minor or moderate symptoms and that the ISI classified more cases at the floor and ceiling of the instrument. The ISI classified 76 of cases as continent compared with 54 of cases for the UDI-6. The prevalence estimate for urinary incontinence derived from the ISI was 24 as contrasted with the estimate from the UDI-6 of 47 which indicated these measures were not fully compatible (Hawthorne, 26). When the cases were dichotomized into no symptoms / any symptoms there was agreement for only 77 of cases. Of the 23 of cases where there was disagreement nearly all of these were where no symptoms were reported on ISI but were classified as symptomatic by the UDI (Hawthorne, 26). The UDI measures a broader range of urological symptoms and their impact / bother whereas ISI measures frequency and volume of urinary leakage. It has earlier been noted that some items in the UDI (frequency of urination and discomfort in lower abdominal or genital area) may be gaining endorsement from those with non-urinary incontinence conditions. When Hawthorne (26) excluded these UDI items from the classification analysis the agreement between the two measures rose to 87. Excluding these items from the UDI also reduced the prevalence estimate for UDI to 36. Given consideration of the above factors the prevalence estimates derived from the ISI are generally preferred (Hawthorne, 26; AIHW, 26). 3.2 Measures of Faecal Incontinence Given concerns that the Wexner FCGS does not include an item on faecal urgency and may require revision, a number of other items were also included in the SAHOS 24 survey. These included items on faecal urgency, bowel patterns and soiling. As indicated above the Wexner FCGS includes an item on flatus that is equally weighted with other faecal incontinence items in this scale. However, the current ICS definition specifically excludes flatus (Norton et al., 22; 25). In the 24 SAHOS it was found that of the 199 Page 12 Refining Continence Measurement Tools