Authors: Derick T. Wade Affiliations: Clinical Trials From the Department of Neurological Disability, Oxford Centre for Enablement, Oxford, UK. FIM is a trademark of the Uniform Data System for Medical Rehabilitation, a division of UB Foundation Activities, Inc. Correspondence: All correspondence should be addressed to Derick T. Wade, Department of Neurological Disability, Oxford Centre for Enablement, Windmill Road, Oxford OX3 7LD, UK. 0894-9115/03/8210(Suppl)-0S26/0 American Journal of Physical Medicine & Rehabilitation Copyright 2003 by Lippincott Williams & Wilkins DOI: 10.1097/01.PHM.0000086996.89383.A1 Analysis Outcome Measures for Clinical Rehabilitation Trials Impairment, Function, Quality of Life, or Value? ABSTRACT Wade DT: Outcome measures for clinical rehabilitation trials: Impairment, function, quality of life, or value? Am J Phys Med Rehabil 2003;82(Suppl):S26 S31. Choosing outcome measures in rehabilitation research depends on the standard research skills of clear thinking, attention to detail, and minimizing the amount of data collected. In rehabilitation, outcome is more difficult to measure because (1) usually several outcomes are relevant, (2) relevant outcomes are affected by multiple factors in addition to treatment, and (3) even good measures rarely reflect the specific interest of any individual patient or member of the rehabilitation team, leading to some dissent. Measurement of general quality of life is not possible because there is little agreement as to the nature of the construct; moreover, measurement of relevant aspects of quality of life would probably give similar results. Cost in terms of resources can be estimated, but there is no validated or even widely accepted method of relating this to benefit in a fair, open, and rational way. Outcome is best measured at the level of behavior (activities), with other measures being used to aid interpretation. Key Words: Rehabilitation, Rehabilitation Research, Outcome Measures Research should be undertaken to answer a question. The question asked will determine two aspects of the research: the most appropriate design and the most appropriate measure or measures to use. The logic of the design determines the logical conclusions that can be validly drawn from the results. The measures used, along with sample size, determine the precision of the esti- S26 Am. J. Phys. Med. Rehabil. Vol. 82, No. 10(Suppl)
mates made. This article discusses measures that might be used in rehabilitation research. The article assumes that all readers are familiar with the basic framework put forward within the World Health Organization (WHO), both in its original (1980) International Classification of Impairments, Disabilities, and Handicaps version 1 and the revised International Classification of Functioning 2 (ICF; see also Wade and Halligan 3 ). The WHO ICF terminology will be used in this article. This article is primarily concerned with evaluative research investigating the effectiveness of specific interventions in well designed studies, usually randomized controlled studies or singlecase study designs. The principles apply in other situations. Outcome: What Is It?. Although the term outcome is simple and its meaning may seem obvious, it is worth some consideration in the context of rehabilitation research. Outcome refers to the expected or looked-for change in some measure or state. In other words, a patient will enter a study or rehabilitation program in one state and may change as a result of the intervention. The new state constitutes the outcome. In this sense, clinical trials are primarily concerned with changes in those aspects of quality of life (QOL) or activities that are due to or caused by the treatment more than QOL or activities as a whole. In rehabilitation, there are often many interventions given over time, sometimes months, and so the relevant outcome is not always obvious. For example, for a patient who has had a stroke and has moved into a rehabilitation unit in a hospital, the outcomes shown in Table 1 might constitute relevant outcomes, depending on who wants to know and why. Other outcomes that might be relevant include stress on family, ability to work or drive, and ability to communicate using the telephone. It is important not to refer to the outcome but to be specific and refer to the outcome of a specified intervention measured in a specific way, reflecting the interests of specific groups/stakeholders, which might include researchers or funding agencies, although patients should remain the primary focus. It is also better to use the phrase measure of outcome rather that outcome measure. This emphasizes that one is focusing on outcome. Of equal importance, it avoids the sloppy thinking associated with such questions as what outcome measures should I use at baseline? The term outcome measure has become synonymous with measure in many situations. What Is Different in Rehabilitation Research?. Clinical trials in rehabilitation are no different in principle from any other clinical trials, but they differ in practice in several ways. First the outcomes of interest are different, and indeed, there are often multiple outcomes of interest. Rehabilitation usually explicitly aims to alter activities (i.e., behavior) or participation (i.e., role functioning in the community), whereas in other medical domains the primary aim may legitimately be to alter body function or structure (also known as pathology, disease, and impairment in the earlier WHO terminology). Thus, rehabilitation research uses primary measures that come from different domains and, more importantly, will usually study more than one major outcome. Next, rehabilitation studies usually involve multifocal and sometime ill-defined interventions rather than the single well defined interventions typical of drug research trials. Core rehabilitative interventions those in physical therapy, occupational therapy, speech-language pathology, and rehabilitation psychology, for instance involve close interaction between the therapist and the patient. These therapies work through the action or behavior, and the thoughts, of the patient, rather than being the rel- TABLE 1. Outcome varies depending on the intervention being evaluated Intervention Outcome Comment One 30-min session with an occupational therapist Being able to put on socks without assistance This relates to one single-patient-therapist interaction Course of therapy in the physiotherapy department Being able to walk about ward safely This relates to an interaction between the patient and one department, Inpatient treatment in stroke unit Full inpatient and outpatient program with follow-on community rehabilitation Scoring 18/20 or more on the Barthel activities of daily living index at discharge Able to live in own first-floor apartment alone, without depending on family, doing shopping, and cooking meals accumulated over time This relates to overall effect of the program but will not reflect much of any specific speech therapy input This depends also on the network of rehabilitation services in the community and the precise location of the apartment October 2003 Outcome Measures for Clinical Trials S27
atively mechanical and predictable effects of drugs. This complexity may lead to some internal debates within the rehabilitation or research team about an appropriate outcome. For example, most stroke-unit trials used a simple measure of activities of daily living (such as the Barthel), which will not reflect much of the input from a speech therapist. Moreover, rehabilitation studies will usually require time both to undertake the intervention and for the intervention s effect to become apparent, whereas medical interventions are often brief and have rapid effects. This not only increases the cost of rehabilitation studies but also makes interpretation of outcome more difficult because many factors may have an influence on the observed outcome. Finally, rehabilitation is often context specific, whereas medical interventions are usually independent of context. 4 The term context is used within the WHO ICF to refer to three aspects of a patient s situation: the physical environment, the social environment, and the patient s own personality, including beliefs and attitudes and experiences and expectations. For example the rehabilitation of a patient left with reduced mobility after stroke will depend on, inter alia, the structure of their home (e.g., stairs, internal steps, ease of getting out), their wishes (e.g., they might be quite content to use a wheelchair), the other people in the house (who may or may not be willing and able to assist with toilet transfers), and the local rules concerning support from the State or other organizations. In other words, social, background, personal motivations, and physical factors in the environment will affect rehabilitation outcome and these contextual factors make it less easy to generalize the results of rehabilitation trials. Nonetheless, any research design aims to minimize bias and to reduce error variance. Randomization is used in place of elective choice to control for selection biases and to ensure that experimental and control groups are comparable (within calculable, probabilistic limits). The effects of expectation and reporting bias are minimized by blinding as many participants and staff as possible at any points at which they may effect the trial. The third objective of a research trial is to maximize the chance of detecting change or the difference between treatment groups, and this requires choosing an appropriate outcome measure. What Domains?. One can argue that rehabilitation trials could, and perhaps should, collect data on all four domains mentioned in the title and, indeed, potentially in other domains also. This article will now consider each of the domains from the WHO ICF, and wider constructs of QOL, and explain why data might be needed from each domain. Pathology. This information is needed to set the other data items in context and to help in interpretation and generalization. Pathology (disease, diagnosis) will determine the expected natural history, will inform the choice of prognostic variables needed to interpret findings and ensure control of severity, and will allow others to draw appropriate conclusions for their practice. It will not be a measure of outcome. Body Structure and Function. Data from this level may have several uses. Impairments (also called impairment, often assessed as symptoms and signs) are usually the best markers of prognosis (case mix) and help in interpreting other data. In addition, change at the level of impairment might be used to show that the intervention had its primary effect (an indication of process), assuming that it is expected to alter impairment. Lastly, there will be occasions when reducing impairment is itself the primary goal. For example, studies investigating control of pain or reduction of spasms might correctly have pain intensity or spasm frequency as primary measures of outcome. Activities. In most rehabilitation trials, the main aim is to reduce limitation on activities (disability, behavior), and the primary outcome is therefore usually at the level of activities. The activities may be within or outside the home and could include the activities needed to take part in work. (Being an employee and having a job are outcomes at the level of participation, but being able to weld or use a word processor are activities needed to work.) The main difficulty is to choose a measure that will detect the change expected. Many measures may either include too many items that are not of relevance or too few to cover the range of possible outcomes. Participation. Most clinical rehabilitation aims to maximize social participation. Unfortunately, measures of participation are currently less developed than measures of more basic activities. Moreover, many factors well outside the control of the rehabilitation team may affect participation. Consequently, it is rarely the focus of rehabilitation research trials, and it is rarely measured. Personal Context. Some interventions, for example, those focused on self-efficacy, are intended to alter personal factors (beliefs, expectations, background, or preinjury factors). Consequently, one might measure some aspect of this both as a measure of initial state and case mix and also to show that the intervention has had some direct effect (i.e., as a measure of process). Psychological studies might focus on self-efficacy as an outcome, but from the viewpoint of the rehabilitation process as a whole, one would want to directly assess effects S28 Wade Am. J. Phys. Med. Rehabil. Vol. 82, No. 10(Suppl)
on activities. The known link between self-efficacy and activity performance is not sufficiently strong to allow its use as a surrogate marker. Similarly, patients knowledge of specialized health self-care practices may be an important process measure, but actual performance of these activities is even more important, and (for example) reducing the rate of complications might be the primary study endpoint. Physical Context. Some studies will investigate the effects, benefits, risks, and costs of providing suitable items of equipment such as wheelchairs, ankle-foot orthoses, adapted clothing, and so on. Measures of their provision and use might be needed, and the effects of equipment provision on community participation may also need to be measured. Equally physical context data (e.g., stairs vs. ramps) might be needed to interpret data on activities. Major environmental factors are noted in the ICF. However, there may be studies in which measures of equipment might constitute an important outcome measure. For example, a trial that investigated different ways of assessing and providing equipment might include the speed of provision of equipment and the provision of appropriate equipment as relevant proximal outcome measures. Conceivably, a study might investigate whether an intervention improves patients independence to the level that the amount of personal support needed or used is reduced. Social Context. Potentially, studies might investigate the effects of altering the attitudes, knowledge, skills, or expectations of family members (for example), and then it would be appropriate to measure this change as an indication of process. However this has only been studied occasionally. 5 Value and QOL. Value may be defined as the relationship between the resources used and the (change in) outcome achieved. Ultimately, data on value are needed to justify the use of resources and to set priorities. However, it is subject to much debate and is not an easy measurement to make. Before value is considered, the intervention has to be shown to have an effect. (Of course, the specific trial may not have had the statistical power to detect a relevant change or difference, or it may have studied the wrong outcome or at least not have studied all important outcomes that might have been affected.) Once an effect has been shown, then the difficulties begin. In essence, there are two questions: what resources are to be included in calculating the cost, and how is the benefit to be valued? It is also important to consider whether the study or trial had adequate statistical power to measure the costs; it may well be that studies that can detect differences at the level of activities are inadequately powered to calculate differences at the level of costs. Resources that might be used in any calculation of cost include: Direct financial costs of the intervention (drugs, equipment, travel) Costs of therapists time Overhead costs of the organization providing the intervention Time, effort, and other resources given by family members and the patient Side-effects of the intervention Aspects of the benefit that can be considered include: Reduction in actual care costs over a lifetime, including both medical care and routine attendant and nursing costs Reduced other medical costs Personal evaluation of the increased independence or benefit achieved Family evaluation of the benefit For some patient groups, increased productive activity and paid work Johnston and others have written on approaches to the study of value, cost-benefits, and cost-effectiveness analysis in rehabilitation. 6 (For a more thorough set of measurement terms and definitions, see Johnston et al., Measurement Standards for Interdisciplinary Rehabilitation.) The primary reason for calculating a value for the benefit is to allow comparison between different interventions so that apparently rational choices can be made. If one restricts benefits to direct financial benefits, then sometimes the benefits of intervention are so great that there is no argument; the intervention is worthwhile. More often, it is necessary to attribute some (financial) value to the benefit. This can only be done successfully if all health studies use the same outcome measure, and QOL seems to offer one solution. But does it? The term quality of life is deceptively simple, but it is telling that the WHO has not included QOL within the new ICF; this was probably because there was insufficient agreement on the construct, which is largely subjective. One way of conceptualizing QOL is to consider that it is the person s (patient s) own evaluation of all of his or her life, including the patient s perception of his or her pathology, impairments, activities, and participation interpreted in the light of the patient s own context. 7,8 If this is accepted, then the only way to establish a QOL (which would apply equally to people who were outside the healthcare system) is to ask the person directly. This is frequently done. In considering more formal measures that supposedly measure QOL, there are two major findings of importance. First, there are over 1200 existing measures that reputedly measure QOL, and they vary greatly in their content. 9 This illustrates dramatically the lack of any agreed phenomenon that is QOL. Moreover, there is now a trend toward developing disease-specific QOL measures that further devalues the concept of QOL in general; if QOL means any- October 2003 Outcome Measures for Clinical Trials S29
thing, it must be independent of any specific disease. Second, there is reasonable evidence that people in a stable clinical condition generally re-normalize their self-assessed QOL. 10 Consequently, there may be little point in using QOL to compare different health interventions because most patients will achieve an approximately similar outcome in terms of QOL, whether an intervention is effective and despite any differential in effectiveness. At the same time, QOL has been little used as an outcome measure, so it is conceivable that certain rehabilitation milieu or approaches substantially increase it. One development within the field of QOL that specifically also considers value is to weight the quality in economic terms, identifying in some way the strength of (economic) preferences that individuals have for specified health states. Such measures have begun to be developed, 11 but they have not been used in rehabilitation trials and have rarely been used in medical care trials, which are concerned with the impact of interventions on the patient s experience of health or illness (a major component of QOL). Their validity and applicability will need further development and experience. Choosing a Measure. When considering outcome measures in rehabilitation, including research but also more generally, it is wise to consider a series of questions. First, why am I measuring outcome? Who wants me to measure outcome? In research, this is probably obvious. You are likely to want to know for yourself whether an intervention works. However, even in this setting it is worth thinking of other relevant interested parties such as the patients, the funding agency, the journal editor and peer reviewers, and so on. In addition, the person actually doing the work needs to know the answer because he or she will only be motivated to acquire the data if the data are considered important. Next, you should specify as closely as possible what effects you expect. This may seem obvious, but it is surprising how many projects either fail to do this or ignore the results of this step. In rehabilitation research, it is not uncommon to expect several effects. Each may need a measure. You then need to discover what measure or measures can capture the specific effect or effects expected. The major considerations when choosing a measure are shown in Table 2, and a few points will be emphasized here. It is vital to choose a measure that as closely as possible matches the range of the expected outcome. If you anticipate changing the ability to dress, do not use the FIM as a whole because only a small part of the FIM covers dressing. Conversely if you are trying to improve mobility after stroke, covering the whole range, simply using gait speed is unwise because a proportion of people will not walk and yet you need to measure TABLE 2. Questions to ask when choosing a measure Clinical Question Will the data collected using this measure answer the clinical question? Does the measure cover the whole range of possible activities or impairments without including items that are not relevant? Do I know how much variability there will be in the data, and is that satisfactory? Is the measure going to deliver data that are able to discriminate or detect the change or difference that I am interested in or expect to find? Will the measure be correctly used by staff and accepted by the patients so that I get a full data set under the circumstances? Is it sufficiently short and simple? Will anyone reading the results of my study understand the data given by this measure? their outcome. One important practical consideration is how the data will actually be collected. Who will undertake the measures, how, where, etc.? This may strongly influence your choice of measure because a complete set of even relatively crude data are much more valuable than an incomplete set of much more detailed data. You must also consider what additional data you and others will sensibly require to interpret the outcome. This must be restricted to major items such as known markers of prognosis. Avoid collecting excessive data. Conclusion. Generally, primary outcome measures in rehabilitation research will be at the level of activities. Impairment-based measures are useful both to measure prognosis (case mix) and to track the (process) effects of interventions. In large or confirmatory trials, some information on costs should be collected, but attempts to measure value through using indicators of QOL as a whole Psychometric Term or Question Is it valid for the purpose? Content validity and avoidance of floor or ceiling effects. What is its reliability when used as I will use it? What is its sensitivity, and is that adequate? Is the measure feasible? How communicable is the result given by this measure? S30 Wade Am. J. Phys. Med. Rehabil. Vol. 82, No. 10(Suppl)
seem unlikely to succeed, either in logic or in practice. By using simple measures that assess relatively large or definite changes in activities that are quite evident to patients, family, and therapists, one takes a great step toward ensuring that the outcomes are of value and toward ensuring that the trial will be feasible. ACKNOWLEDGMENTS I thank Dr. Mark Johnston for his helpful comments on an earlier draft, the Kessler Medical Rehabilitation Research and Educational Corporation for their support, and the very many people over the years who, by asking me for help or by disagreeing with me, have helped me to develop and clarify my thoughts (assuming that they are clear!). REFERENCES 1. International Classification of Impairments, Disabilities, and Handicaps. Geneva, World Health Organization, 1980 2. International Classification of Functioning, Disability, and Health. Geneva, Switzerland, World Health Organization, 2001. Available at: http://www3.who.int/ icf/icftemplate.cfm. Accessed April 3, 2003 3. Wade DT, Halligan PW: New wine in old bottles: The WHO ICF as an explanatory model of human behaviour. Clin Rehabil 2003;17:349 54 4. Haley SM, Costner WJ, Binda-Sundberg K: Measuring physical disablement: The contextual challenge. Phys Ther 1994;74:443 51 5. Buchbinder R, Jolley D, Wyatt M: Population based intervention to change back pain beliefs and disability: Three part evaluation. BMJ 2001;322:1516 20 6. Johnston MV: Cost-benefit methodologies in rehabilitation, in Fuhrer M (ed): Rehabilitation Outcomes: Analysis and Measurement. Baltimore, Brookes, 1986, pp 99 114 7. Carr AJ, Gibson B, Robinson PG: Measuring quality of life: Is quality of life determined by expectations or experience?. BMJ 2001;322:1240 3 8. Carr AJ, Higginson IJ: Are quality of life measures patient centred? BMJ 2001; 322:1357 60 9. Garratt A, Schmidt L, Mackintosh A, et al: Quality of life measurement: Bibliographic study of patient assessed outcome measures. BMJ 2002;324:1417 22 10. Albrecht GL, Devlieger PJ: The disability paradox: High quality of life against all odds. Soc Sci Med 1999;48: 977 88 11. Brazier J, Deverill M: A checklist for judging preference-based measures of health related quality of life: Learning from psychometrics. Health Econ 1999;8: 41 51 October 2003 Outcome Measures for Clinical Trials S31