The MedRisk Instrument for Measuring Patient Satisfaction With Physical Therapy Care: A Psychometric Analysis Paul Beattie PT, PhD, OCS 1 Christine Turner, PT 2 Marsha Dowda, DrPH 3 Lori Michener, PT, PhD, ATC, SCS 4 Roger Nelson, PT, PhD, FAPTA 5 Journal of Orthopaedic & Sports Physical Therapy Study Design: Psychometric evaluation of a cross-sectional survey. Objectives: To determine the validity of measures obtained from the MedRisk Instrument for Measuring Patient Satisfaction With Physical Therapy Care (MRPS) to differentiate between patient satisfaction with internal and external factors. Background: Self-report measures that sample a variety of items provide clinicians with an array of information that may assist in assessing patient satisfaction. An important measurement characteristic of these instruments is the ability to discriminate between different factors that may influence patient reports of satisfaction with care, ie, discriminant validity. In previous work, exploratory factor analysis suggested that the MRPS questionnaire has a 2-factor structure: internal, relating to the patient-therapist interaction, and external, describing nontherapist issues such as admissions and clinic environment. Methods and Measures: One thousand four hundred forty-nine adult patients completed the MRPS questionnaire upon finishing their course of outpatient physical therapy treatment. Discriminant validity of the 2-factor model was assessed using confirmatory factor analysis. The measures from the 2 factors were then evaluated for reliability by calculating the standard error of measurement (SEM), and for concurrent validity by correlating the mean score of the factors and individual items to global measures of satisfaction. Results: Confirmatory factor analysis supported a good to excellent model fit for the internal factor (7 items) and external factor (3 items). The SEM for the 2 factors was 0.19 and 0.24, indicating a low degree of measurement error. Both factors had high significant correlation with global measures of satisfaction (internal, r = 0.83 and 0.80; external, r = 0.71 and 0.71). All individual items within the 2 factors had significant correlations with global measures ranging from r = 0.33 to 0.80. Conclusions: Our findings provide evidence of discriminant and concurrent validity of the 2-factor solution for the MRPS questionnaire for the sample that was tested. This 2-factor solution yields 1 Clinical Associate Professor, Program in Physical Therapy, Department of Exercise Science, School of Public Health, University of South Carolina, Columbia, SC. 2 Clinical Specialist, MedRisk, Inc, King of Prussia, PA. 3 Biostatistician, Department of Exercise Science, School of Public Health, University of South Carolina, Columbia, SC. 4 Assistant Professor, Department of Physical Therapy, Virginia Commonwealth University, Richmond, VA. 5 Professor and Chair, Department of Physical Therapy, Lebanon Valley College, Annville, PA. This study was funded by MedRisk, Inc, and approved by the MedRisk, Inc, Institutional Review Board. Financial disclosure and conflict of interest: Authors Drs Beattie and Michener are consultants for Expert Clinical Benchmarks, MedRisk, Inc; Christine Turner is an employee of MedRisk, Inc. Dr Nelson is Vice President of Expert Clinical Benchmarks LLC, MedRisk, Inc. Address all correspondence to Dr Paul Beattie, PhD, PT, OCS, Program in Physical Therapy, Department of Exercise Science, School of Public Health, University of South Carolina, Columbia, SC 29208. E-mail: pbeattie@gwm.sc.edu measures that are relatively free of error and may discriminate between internal and external factors influencing patient satisfaction. Patients who complete their course of physical therapy report that the professional interaction between the therapist and patient, especially the meaningful exchange of relevant information, is critical for patient satisfaction with care. The generalizability of our data to patients who do not complete their physical therapy care or who are receiving care in other health care environments is unknown. J Orthop Sports Phys Ther 2005;35:24-32. Key Words: instrument validation, questionnaire, self-report, survey Patient satisfaction with care is an important variable for assessing physical therapy practice. 1,4,28,32,36,41,44 Measures of patient satisfaction have been used as indicators of quality of care as a means of identifying patients who have a higher or lower likelihood of compliance with treatment programs, and as a benchmark upon which to assess market competitiveness. 1,2,6,9,10, 13,15,19-21,25,28,31,33,37,38,42,46 Numerous methods have been proposed for the measurement of patient satisfaction with care, ranging from the use of a single global question such as, Overall, to what 24 Journal of Orthopaedic & Sports Physical Therapy
degree are you satisfied with your care? to lengthy questionnaires. 2,4,12,13,15,16,21,30,32,33,36,41,45-47 While single global measures have the advantage of being quick and easy to administer, they do not provide the specific reasons for a patient s degree of satisfaction with care. 25,29,30,35 Identifying those items or factors influencing patient satisfaction provides a richer understanding, and may allow clinicians to make those modifications necessary to maintain optimal levels of patient satisfaction with care. 4,25,29 Lengthy, multiitem questionnaires may, however, be expensive and time consuming to complete, resulting in reduced compliance by the patient and the clinician. Thus, a clinically useful measure of patient satisfaction with care should be relatively succinct, easy and inexpensive to administer, contain appropriate items, and be psychometrically sound. 25,35,40,45,47 A useful property of a patient satisfaction measure is the ability to discriminate between different factors affecting satisfaction. 19,25,29 For example, a patient may be very satisfied with his or her interaction with the therapist, but may be dissatisfied with the admissions process. The ability of a measure to reflect these differences is known as discriminant validity. During questionnaire development, discriminant validity is often initially assessed by performing various types of exploratory factor analysis (ie, statistical tests that suggest factors or groups of correlated items within the questionnaire). 8,14,34 Findings from exploratory factor analysis may vary, however, when different statistical tests or different samples are used. 17 Discriminant validity can be strengthened when a proposed factor structure is assessed on a different sample using more rigorous testing (ie, confirmatory factor analysis [CFA]). CFA assists in determining the appropriateness of fit (the degree to which the proposed factor structure is supported by the data), by correcting for measurement error and using multiple statistical tests that provide fit indices. 5,7,8,19,22-25,27,41,43 Interestingly, when evaluating published measures of patient satisfaction with outpatient physical therapy care, there is complete lack of agreement relative to the number and nature of factors between various instruments (Table 1). 4,16,32,41 The reason for this dissimilarity is uncertain but presumably results from variation in the nature of the type of items initially included in the developing questionnaire. For example, 2 of the instruments include 1 or more items regarding cost of care, 16,41 while the other 2 instruments include no items regarding cost of care. 4,32 Thus, the choice of which patient satisfaction measure to use may be setting specific, eg, a facility that treats many patients who self-pay may be more interested in a cost-of-care factor than one that has a low self-pay population. In a previous study, we reported preliminary data for the MedRisk Instrument for Measuring Patient Satisfaction With Physical Therapy Care (MRPS). 4 In a 2-part study, evidence of reliability, content, and criterion-referenced validity were reported from a large sample of patients (n = 1868) receiving worker s compensation for outpatient physical therapy. The TABLE 1. A summary of measures used for assessing patient satisfaction with outpatient physical therapy care that provide evidence of factor structure. Author (Year) Roush and Sonstroem (1999) 41 Number of Groups (Total Sample Size) Exploratory Factor Analysis 3 (607) PCA 2 with Varimax and Oblimin rotation 5 factors explained 47% of variance Goldstein et al 1 (289) PCA (2000) 16 1 factor explained 83% of variance Beattie et al (2002) 4 2 (2059) PCA with Varimax rotation 2 factors explained 50.12% of variance Monnin and Perneger (2002) 32 * 1 (528) PCA with Varimax rotation 3 factors explained 60% of variance Confirmatory Factor Analysis 4-factor solution was best fit Factor Labels Enhancer, detractor, locator, cost Number of Total Items in the Final Instrument Not reported N/A 20 Not reported Internal, external 12 Not reported Abbreviation: PCA, exploratory factor analysis using principal component analysis. * Developed in French language. Treatment, admission, logistics 34 14 RESEARCH REPORT J Orthop Sports Phys Ther Volume 35 Number 1 January 2005 25
proposed final instrument consisted of 10 specific items and 2 global items (Figure 1). During the initial development of the MRPS, a primary goal was to create an instrument that was able to discriminate between those items relating to the patient-therapist interaction, such as communication and respect (internal factor), and those items not specifically related to the patient-therapist interaction, such as the registration process or the courtesy of the receptionist (external factor). To assess this, we initially used exploratory factor analysis on the original data set. This provided preliminary data which suggested a 2-factor structure of the MRPS: internal factor (items 4-10), and external factor (items 1-3) (Figure 1). Because the study sample was limited to those individuals receiving workers compensation the generalizability of these findings is limited. The purpose of the present study was to determine the discriminant validity of the MRPS to differentiate between patient satisfaction measures relating to internal factors (patient-therapist interaction) and external factors (not related to the patient-therapist interaction) in large, diverse group of patients. In addition, this study assessed concurrent validity by determining the correlations between the 2 factors of the MRPS and its individual items with global measures of satisfaction. Describing the discriminant and concurrent validity of the MRPS will broaden its utility by allowing examiners to assess factors and items contained within this instrument that help explain patient satisfaction with physical therapy care. METHODS Subjects Data were obtained from consenting subjects who were over 18 years of age. All subjects had completed a course of physical therapy intervention at 1 of 6 outpatient clinics, 5 of which were in the state of Pennsylvania, and 1 of which was in the state of New York. The clinics were all privately owned and specialized in the evaluation and treatment of people with musculoskeletal problems. Each of these clinics was a participating subscriber of a patient satisfaction benchmarking service offered by Expert Clinical Benchmarks, LLC (ECB) (MedRisk, Inc, King of Prussia, PA). All subjects had to be able to read English and be able to complete the questionnaire. Subjects who had not completed their course of care did not participate in this study. All subjects who completed their course of care were asked to participate. The rights of the subjects were protected. Each subject signed a consent form approved by the MedRisk, Inc, Institu- 1. The office receptionist is courteous. 2. The registration process is appropriate. 3. The waiting area is comfortable. 4. My therapist does not spend enough time with me. 5. My therapist thoroughly explains the treatment(s) I receive. 6. My therapist treats me respectfully. 7. My therapist does not listen to my concerns. 8. My theapist answers all my questions. 9. My therapist advises me on ways to avoid future problems. 10. My therapist gives me detailed instructions regarding my home program. 11. Overall, I am completely satisfied with the services I receive from my therapist. 12. I would return to this office for future services or care. Copyright 2000 Expert Clinical Benchmarks, LLC. All rights reserved. FIGURE 1. The items contained within the MedRisk Instrument for Measuring Patient Satisfaction With Physical Therapy Care. Items 1 to 3 represent the external factor, while items 4 to 10 represent the internal factor. Items 11 and 12 are global measures of satisfaction. Patients were instructed to complete a1to5ratingscale for each item (1, strongly disagree; 2, disagree; 3, neutral; 4, agree; 5, strongly agree). Items 4 and 7 were recoded as positive during scoring (strongly disagree, 5; strongly agree, 1). tional Review Board that was included in the survey instrument. Specific patient names were not included in the database. Procedure The ECB proprietary benchmarking product was purchased by the 6 clinics noted in this data set. This subscription service supplied a secure password account on the Internet for the clinics to use as they set up their individualized databases. All transactions of patient data were compliant with the Health Insurance Portability and Accountability Act (HIPAA). As a new patient entered the clinic, the patient information was entered into the database (the name was stripped on any transactions to ECB). The MRPS was printed on the first date of treatment and was given to the patient on the last day of the episode of care. The patient was asked to complete the questionnaire in the waiting room of the clinic and to place it in an envelope, seal it, and put that envelope in a special container in the waiting room. The envelopes where gathered each week and mailed to the ECB corporate offices. The data were coded and entered into a 26 J Orthop Sports Phys Ther Volume 35 Number 1 January 2005
master database that included patient and provider demographics as well as diagnostic information. Scores for each item from the MRPS were coded from 1 (strongly disagree) to 5 (strongly agree). To reduce the likelihood of a subject scoring all items equally without carefully reading each question, items 4 and 7 (Figure 1) were intentionally negatively worded in the instrument, but were recoded as positive variables in the final database. The means of items 1 through 3 and 4 through 10 were calculated to determine the mean score for external and internal subscales, respectively. Items 11 and 12 were used as global references (construct variables) to assess the concurrent validity. Data Analysis Descriptive statistics, mean differences, correlations, and alpha testing were generated using SPSS Version 10.0 base, regression, and advanced software (SPSS, Chicago, IL). AMOS Version 4.0 software (Small Waters Corp, Chicago, IL) was used for CFA. Discriminant Validity In CFA the goal is to determine the degree to which the data are similar to, or fit the model being tested. To indicate optimal fit, the chi-square value should not be significant, implying that the model is not different from the data. However, a large sample size may often result in significance. 43 Therefore, we chose the following indices and decision rules for assessing model fit for questionnaire data, as described by Hoyle 22 and other authors. 5,7,23-25,27,34,43 Root-mean-square residual represents the square root of the fitted residuals that are calculated by subtracting the sample covariance from the estimated covariance matrix. The root-mean-square residual should be very small, approaching 0. 23 Two important indicators of the degree to which the data fit the model being tested are the comparative fit index and nonnormed fit index. These are considered to be incremental fit indices that test the proportionate improvement in fit by comparing the model being tested with a more restricted null model. Values greater than 0.90 are acceptable, while those greater than 0.95 represent a good fit. 24 The root-meansquare error of approximation describes closeness of fit. Values of 0.05 represent a close fit, while 0 represents an exact fit. 7 Following the CFA, Cronbach s alpha was determined for each of the 2 factors. Cronbach s alpha provides an additional index to assess the internal consistency of a subscale, or the degree to which the questions are similar. 40 In addition, Cronbach s alpha was used to calculate the standard error of measurement (SEM). 11,39 Reliability The reliability of the mean values from each of the 2 factors (internal and external) was determined by calculating the SEM. This procedure has been described in previous work as an appropriate way to assess reliability of measures obtained from crosssectional data. 4,11,30,39 The SEM was calculated using the following equation: SEM = SDs 1, where SDs equals the standard deviation of the mean of the observed factor and is the Cronbach s alpha for that factor. 4,11,39 The SEM functions as an index of reliability by describing the degree of error in a given measure, using the same units as that measure. 11,39 For example, when one is assessing the score of an individual, the higher the value of the SEM, the more error is associated with the measure (that is, the farther away the observed score may be from the true score). The SEM can provide a confidence interval of scores about which the true measure occurs. By adding to and subtracting 1.96 SEM from an observed score one may determine the range about which the true score will appear in 95% of persons with that observed score. For example, if the SEM is 0.2, the SEM with 95% confidence interval would be calculated by multiplying 1.96 0.2, and subtracting and adding this value (0.392) to the observed score. Thus, if the observed score is 4.0, one may be certain that 95% of the time, the true score lies between 3.608 and 4.392 (4.0 0.392 and 4.0 + 0.392). Concurrent Validity Concurrent validity is indexed by the association between 2 measures that are obtained in the same time period. 40 In survey research, this may be investigated by determining the correlation between the measures from an instrument with a variable of similar category that is obtained at the same time. The current study assessed correlations between individual items and global measures of satisfaction, (ie, MRPS items 11 and 12, Overall I am completely satisfied with the services I receive from my therapist, and I would return to this office for future services or care, respectively). Correlations were assessed using a Pearson s product moment correlation to determine the relationships between the mean scores of the 2 factors and the 2 global measures. This provided an estimate of the degree to which each of the factors correlated with the 2 global variables. In addition, correlations were assessed using a Pearson s product moment correlation to determine the relationships between each of the items in the 2 factors and the 2 global measures. This provided an estimate of the degree to which individual items were correlated to each of the global measures and may provide insight relative to criterionreferenced validity. 4,16,40 RESEARCH REPORT J Orthop Sports Phys Ther Volume 35 Number 1 January 2005 27
TABLE 2. Primary location of symptoms reported by patients (number of patients and percentage of patients). Female Male Total Lower extremity 295* (35) 177 (29) 472 (33) Lumbar and/or thoracic spine 237* (28) 169 (28) 406 (28) Upper extremity 125 (15) 118 (19) 243 (17) Cervical spine 130* (16) 92 (15) 222 (15) Other 45 (5) 51* (8) 96 (7) Hand and/or wrists 8 ( 1) 2 ( 1) 10 ( 1) Total 840 (100) 609 (100) 1449 (100) * Observed exceeded expected counts (X 2, 15.129; df, 5;P =.01). Journal of Orthopaedic & Sports Physical Therapy TABLE 3. Type of insurance coverage for the total sample (n = 1449). Insurance Coverage Frequency (n) Percentage Auto 50 3 Group health 202 14 HMO 304 21 Managed care 41 3 Medicare 259 18 Other 14 1 PPO 464 32 Personal injury 30 2 Workers compensation 85 6 Total 1449 100 TABLE 4. Descriptive statistics (means and SD) for the entire sample (n = 1449). Each item is rated on a 5-point scale (1, strongly disagree; 2, disagree; 3, neutral; 4, agree; 5, strongly agree). Two items ( My therapist spends enough time and My therapist listens to my concerns ) have been restated from their original negative wording (Figure 1). Item Mean SD Office receptionist is courteous 4.60 0.69 Registration process appropriate 4.49 0.70 Waiting area is comfortable 4.37 0.76 My therapist spends enough time 4.46 0.81 My therapist explains treatment 4.44 0.83 My therapist treats me respectfully 4.57 0.77 My therapist listens to my concerns 4.57 0.82 My therapist answered all my questions 4.53 0.75 My therapist advises ways to avoid 4.32 0.85 future problems My therapist gives detailed instructions regarding my home program 4.41 0.82 RESULTS Subjects Of the 4065 consecutive patients entered into the database by the 6 participating clinics, 1545 returned satisfaction surveys at discharge (40% return rate). Ninety-six surveys were not used because the individuals were less than 18 years of age. A total of 1449 surveys were used for this study. The mean age was 55.2 years, with a range of 18 to 101 years. Eight hundred forty (58%) of the subjects were females. The mean age of female subjects (56.2 years) was greater than that of the males subjects (53.7 years) (P =.009). The location of symptoms for which each patient was treated is identified in Table 2. Females were more likely than males to receive care for lower extremity or spine problems, while males were more likely than females to receive care for other conditions. The frequency counts and percentages for insurance types are listed in table 3. The highest percentage was preferred provider organization (PPO) at 32% (n = 464), followed by Health Maintenance Organization (HMO) at 21% (n = 304). The least percentage was other at 1% (n = 14), followed by personal injury at 2% (n = 30). Summary of Item Responses Descriptive statistics for item endorsement are summarized in Table 4. The overall highest mean score was 4.60/5 for The receptionist is courteous, while the lowest mean score was 4.32 for My therapist advises me on ways to prevent further problems. Discriminant Validity Figure 2 demonstrates the factor loadings and error terms for the confirmatory factor analysis. The results of the confirmatory factor analysis are presented in Table 5. The fit indices (root-mean-square residual, comparative fit index, nonnormal fit index, and root-mean-square error of approximation) all indicated a good to excellent degree of fit, supporting the hypothesis that the internal and external factors discriminate between one another. The intercorrelation between factors was 0.83. Cronbach s alpha was 0.87 for the external factor and 0.90 for the internal factor (Table 6). Reliability The SEM was 0.24 for the external factor and 0.19 for the internal factor (Table 6). 28 J Orthop Sports Phys Ther Volume 35 Number 1 January 2005
.83 External Internal.86.94.79.40.79.78.36.89.73.79 Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10.73.89.63.16.62.61.13.79.53.63 Journal of Orthopaedic & Sports Physical Therapy FIGURE 2. Confirmatory factor analysis of the MedRisk Instrument for Measuring Patient Satisfaction With Physical Therapy Care. The curved arrow shows the intercorrelation (.83) between external and internal factors. Questions 1 through 3 refer to items 1 through 3 in Figure 1. Questions 4 through 10 refer to items 4 through 10 in Figure 1. Standardized factor loadings are listed above each item (questions 1 to 10) and ranged from.79 to.94 for the external, and.40 to.89 for the internal factors. The error terms are listed below each item. The following error terms were correlated: Q4 and Q7, Q5 and Q6, and Q9 and Q10. Criterion-Referenced Validity The correlations of the external factor and internal factors to the 2 global measures ( Overall, I am completely satisfied with the services I receive from my therapist, and I would return to this office for future services or care. ) are listed in Table 7. Both factors were significantly correlated with the 2 global items ranging from r = 0.71 to 0.83. Table 8 lists the correlations of individual items with the 2 global questions. All comparisons had significant positive correlations ranging from r = 0.33 to 0.80. DISCUSSION The results of confirmatory factor analysis suggest that our data set has an acceptable fit with the 2-factor model that was tested. Although the intercorrelation between factors was high (0.83), indicating that the internal and external factors are related, the fit indices suggest a 2-factor model supporting the hypothesis that the MRPS discriminates between the factor relating to the patient-therapist interaction (internal), and the factor relating to the process of interacting with the receptionist and waiting to be seen by the therapist (external). The high alpha values for each of the factors illustrate internal consistency. The small values for the SEM indicate that the mean scores from the 2 factors of the MRPS have a low amount of measurement error, while the TABLE 5. Model fit indices derived from confirmatory factor analysis. Index Observed Value Target Value RMR.014 Should approach 0 CFI.989.95 NNFI.983.95 RMSEA (95% CI).050 (.042-.058).05 Abbreviations: RMR, root-mean-square residual; CFI, comparative fit index; NNFI, nonnormed fit index; RMSEA, the root-mean-square error of approximation; CI, confidence interval. TABLE 6. Internal consistency (Cronbach s alpha), mean, standard deviation (SD), and reliability index (standard error of measurement [SEM]) of the 2 factors. Scale Alpha Mean SD SEM External (items 1-3) 0.87 4.49 0.65 0.24 Internal (items 4-10) 0.90 4.47 0.61 0.19 high correlations of each of the 2 factors to the global reference measures support criterionreferenced validity of these 2 factors for the population that was sampled. Our purpose was to assess our 2-factor model for one of its intended uses (ie, discriminating between internal and external factors). The data suggest that the MRPS is psychometrically sound for this task when applied to those subjects who complete their RESEARCH REPORT J Orthop Sports Phys Ther Volume 35 Number 1 January 2005 29
course of physical therapy. Because we did not perform comparisons of the MRPS to other factor structures, we do not know if other models, such as a 1-factor or 3-factor model, would have superior psychometric properties. It is important to note that factor analyses are sensitive to the unique characteristics of the sample, thus the factor structure may vary among different populations. 17 Considering this, in previous work, 4 as well as in the current study, we assessed 3 large samples, totaling 3317 Englishspeaking subjects who reported various diagnoses and payment characteristics and who completed outpatient physical therapy, from which to validate the MRPS. We believe that this improves the generalizability of our findings; however, the applicability of our data to people who do not complete their course of physical therapy or who are in other cultures or healthcare settings is not known. In addition to discriminating between factors, multi-item satisfaction measures can provide useful information by assessing the relationship of specific items to overall patient satisfaction. For example, in the present study the item most closely correlated with the 2 global questions was My therapist answers all my questions, followed by My therapist gives me detailed instructions regarding my home program (Table 8). Other items that were highly correlated included My therapist respects me, My therapist advises me, and My therapist explains treatment. These findings support the concept that the patient s perception of the quality of the professional interaction with the therapist, especially the meaningful exchange of relevant information, is a critical component of patient satisfaction with physical therapy care. Being treated with respect by health care providers and being involved in treatment decisions are strongly linked to patient satisfaction. 3,6,9,29,37,42 Interestingly, the item The registration process is appropriate had a much higher correlation to the global measures in the present study than in our previous work 4 (r = 0.693 compared to r = 0.395), while My therapist spends enough time and My therapist listens to my concerns was much lower (r = 0.369 compared to r = 0.640, and r = 0.344 compared to r = 0.662, respectively). 4 The reason for this is unclear. Although My therapist spends enough time and My therapist listens to my concerns were intentionally negatively worded in the questionnaire and recoded for analysis, correlations using unrecoded data yielded similar but negative correlations; therefore, a coding error is unlikely. These findings, therefore, may illustrate differences in the preferences between the populations studied. In the initial study, all subjects (n = 1868) were receiving worker s compensation, the mean age was 46.9 years, and 64% were male. In the current study, only 6% were receiving worker s compensation, the mean age of all subjects was 55.2 years, and 42% were male. These data suggest that there may be differences in the degree to which various items affect satisfaction with care for those people receiving worker s compensation compared to those who are not. For example, having enough time with the physical therapist may be a bigger concern related to satisfaction than is the registration process for an individual receiving worker s compensation; but it may not be so for an individual with a different type of insurance coverage. In all cases, however, having the physical TABLE 7. Pearson product-moment correlations of the 2 factors to each of the 2 global measures and to the mean of these 2 measures. All correlations are significant at P.01. Factor Overall Satisfaction* Global Measures Would Return Mean of Global Measures External (items 1-3) 0.709 0.705 0.715 Internal (items 4-10) 0.826 0.803 0.830 * Overall, I am completely satisfied with the services I receive from my therapist. I would return to this office for future services or care. TABLE 8. Pearson product-moment correlations of each of the Items from the internal and external factors to the 2 global measures and to the mean of these 2 measures. All correlations are significant at P.01. Global Measures Mean of Global Measures Item Overall Satisfaction* Would Return PT answers my 0.794 0.781 0.803 questions PT gives detailed 0.759 0.746 0.768 instructions regarding my home program PT respects me 0.714 0.721 0.761 Registration is appropriate 0.688 0.681 0.693 0.681 0.663 0.685 PT advises me on ways to prevent future problems PT explains treatment Receptionist is courteous Waiting area is comfortable PT spends enough time PT listens to my concerns 0.676 0.662 0.683 0.650 0.660 0.663 0.579 0.586 0.559 0.386 0.334 0.369 0.343 0.328 0.344 Abbreviation: PT, physical therapist. * Overall, I am completely satisfied with the services I receive from my therapist. I would return to this office for future services or care. 30 J Orthop Sports Phys Ther Volume 35 Number 1 January 2005
therapist answer the patient s questions remains the item most highly correlated with satisfaction with care. We are currently investigating the relationship of patient satisfaction with care relative to other factors such as health insurance coverage. It is important to note that the mean item responses were uniformly high, ranging from 4.3 to 4.6 (Table 4). This indicates that for the questions asked in our survey, patients typically agreed or agreed strongly. These findings are consistent with our previous work and with those reported by others. 4,41 In our sample, patients were asked to complete the instrument at the time of their last scheduled visit; thus these data may be skewed toward those patients who were likely to be satisfied with care (ie, dissatisfied patients may be less likely to return for follow-up appointments and would not be well represented). This could result in misleading high scores for patient satisfaction. Interestingly, however, Roush and Sonstroem 41 also described high mean scores (4.2/5 to 4.3/5) on 2 separate samples, for a factor labeled as enhancers (items relating to contentment with physical environment and personal interactions that occur during the clinic visit). While these authors used very different sampling procedures, which assessed patients from a variety of settings at certain points during the course of care, our data are very similar. The degree, however, to which our data and that of Roush and Sonstroem 41 can be generalized to all patients receiving outpatient physical therapy remains uncertain. To provide a richer description of patient satisfaction with physical therapy care, further investigation using random selection of subjects is necessary. It is important to consider that patient satisfaction with care, as measured by the MRPS, should be differentiated from satisfaction with outcome. 25 Conceptually, satisfaction with care describes the service a patient receives during his or her course of treatment, while satisfaction with outcome relates to the effect of treatment on overall health status. These 2 constructs, although potentially linked to one another, arguably are different and should be assessed separately with appropriate measures. The relatively short length of the MRPS, coupled with its strong psychometric properties assessed on a large, diverse population make it a potentially useful tool for clinical practice and research related to patient satisfaction with care when applied to populations from which it has been validated. This instrument is intended for use in an outpatient physical therapy environment and has not been assessed for inpatient use. Although the MRPS has been validated in the United States health care system, it is currently being tested in other countries to determine its suitability for use in a variety of cultures and health care systems. CONCLUSION Our findings provide evidence of discriminant and concurrent validity of the 2-factor solution for the MRPS questionnaire in the sample that was tested. This 2-factor solution provides measures that are relatively free of error and may discriminate between internal and external factors that influence patient satisfaction with care. In our sample, overall patient satisfaction was most closely correlated with the degree to which the physical therapist answered questions, provided information, and was respectful toward the patient. Patients who complete their course of physical therapy report that the professional interaction between the therapist and patient, especially the meaningful exchange of relevant information, is critical for patient satisfaction with care. The generalizability of our data to patients who do not complete their physical therapy care or who are receiving care in other health care environments is unknown. REFERENCES 1. Baker J, Lamb CW, Jr. Physical environment as a hospital marketing tool. J Hosp Mark. 1992;6:25-35. 2. Baker R. Development of a questionnaire to assess patients satisfaction with consultations in general practice. Br J Gen Pract. 1990;40:487-490. 3. Baker SM, Marshak HH, Rice GT, Zimmerman GJ. Patient participation in physical therapy goal setting. Phys Ther. 2001;81:1118-1126. 4. Beattie PF, Pinto MB, Nelson MK, Nelson R. Patient satisfaction with outpatient physical therapy: instrument validation. Phys Ther. 2002;82:557-565. 5. Browne MW, Cudeck R. Alternative ways of assessing model fit. In: Bollen KA, Long JS, eds. Testing Structural Equation Models. Thousand Oaks, CA: Sage Publications; 1993:136-162. 6. Bryant R, Graham MC. Advanced practice nurses: a study of client satisfaction. J Am Acad Nurse Pract. 2002;14:88-92. 7. Carmines EG, McIver JP. Analyzing models with unobserved variables. In: Bohrnstedt GW, Borgatte EF, eds. Social Measurement: Current Issues. Thousand Oaks, CA: Sage Publications; 1981:65-115. 8. Cattell RB. The Scientific Use of Factor Analysis in Behavioral and Life Sciences. New York, NY: Plenum Press; 1978. 9. Cleary PD, Edgman-Levitan S. Health care quality. Incorporating consumer perspectives. JAMA. 1997;278:1608-1612. 10. Deyo RA, Diehl AK. Patient satisfaction with medical care for low-back pain. Spine. 1986;11:28-30. 11. Diamond JJ. A practical application of reliability theory to family practice research. Fam Pract Res J. 1991;11:357-362. 12. Dilman DA. Mail and other self-administered questionnaires. In: Rossi PH, Wright JD, Anderson AB, eds. Handbook of Survey Research. New York, NY: Academic Press; 1982:68-95. RESEARCH REPORT J Orthop Sports Phys Ther Volume 35 Number 1 January 2005 31
13. DiTomasso RA, Willard M. The development of a patient satisfaction questionnaire in the ambulatory setting. Fam Med. 1991;23:127-131. 14. Domholdt E. Physical Therapy Research: Principles and Applications. Philadelphia, PA: WB Saunders Co; 1993. 15. Elliott-Burke TL, Pothast L. Measuring patient satisfaction in an outpatient orthopedic setting, part 1: key drivers and results. J Rehabil Outcomes Meas. 1997;1:18-25. 16. Goldstein MS, Elliott SD, Guccione AA. The development of an instrument to measure satisfaction with physical therapy. Phys Ther. 2000;80:853-863. 17. Gracely RH. Evaluation of multi-dimensional pain scales. Pain. 1992;48:297-300. 18. Hair JF, Anderson RE, Tatham RL, Black WC. Multivariate Data Analysis With Readings. 3rd ed. New York, NY: MacMillan Publishing; 1987. 19. Hall JA, Dornan MC. Meta-analysis of satisfaction with medical care: description of research domain and analysis of overall satisfaction levels. Soc Sci Med. 1988;27:637-644. 20. Hall JA, Dornan MC. What patients like about their medical care and how often they are asked: a metaanalysis of the satisfaction literature. Soc Sci Med. 1988;27:935-939. 21. Hendriks AA, Oort FJ, Vrielink MR, Smets EM. Reliability and validity of the Satisfaction with Hospital Care Questionnaire. Int J Qual Health Care. 2002;14:471-482. 22. Hoyle RH. The structural equation modeling approach: basic concepts and fundamental issues. In: Hoyle RH, ed. Structural Equation Modeling Concepts, Issues and Applications. Thousand Oaks, CA: Sage Publications; 1995:1-15. 23. Hu LT, Bentler PM. Cutoff criteria for fit indices in covariance structure analysis: conventional criteria versus new alternatives. Struc Equat Model. 1999;6:1-55. 24. Hu LT, Bentler PM. Evaluating model fit. In: Hoyle RH, ed. Structural Equation Modeling Concepts, Issues and Applications. Thousand Oaks, CA: Sage Publications; 1995:76-99. 25. Hudak PL, Wright JG. The characteristics of patient satisfaction measures. Spine. 2000;25:3167-3177. 26. Hutchinson SR, Olmus A. Behavior of descriptive fit indices in confirmatory factor analysis using ordered categorical data. Struc Equat Model. 1998;5:344-364. 27. Jó reskog KG, Só rbom D. LISREL 8: Structural Equation Modeling With the SIMPLIS Command Language. Chicago, IL: Scientific Software International; 1993. 28. Keith RA. Patient satisfaction and rehabilitation services. Arch Phys Med Rehabil. 1998;79:1122-1128. 29. Linder-Pelz S, Struening EL. The multidimensionality of patient satisfaction with a clinic visit. J Community Health. 1985;10:42-54. 30. McHorney CA, Tarlov AR. Individual-patient monitoring in clinical practice: are available health status surveys adequate? Qual Life Res. 1995;4:293-307. 31. Meredith LS, Orlando M, Humphrey N, Camp P, Sherbourne CD. Are better ratings of the patientprovider relationship associated with higher quality care for depression? Med Care. 2001;39:349-360. 32. Monnin D, Perneger TV. Scale to measure patient satisfaction with physical therapy. Phys Ther. 2002;82:682-691. 33. Nitse PS, Rushing V. Patient satisfaction: the new area of focus for the physician s office. Health Mark Q. 1996;14:73-84. 34. Nunnally JC. Psychometric Theory. New York, NY: McGraw-Hill; 1978. 35. Olejnik S, McKinely CO, Ellis RA, Buchanan JR, Kersey GE, Clark KG. Construct validation of customer satisfaction inventories. J Rehabil Outcomes Meas. 1998;2:30-38. 36. Patient Satisfaction Instruments: A Compendium. Alexandria, VA: American Physical Therapy Association; 1995. 37. Perreault M, Katerelos TE, Sabourin S, Leichner P, Desmarais J. Information as a distinct dimension for satisfaction assessment of outpatient psychiatric services. Int J Health Care Qual Assur Inc Leadersh Health Serv. 2001;14:111-120. 38. Pinto MB, Leonidas L. The impact of office characteristics on satisfaction with medical care: a before and after analysis. Health Mark Q. 1994;12:43-54. 39. Roddey TS, Olson SL, Cook KF, Gartsman GM, Hanten W. Comparison of the University of California-Los Angeles Shoulder Scale and the Simple Shoulder Test with the shoulder pain and disability index: singleadministration reliability and validity. Phys Ther. 2000;80:759-768. 40. Rothstein JM, Echternach JL. Primer on Measurement: An Introductory Guide to Measurement Issues. Alexandria, VA: American Physical Therapy Association; 1993. 41. Roush SE, Sonstroem RJ. Development of the physical therapy outpatient satisfaction survey (PTOPS). Phys Ther. 1999;79:159-170. 42. Safran DG, Taira DA, Rogers WH, Kosinski M, Ware JE, Tarlov AR. Linking primary care performance to outcomes of care. J Fam Pract. 1998;47:213-220. 43. Shumaker RE, Lomax RG. A Beginner s Guide to Structural Equation Modeling. Mahwah, NJ: Lawrence Erlbaum Associates, Inc; 1996. 44. Simon SE, Patrick A. Understanding and assessing consumer satisfaction in rehabilitation. J Rehabil Outcomes Meas. 1997;1:1-14. 45. Ware JE, Jr., Hays RD. Methods for measuring patient satisfaction with specific medical encounters. Med Care. 1988;26:393-402. 46. Wensing M, Elwyn G. Methods for incorporating patients views in health care. BMJ. 2003;326:877-879. 47. Yellen E, Davis GC, Ricard R. The measurement of patient satisfaction. J Nurs Care Qual. 2002;16:23-29. 32 J Orthop Sports Phys Ther Volume 35 Number 1 January 2005