EDUCATION. Abstract. Keywords Education; nurse practitioners; students; communication.

EDUCATION Effective use of interpreters by family nurse practitioner students: Is didactic curriculum enough? Susanne J. Phillips, MSN, RN, FNP-BC (Associate Professor) 1, Desiree Lie, MEd, MD (Professor) 2, Jennifer Encinas, BA (Statistician) 3, Carol Sue Ahearn, BSN, RN (Director) 3,&SusanTiso,MN, RN, FNP-BC (Associate Professor) 1 1 Program in Nursing Science, University of California, Irvine, California 2 Department of Family Medicine, School of Medicine, University of California, Irvine, California 3 Clinical Skills Center, School of Medicine, University of California, Irvine, California Keywords Education; nurse practitioners; students; communication. Correspondence Susanne J. Phillips, MSN, RN, FNP-BC, Program in Nursing Science, University of California, 209 Irvine Hall, Irvine, CA 92697. Tel: 949-824-4274; Fax: 949-824-0470; E-mail: sjphilli@uci.edu Received: June 2009; accepted: January 2010 doi: 10.1111/j.1745-7599.2011.00612.x Abstract Purpose: Nurse practitioners (NPs) care for patients with limited English proficiency (LEP). However, NP education for improving communication in interpreted encounters is not well reported. We report a single school study using standardized encounters within a clinical practice examination (CPX) to assess the adequacy of current curriculum. Data sources: Entering family NP (FNP) students (n = 26) participated in a baseline CPX case. They were assessed by standardized patients using the validated Interpreter Impact Rating Scale (IIRS) and Physician-Patient Interaction (PPI) scale, and by interpreters using the Interpreter Scale (IS).The case was re-administered to 31 graduating students following completion of existing curriculum. Primary outcome was aggregate change in skills comprising global IIRS, PPI and IS scores. Pre- and post-performance data were available for one class of 10 students. Secondary outcome was change in skill scores for this class. Conclusions: Mean aggregate global scores showed no significant improvement between scores at entry and graduation. For 10 students with pre- and post-performance data, there was no improvement in skill scores for any measure. Skill assessed on one measure worsened. Implications for practice: FNP students show no improvement in skills in working with interpreters with the current curriculum. An enhanced curriculum is needed. Nurse practitioners (NPs) are important providers of primary care to underserved and ethnically diverse populations in the United States (U.S. Government Accountability Office [US GAO], 2008). With an increasing number of linguistically and culturally diverse patients seeking primary care within the U.S. healthcare system (Hayakawa, 1982; U.S. Census Bureau, 2000, 2004), there is a growing need for healthcare providers to be formally trained in working effectively with medical interpreters (Civil Rights Act, 1964; Grubbs, Chen, Bindman, Vittinghoff, & Fernandez, 2006; U.S. Department of Health and Human Services, Office of Minority Health, 2000). Studies have shown that the use of medical interpreters in the healthcare setting reduces health disparities (Flores, 2005; Karliner, Perez-Stable, & Gildengorin, 2004). Cultural and ethnic competency standards for NPs emphasize the need to be sensitive to and increase communication with ethnically diverse populations, highlighting the need to provide such training in NP programs (U.S. Department of Health and Human Services [US DHHS], 2002). Professional nursing practice is founded on the principles of maintaining respect for the cultural, racial, and ethnic diversity of patients (American Nurses Association, 2004). In 1990, the National Organization of Nurse Practitioner Faculties and the American Association of Colleges of Nursing developed domains and core competencies of NP practice in response to federal concerns for improving quality of educational programs supported by Title VIII, Nursing Workforce Journal of the American Academy of Nurse Practitioners 23 (2011) 233 238 C 2011 The Author(s) Journal compilation C 2011 American Academy of Nurse Practitioners 233

Effective use of interpreters S.J. Phillips et al. Development (Department of Health and Human Services, 2002). Included in the competencies are essential behaviors that must be demonstrated upon graduation from an NP educational program. Two of these essential behaviors, defined within Domain VII: Cultural Competence (US DHHS, 2002), require the NP to demonstrate culturally sensitive care and to assist patients of diverse cultures to access quality care. Appropriate use of medical interpreters assists NPs in achieving this standard. Despite the description of formal competencies for the provision of culturally and ethnically appropriate care in NP education, there is a lack of research in the nursing literature that assesses the need for and the effectiveness of formalized curricula in working effectively with medical interpreters. Until recently, validated assessment tools for evaluating these specific communication skills have been lacking, limiting the ability to conduct pre- to postassessment of skills around newly introduced curricula. With recent access to validated assessment tools for rating encounters involving interpreters, we conducted a needs assessment to examine skills of family nurse practitioner (FNP) students for working effectively with interpreters at entry into the program versus at graduation, to determine the effectiveness of our current didactic curriculum and to assess the need for curricular enhancement. We hypothesized that without a skills-based competency component in the current didactic curriculum, over a 2-year program, FNP students would not demonstrate improvement in these skills as assessed by standardized patients (SPs) and standardized interpreters (SIs). Our study, conducted from 2006 to 2008, was approved by the Institution s Review Board. Methods Curriculum The FNP program at the University of California, Irvine, was, until 2009, comprised of a postmaster s certificate track and MSN/FNP program in partnership with the California State University at Fullerton (nb: in 2009 it became an independent academic program at UCI). Each year, since 1995, 7 15 students were enrolled in the program. The curriculum requirements are 45 semester units in the postmaster s track and 64 semester units in the MSN/FNP track. Included within the total units is 720 h of clinical internship. Assessment of FNP student skills is conducted by a variety of methods, including written examinations, case presentations, direct observation of performance, a baseline skills-appraisal assessment (SKA), a mid-program objective structured clinical examination (OSCE), and a final multistation clinical practice examination (CPX) conducted prior to graduation. At the time Table 1 Student classes for data analysis Characteristics Total, n (%) Student gender Male 4, (7.0) Female 53, (93.0) Student class size by graduating year 2006 12 2007 7 2008 13 2009 15 Student level (aggregate) FNP1 26 FNP2 31 Subgroup with complete pre/post data. of the study, there was no formal skills-based or clinical curriculum for working with interpreters. FNP students received a didactic session lasting 2.5 h in their Vulnerable Populations course that covered content related to culture, communication, and use of interpreters, including legal requirements for the use of interpreters. The curriculum, designed by the course instructor, included learning activities such as assigned reading, lecture/discussion, role-playing and observation, and video case presentations with critique, which covered impact of body language, cultural sensitivity, and styles of communication. Because of the demographics of patients served in our local community with limited English proficiency (predominantly Latino with Vietnamese and other Asian subgroups), and the low level of Spanish and other language fluency among our FNP students, a curriculum to teach skills for working with interpreters was considered a significant program need. Study participants Participants were FNP students in four successive classes (see Table 1) attending our FNP Program who matriculated in the year 2004 2008, some of whom graduated in 2006 2008. Students were assessed in a baseline and a graduation CPX and their skills for working with interpreters rated by trained SPs and SIs who performed in a standardized case. Because of the small numbers of students (range 7 15 in each class), aggregate data from four successive classes were used. Interpreter CPX station The 15-min interpreter station was administered within a multistation OSCE-type CPX. The OSCE is a set of standardized clinical encounters using trained SPs to rate students in their clinical skills and competencies. Skills that 234

S.J. Phillips et al. Effective use of interpreters are assessed using OSCEs include history taking, communication, physical examination, and counseling (Harden & Gleeson, 1979; Van der Vleuten & Swanson, 1990). The interpreter case involved a monolingual Spanishspeaking patient requiring counseling in smoking cessation. Spanish language was chosen because of local community demographics. To prepare for their roles, both the SPs and the SIs (actors who were bilingual in Spanish and English) received four 2-h training sessions as a pair by English-speaking trainers. The training sessions included learning and rehearsing the case study, roleplaying, and review of recorded encounters. One SP SI pair was used to standardize case performance by having them rate videotaped encounters prior to the examination. Extensive anchors were developed for the behaviors being rated and used for training. While training, trainers directly observed and critiqued the performance of the SPs and interpreters both individually and as a pair in their dual purpose roles as case actors and student raters. All raters were required to consistently score recorded training encounters with an error rate of no more than one error per encounter (Lie et al., 2007, 2009). Following the CPX, SPs completed both the Interpreter Impact Rating Scale (IIRS) and Patient Physician Interaction Scale (PPI). Interpreters completed only the Interpreter Scale (IS). FNP students completed a self-rating of their performance immediately after the encounter outside the room. Over the 2-year study, none of the FNP students had Spanish fluency sufficient to conduct medical interviews in Spanish, by self-report. Because some FNP students had limited Spanish proficiency, the station instruction specified that students were being assessed for their ability to perform in an interpreted encounter and students should not use Spanish to conduct the interview even if they had some Spanish proficiency. Outcome measures-rating scales The IIRS is a validated scale assessing communication in an interpreted encounter from the patient s perspective (Lie et al., 2007). It consists of six items, including eye contact, addressing patient concerns, acknowledging patient beliefs, addressing patient in first person, seating at comfortable distance, and nonverbal language, describing verbal and nonverbal behaviors. A 5-point Likert scale (from 1 = marginal/low to 5 = outstanding performance) was used to rate each behavior. The global measure rating overall satisfaction of the patient (item 7 on the IIRS) was found to be predictive of performance (Lie et al., 2007) and thus used for our study. The IS is a similar validated measure, rating provider communication performance in an interpreted encounter from the interpreter s perspective, using a similar 5-point Likert scale for each behavior (Lie et al., 2009). It consists of 12 behavior items, with the first four rating the ability of the provider to set the stage for the encounter and the remaining eight items rating communication verbal and nonverbal behaviors, including seating, eye contact, not interrupting, clarifying statements, correcting mistakes, use of first person for patient, summarizing and closing the interview, and staying on track. The global measure (item 13 on the IS) of overall satisfaction was found to be predictive of overall performance in an interpreted encounter and was thus used for our study (Lie et al., 2009). The PPI, performed by the SP, is a validated seven-item rating scale using a 6-point Likert scale (ranging from 1 = unacceptable to 6 = outstanding) to measure the competency of communication in the context of professional behavior (Makoul, 2001). The seven items are: appeared professionally competent, effectively gathered information, listened actively, established personal rapport, appropriately explored my perspective, addressed my feelings, and met my needs. The mean of the summed score of the PPI (maximum of 7) was used to rate overall communication skills in the FNP students. For the self-assessment, FNP students were asked to rate their own skills for 11 items on a 6-point Likert scale (from 1 = very poor to 6 = excellent). The behaviors they self-rated included ability to involve the interpreter, elicit patient s cultural beliefs, work effectively with the interpreter, focus on the patient, assess accuracy of interpretation, set boundaries, and the perceived satisfaction of the patient and interpreter with their communication skills (Lie et al., 2007). Their overall self-assessed skill for interacting with the patient ( How satisfied do you think the patient was with how you involved the interpreter during this encounter? ) was used as a comparison with the patient (IIRS 7) and interpreter (IS 13) measures. Descriptive statistics were used. We calculated the mean score for each measure (IIRS, IS, self and the mean of summed PPI scores) at program entry and just prior to graduation for the CPX interpreter station. We first compared the mean scores for the aggregate of entering versus graduating students. We then performed a subgroup analysis for the single FNP class (class of 2008) for which we had both entering and graduating data (n = 10), to assess if our aggregated results were supported by those from a single longitudinal cohort followed from enrollment to graduation. Statistical analysis was performed using the software package JMP version 7.0.1 (SAS Institute Inc., Cary, NC). A one-way analysis of variance with similar group sizes was conducted to determine if differences existed within the groups being compared. The Student s t-test was used for comparisons of each pair. A value of p <.05 was used 235

Effective use of interpreters S.J. Phillips et al. as the standard for statistical significance for paired comparisons. Because this was a needs assessment study, FNP students were not given feedback by faculty observers about their individual performance after the baseline CPX station. They also did not receive additional training based on their performance, for interacting with interpreters. Results Over the 2 years of the study, complete skills data were available for an aggregate of 26 entering and 31 graduating FNP students (Table 1) who matriculated between 2004 and 2008. Data for 5 student encounters in the interpreter stations for 2006 are missing as a result of technical problems with audio recording. These students did not differ in mean age or experience from students included in this study. Of FNP students whose scores were included for analysis, 90% were female. Mean age was 39.7 (range 29 55 years). Both baseline and graduating information was available for one class of 13 FNP students matriculating in 2006 and graduating in 2008, with a mean age of 43.5 years. However for this class, two students graduated early and one student extended her program of study, leaving 10 students with both pre- and post-training data for the CPX interpreter station in this cohort. Aggregate data for entering and graduating classes Overall performance as assessed by the SPs on the IIRS global score was average, with a mean score of 3.35 for the entering class versus a lower score of 2.87 of 5 for the graduating class (difference of 0.48 points, p =.054; see Table 2). Performance as assessed by the SI using the IS global score improved from a mean of 3.31 for the en- Table 2 Aggregate comparison- skill scores for FNPs 2006 2008 FNP1 FNP2 Mean Difference (n = 26) (n = 31) Entering-graduating p-value Scale Total, n (%) Mean,SD (95%CI) IIRS 3.35 (0.89) 2.87 (0.92) 0.47 ( 0.01; 0.96).0542 IS 3.31 (0.74) 3.77 (0.67) +0.46 (0.09; 0.84).0152 Self 5.38 (0.57) 5.42 (0.56) +0.04 ( 0.27; 0.34).8187 PPI 4.78 (0.91) 4.90 (0.85) +0.12 ( 0.35; 0.59).6144 p <.05. IIRS = Interpreter Impact Rating Scale, scored on 5-point Likert scale (1 = marginal/low to 5 = outstanding). IS = Interpreter Scale, scored on 5-point Likert scale (1 = marginal/low to 5 = outstanding). Self = Self-rating of perceived patient satisfaction with provider, score on 6-point rating scale (1 = very poor to 6 = excellent). PPI = summed score of 7 items with maximum score of 7. Table 3 Pre to post skills scores for single FNP class of 2008 Pre Post Mean difference (n = 10) (n = 10) entering-graduating p-value Scale Total, n (%) Mean,SD (95%CI) IIRS 3.90 (0.74) 2.90 (0.57) 1.00 (0.38; 1.62).0032 IS 3.20 (0.63) 3.50 (0.53) +0.30 ( 0.25; 0.85).2643 Self 5.40 (0.52) 5.30 (0.48) 0.10 ( 0.37; 0.57).6601 PPI 5.34 (0.49) 5.34 (0.21) 0.00 ( 0.35; 0.35) 1.0000 p < 0.005. IIRS = Interpreter Impact Rating Scale, scored on 5-point Likert scale (1 = marginal/low to 5 = outstanding). IS = Interpreter Scale, scored on 5-point Likert scale (1 = marginal/low to 5 = outstanding). Self = Self-rating of perceived patient satisfaction with provider, score on 6-point rating scale (1 = very poor to 6 = excellent). PPI = summed score of seven items with maximum score of 7. tering to 3.77 of 5 for the graduating class (difference of +0.47 points, statistically significant at p =.015). The PPI rating by patients remained unchanged with a mean score of 4.78 of 7 at entry and a graduation mean score of 4.90 (difference +0.12, p =.614). Self-assessed perceived patient satisfaction with the encounter remained high at entry and graduation (5.38 and 5.42 of 6, respectively, difference +0.03, p =.819). Subgroup analysis of one student cohort For the 10 FNP students who had complete data at entry and at graduation, performance as rated by the SP on the global IIRS item fell from a mean of 3.90 at entry to 2.90 at graduation (mean difference 1.00, significant at p =.003; see Table 3). SI rating improved slightly from a mean global score of 3.20 to 3.50 at graduation but the difference of +0.3 points was not significant (p =.26). PPI scores remained similar at a mean of 5.40 at entry and 5.30 at graduation of 7 (difference of +0.1, not statistically significant). PPI scores remained unchanged at 5.34 of 7 at both entry and graduation for this cohort. Further analysis of individual items of the IIRS identified no particular behaviors out of six individual items rated, which accounted for the drop in score for the IIRS. In addition, no particular student accounted for the drop in IIRS global score between matriculation and graduation. Apart from one student who improved in the IIRS 7 score, all others had scores that remained unchanged or fell between matriculation and graduation (data not shown). Discussion Is didactic curricular content alone adequate to improve FNP students competency for working effectively with 236

S.J. Phillips et al. Effective use of interpreters medical interpreters? To answer this question, we assessed the skills of FNP students in one U.S. FNP program. We used a combination of two global communication measures that have been developed and validated for this purpose: the IIRS (Lie et al., 2007) and the IS (Lie et al., 2009); a general communication skill measure, the PPI (Makoul, 2001); and student self-ratings (Lie et al., 2007) to provide multiple perspectives on communication skills between matriculation and graduation. The study was conducted in the absence of a skills-based curriculum for effective use of medical interpreters. We documented a lack of improvement in all the skills assessed between matriculation and graduation with some decrease in skills. The trend for deterioration of skills instead of improvement in both the aggregate data of the entering and graduation groups, and the individual class of 2008, as assessed by the SP and SI were of significant concern. There are several possible explanations for this observation. First, after their baseline assessment in the interpreter skill station, FNP students were not informed of their skill deficits and had no opportunity to correct mistakes or improve their skills specific to interpreted encounters. Second, during their internships in clinical practices, they may have observed behaviors inconsistent with appropriate skills for working with interpreters, and picked up bad habits, which were not corrected or were reinforced. Of interest in this study, their PPI scores remained high and unchanged over 2 years, suggesting that skill deficits were specific to interpreted encounters and did not necessarily apply to communication skills in general. As well, and of potentially high significance, their self-assessed measure of perceived patient satisfaction remained high at baseline and graduation, suggesting that they were unaware of the relatively low scores from the interpreter CPX station. There are several strengths of our study. We used validated scales (IIRS and IS) for measuring communication in the interpreted encounter. We standardized the training of SPs and SIs and their scoring accuracy. We used rigorous training methods for assessment and examined specific behaviors amenable to individual teaching intervention. We were able to compare performance in the skills for effective use of medical interpreters, with generic communication skills using the PPI. We did not introduce any new curricula for the skill set during the study. This study had some limitations. First, we had a small number of participants and used aggregate data from four different classes during the timeframe of the study, to maximize our statistical power. Pre/post data were available for only one cohort of 10 FNP students. The impact of student Spanish fluency on performance in the station was not assessed in this study, and may have been a confounding factor, making it harder for the partially proficient student to perform well in the station. However, from direct observation of the DVDs, we found no evidence that any student spoke in Spanish during the encounters. The behavior scales had been validated in medical students and may not apply to FNP students, but we have found no study or report that suggests that skills for working effectively with interpreters should be different among these different health professional groups. As well, our results are based on standardized encounters in a high stakes examination rather than actual clinical encounters. Lastly, the data are limited to Spanish speaking monolingual patients only and generalization to other languages cannot be made. Implications Program-based implications As a result of our findings, we plan to enhance current didactic teaching methods and learning strategies with interactive learning tools and workshops. Additionally, faculty and SPs will provide feedback to students at their baseline skills examination. We will require that students view and critique their own taped performance with faculty, to identify learning deficits and strengths for improvement. Our long-term goal is to demonstrate an improvement in these skills with training using the same scales. Of greatest importance, we hope to demonstrate that these skills for working effectively with interpreters ultimately translate into more effective patient care outcomes, including adherence to clinician advice, patient satisfaction, and improved quality of care. This study presents a small but much needed first step to link health professions education and quality of care, on the continuum from knowledge to competency and proficiency, to improved patient outcomes. General implications Our study has implications for NP education in general. Needs assessment is the basis of curriculum development. In the topic area of cultural competence, NP programs should develop skill-specific measures to identify their trainees learning needs before introducing new curricula. Our findings in learning to work effectively with interpreters represent one example of this recommended approach. Learning outcomes should then be tracked using validated tools to assure that learning has indeed occurred, and the efficacy of teaching methods should be continually assessed through trainee performance and evaluations. 237

Effective use of interpreters S.J. Phillips et al. Acknowledgments This project was supported in part by a grant from the National Institutes of Health (NIH), National Heart, Lung and Blood Institute, award number K07 HL079256 01, RFA-HL-04 012, (2004 2009) and from the Association of American Medical Colleges (AAMC) grant initiative Enhancing Cultural Competence in Medical Schools (2005 2008) supported by the California Endowment. The contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH or AAMC. The Student Skills Training Center at the UCI School of Medicine provided invaluable support and expertise in data collection and case implementation. References American Nurses Association. (2004). Nursing: Scope and standards of practice (1st ed.). Silver Spring, MD: The Publishing Program of ANA. Civil Rights Act of 1964, July 2. (1964). (Document Number PL 88 352), 88th Congress, H. R. 7152. Retrieved April 3, 2008, from http://usinfo.state. gov/usa/infousa/laws/majorlaw/civilr19.htm Flores, G. (2005). The impact of medical interpreter services on the quality of health care: A systematic review. Medical Care Research Review, 62, 255 299. Grubbs, V., Chen, A. H., Bindman, A. B., Vittinghoff, E., & Fernandez, A. (2006). Effect of awareness of language law on language access in the health care setting. Journal of General Internal Medicine, 21, 683 688. Harden, R. M. & Gleeson, F. A. (1979). Assessment of clinical competence using an objective structured clinical examination (OSCE). Medical Education, 13(1), 41 54. Hayakawa, S. I. (1982). English as our official language: Why official English? Retrieved April 3, 2008, from http://www.proenglish.org/issues/ offeng/index.html Karliner, L. S., Perez-Stable, E. J., & Gildengorin, G. (2004). The language divide: The importance of training in the use of interpreters for outpatient practice. Journal of General Internal medicine, 19, 175 183. Lie, D., Bereknyei, S., Braddock, C., Encinas, J., Ahearn, S., & Boker, J. (2009). Assessing medical students skills in working with interpreters during patient encounters: A validation study of the interpreter scale. Academic Medicine, 84(5), 643 650. Lie, D., Boker, J., Bereknyei, S., Ahearn, S., Fesko, C., & Lenahan, P. (2007). Validating measures of third year medical students use of interpreters by standardized patients and faculty observers. Journal of General Internal Medicine, 22(Suppl. 2), 336 340. Makoul G. (2001). Essential elements of communication in medical encounters: The Kalamazoo consensus statement. Academic Medicine, 76, 390 393. U. S. Census Bureau. (2000). Language use and English-speaking ability, 1 3. Retrieved April 3, 2008, from http://www.census.gov/prod/2003 pubs/c2kbr-29.pdf U.S. Census Bureau. (2004). Percent of people 5 years and over who speak English less than very well. Retrieved April 3, 2008, from http://factfinder.census. gov/home/saff/main.html? lang=en& ts U.S. Department of Health and Human Services, Office of Minority Health. (2000). National standards of cultural and linguistically appropriate services in health care. Retrieved April 3, 2008, from http://www.omhrc.gov/ templates/content.aspx?id=87 U.S. Department of Health and Human Services. (2002). Nurse practitioner primary care competencies in specialty areas. (HRSA 00 0532P). Rockville, MD: National Organization of Nurse Practitioner Faculties and American Association of Colleges of Nursing. U.S. Government Accountability Office. (2008). Primary care professionals: Recent supply trends, projections, and valuation of services. Testimony by Steinwald, A.B. to Committee on Health, Education, Labor, and Pensions, U.S. Senate. Washington, DC: Author. (GAO Publication No. 08 472T, pp. 1 18). Van Der Vleuten, C. P. & Swanson, D. B. (1990). Assessment of clinical skills with standardized patients: The state of the art. Teaching and Learning in Medicine, 2, 58 76. 238