BILINGUAL SCHOOL PSYCHOLOGISTS ASSESSMENT PRACTICES WITH ENGLISH LANGUAGE LEARNERS

Transcription

1 , Vol. 47(10), 2010 View this article online at wileyonlinelibrary.com C 2010 Wiley Periodicals, Inc BILINGUAL SCHOOL PSYCHOLOGISTS ASSESSMENT PRACTICES WITH ENGLISH LANGUAGE LEARNERS ELISABETH C. O BRYON AND MARGARET R. ROGERS University of Rhode Island This study explored bilingual school psychologists assessment practices with students identified as English language learners (ELL). One thousand bilingual National Association of School Psychologist members were recruited nationwide, and 276 participated. Among those conducting language proficiency assessments of ELLs, many (58%) use comprehensive methods across four domains of language use. Participants generally use multifaceted assessment approaches and most (84.1%) assess acculturation as part of the assessment battery. Although few use interpreters, those who did reported a mix of best practices and discouraged practices in their use. Multiple regression results showed that engaging in continuing education activities about assessing ELLs predicted best practices when assessing language proficiency, selecting and using measures with ELLs with whom they share a second language, and using interpreters. A relationship was also found between engaging in applied preservice training with bilingual supervision and best practice behaviors assessing student acculturation. C 2010 Wiley Periodicals, Inc. U.S. demographic data reveal that the number of school-age youngsters identified as English language learners (ELLs) is rising and currently comprises 20% of the overall student body (National Center for Education Statistics, 2009). Between 1979 and 2007, the percentage of school-age ELLs increased by 140%, reflecting one of the fastest growing subgroups attending our nation s schools. Looking at the representation of ELLs across the country, the West is the most concentrated (34%), followed by the Northeast (20%), the South (17%), and the Midwest (11%). Across the nation, school psychologists are working with ELLs with varied but increasing frequency. Students speak more than 400 languages, with Latino youth who speak Spanish as their native language representing approximately 80% of ELLs (U.S. Department of Education, 2008a). Following Spanish, Vietnamese, and Hmong are the languages most frequently spoken (Kindler, 2002). In the schools, indices of successful outcomes for ELLs are not consistently documented. For Latino youth in particular, the research has shown distressing patterns. Latinos have the highest school dropout rates of any racial or ethnic group (U.S. Department of Education, 2008b), show elevated rates of grade retention (National Association of School Psychologists, 2003), and, when they are ELLs, are more likely than English-speaking youngsters to struggle academically (U.S. Department of Education, 2008b). When working with ELLs who are referred for a psychoeducational assessment, it is difficult to isolate and disentangle what are the effects of the language acquisition process, from poor quality educational services, from what may be academic difficulties that result from processing disorders, attention problems, and learning disabilities (Klingner & Harry, 2006). The psychoeducational assessment process when working with ELLs presents several challenges. When a student is referred, best practice dictates that one of the earliest steps is to establish his or her language proficiency in both languages (American Educational Research Association [AERA], American Psychological Association, & National Council on Measurement in Education, 1999). In an excellent guide to assessing ELLs, Rhodes, Ochoa, and Ortiz (2005) recommend that language proficiency assessments include formal and informal measures. Informal methods are needed as formal measures may be normed on samples not representative of the student, may We extend our appreciation to Ginette Ferszt, JoAnn Hammadou Sullivan, and Joseph S. Rossi for their helpful input on an earlier version of this article. Portions of the study were presented at the 2010 NASP conference in Chicago, IL, and the 2010 APA conference in San Diego, CA. Correspondence to: Margaret R. Rogers, Psychology Department, Chafee Building, University of Rhode Island, Kingston, RI mrogers@uri.edu 1018

2 English Language Learners 1019 not consider dialectal variations, and often assume that language is used similarly across distinct languages (Rhodes et al., 2005). Informal methods include observations, questionnaires, teacher rating scales, storytelling, cloze techniques, and language samples (Rhodes et al., 2005). Unfortunately, little is known about how ELLs language proficiency assessments are carried out and whether they rely on formal and informal methods. The literature is also unclear about who is responsible among school personnel for conducting language proficiency evaluations. Bilingual assessors, ELL specialists, speech language pathologists, and school psychologists may assess the language skills of an ELL, but neither research nor the major professional organizations provide guidance about who has the primary responsibility to do so when an ELL is referred (Klingner & Harry, 2006; Roseberry-McKibbin, Brice, & O Hanlon, 2005). In the absence of any clear professional leadership or ownership, there is often confusion about who is responsible. One purpose of the present study was to determine how language proficiency is assessed (i.e., through formal and/or informal measures) and who is responsible for performing those assessments. AsecondandrelatedchallengewhenassessingELLsconcernsthelanguageskillsneededby the school psychologist. Bainter and Tollefson (2003) studied the acceptability of various methods used to assess ELLs and found that using a bilingual school psychologist was the most acceptable practice. Using a bilingual school psychologist who is well trained to perform assessments with ELLs has long been considered to be best practice (AERA et al., 1999; Figueroa, 1990; Lopez, 1995). Yet there is a severe shortage of bilingual school psychologists in the United States as they make up approximately 10.8% of all school psychologists (Curtis et al., 2008). A further complication is the large number of second languages that ELLs speak and the need to match the second language between the practitioner and the student. Conventional wisdom also suggests that bilingual school psychologists need to be fluent in both languages, but there is debate about what constitutes fluency. When an ELL is referred, finding a bilingual practitioner who is fluent and is matched to the second language of the student with the skills needed to establish the ELL s language proficiency as well as conduct the remainder of the assessment can be difficult. The training received by bilingual school psychologists in the use of their second language in professional contexts is also an issue. Few school psychology programs offer training specifically aimed at preparing practitioners to deliver services to bilingual students. Only a handful of states have bilingual certification requirements that must be met to deliver services to ELLs. In addition, 80% or more of the school psychologists practicing in the eight states in the United States with the greatest concentration of Latino ELLs reported that their training in second language acquisition, methods to conduct bilingual assessments, and how to interpret their results was less than adequate (Ochoa, Rivera, & Ford, 1997). Preparation for performing bilingual assessments can have many components, including relevant coursework, applied training, supervision by a bilingual supervisor, and participation in relevant continuing education, and can occur during and/or after graduate preparation. Although advancements in training have likely occurred since the study by Ochoa and colleagues was published, little is presently known about bilingual practitioners preparation. The present study sought to learn about the preservice and continuing education experiences of bilingual school psychologists concerning the assessment of ELLs to see if they predict use of best practice methods when assessing ELLs. Athirdchallengefacedwhenperformingpsychoeducationalassessmentsisselectingmeasures appropriate for use with ELLs (Fernandez, Boccaccini, & Noland, 2007). It is often difficult to find instruments with norms representative of an ELL examinee. Certain assessment tools may be inappropriate if the student has not had adequate test-taking experience, which may be the case for those who have recently entered American schools (Lopez, 1995). Using instruments not designed

3 1020 O Bryon and Rogers for use with ELLs raises issues surrounding the construct equivalence, functional equivalence, and translation equivalence of the test (AERA et al., 1999). Historically, the options available have included using translations of widely used measures, using nonverbal tests, adjusting scores, using foreign-normed tests, and administering measures in English options that have all proved problematic (Figueroa, 1990). The problem with many techniques is that they rely on measures not standardized in the language of the examinee or with individuals not representative of the examinee (Sattler, 2001). Some best practices include using measures with representative samples, using informal and formal assessments, verifying the psychometric properties of the measure, considering the child s test-taking experiences, and noting deviations in standardized test administration during interpretation and report writing (Rhodes et al., 2005). It is important to understand the factors that guide decision making in the selection and use of measures with ELLs, yet little is presently known about the factors that school psychologists use. While no research suggests that bilingual school psychologists are more or less likely to consider using best practices whether they share or do not share the student s second language, the present study examines this issue. To date, previous research has identified specific assessment instruments that bilingual school psychologists (Lopes, 2005) and monolingual school psychologists (Ochoa, Riccio, Jimenez, Garcia de Alba, & Sines, 2004) use with ELLs. The present study will instead identify factors, based on best practices, that bilingual school psychologists consider when selecting assessment instruments with ELLs with whom they share and do not share a second language. An important part of the assessment process with ELLs is understanding their level of acculturation. Assessing acculturation helps identify cultural experiences the student has had that may impact their frame of mind and assessment responses (Acevedo-Polakovich et al., 2007). This assessment provides a context for understanding school behavior and test performance (Rhodes et al., 2005). Rhodes and colleagues (2005) recommend using interviews and formal assessment instruments to assess acculturation. To date, little is known about how school psychologists assess acculturation, although experts (e.g., Rhodes et al., 2005; Sattler, 2001) agree on its importance. In the only previously published study addressing the use of acculturation assessments (Ochoa et al., 2004), the mostly monolingual sample used a single instrument, the Acculturation Rating Scale for Mexican Americans. Little else is known about methods presently used to assess acculturation, and no studies report how bilingual school psychologists perform such assessments. The present study explored whether and how bilingual school psychologists assess acculturation as part of psychoeducational assessments of ELLs. Another challenge that school psychologists face when assessing ELLs concerns the use of interpreters. Given the wide array of second languages spoken, bilingual school psychologists may seek out interpreters when students speak a second language they do not speak. Although interpreters may be needed when language matches are not possible, their use raises several issues. First, interpreters may delete or add information that the examiner did not wish to convey (Lopez, 2002). Second, interpreters may not be able to translate concepts that lack equivalence in the examinee s language (Sattler, 2001). Third, the difficulty level of words may not translate to the same difficulty level in another language (Lopez, 2002). Fourth, to maintain objectivity, the interpreter must not have a prior personal relationship with the examinee, which may prove difficult in communities where few people speak a language (AERA et al., 1999). Fifth, interpreters may not be adequately familiar with standardization procedures for instruments and with the assessment process in general (AERA et al., 1999). In addition, it is important that school psychologists consider their own training in the use of interpreters. Ochoa and colleagues (2004) found that 78% of school psychologists they studied used interpreters, whereas only 52% had training in the appropriate use of interpreters. Anecdotal evidence also suggests that it is often difficult to locate well-trained interpreters. Several scholars

4 English Language Learners 1021 and major professional organizations, noting the challenges associated with interpreter use, have made recommendations for the selection and appropriate use of interpreters (AERA et al., 1999; Lopez, 2002; Rhodes et al., 2005). The recommendations for selecting interpreters include screening them by evaluating their proficiency in English and the native language of the student being assessed, their understanding of the legal requirements of nonbiased assessment, and their familiarity with education, special education, ethics, and the referred student s cultural and linguistic background (AERA et al., 1999). Recommendations regarding working with interpreters include briefing them prior to the assessment, practicing with them, speaking slowly and clearly, and debriefing (Rhodes et al., 2005). Although the literature has identified difficulties that may arise when using interpreters and suggestions for addressing those problems, no published studies have shown how bilingual school psychologists use interpreters during assessments. The present study thus examined bilingual school psychologists backgrounds in the use of interpreters, how they find interpreters, how interpreters skills are screened, what tasks interpreters are asked to accomplish (i.e., translate permission forms, etc.), and the problems encountered working with them. To clarify our understanding of the assessment practices used with ELL students, the study had two aims. One was to explore bilingual school psychologists experiences with language proficiency assessments, the factors they consider when selecting assessment instruments to use with ELLs with whom they do and do not share a second language, their experiences assessing acculturation, and their access to, selection of, use of, and problems encountered when working with interpreters. A second purpose was to explore the relationship between bilingual school psychologists education and training (i.e., preservice coursework, applied training/bilingual supervision, and continuing education) focused on assessing ELLs and their use of best practice assessment methods. Specifically, we wanted to determine if their training background predicted their use of best practice methods (i.e., when conducting language proficiency assessments, considering factors that guide the selection of assessment instruments when they share and when they do not share a second language, performing acculturation assessments, and using interpreters) during assessments with ELLs. Participants METHOD One thousand bilingual school psychologists were randomly sampled from the NASP membership directory, and, of those, 369 (37%) responded. Although the 369 self-identified as bilingual according to NASP membership records, 276 (28%) reported speaking a language other than English fluently and were therefore included in all analyses. Most were women (n = 221; 80.1%) and identified as either White/Caucasian (55.8%) or Latina/o (32.6%). Fewer identified as Asian American (6.9%), Black/African American (0.4%), Native American Indian (0.4%), or Other (2.5%). In comparison to the general membership of NASP, the present sample was bilingual (100% vs. an estimated 10.8% of NASP members), with a slightly higher representation of women (80.1% vs. 74% of NASP members), and included six times as many school psychologists of color (44.2% vs. 7.2% of NASP members) (Curtis et al., 2008). Most (71%) practice in a public school, with the remainder in private schools, private practice, at hospitals, the state department, or in university settings. Most had a master s +30 or specialist degree (57.6%), 33.7% a doctoral degree, and 8.7% a master s degree. Nearly all (94%) work with elementary- and secondary-age children, with few serving 0- to 5-year-olds or students older than 18. They worked in 40 U.S. states and the District of Columbia, in urban (36.2%), suburban (30.8%), rural (7.6%), or some combination (22.1%) of settings, with 3.3% not responding. The greatest

5 1022 O Bryon and Rogers concentration came from California (14%), New York (9%), and Florida (8%). Most (75%) work in a state that does not require bilingual certification to deliver services to ELLs, and 79% did not attend a preparation program specializing in training bilingual school psychologists. Instrumentation This study was part of a larger investigation representing the first author s master s thesis research and contained 43 questions designed to represent important issues and themes found in an extensive literature review about psychologists and school psychologists assessment practices with ELL clients (e.g., AERA et al., 1999; Figueroa, 1990; Lopez, 1995, 2002; Ochoa et al., 1997, 2004; Paredes Scribner, 2002; Rhodes et al., 2005; Sattler, 2001). Two individuals with expertise in second language issues and bilingual assessment reviewed the items for clarity and relevance. Based on their feedback, items were modified, deleted, or retained. The final survey contained six sections and included checklist, fill-in-the-blank, Likert-scale, yes/no, and open-ended items. A copy of the survey questions is available upon request from the first author. The first section had 10 background questions including gender, race/ethnicity, highest degree earned, languages spoken fluently, language(s) spoken by clients, employment setting, state practicing in, whether their state requires bilingual certification to deliver services to ELLs, geographic setting, and the age groups worked with. The second section had 10 questions about education and training in the assessment of ELLs. Participants indicated whether their graduate program specialized in training bilingual school psychologists and their educational experiences addressing ELLs in preservice courses and continuing education. The question In your training program, how many courses did you take devoted to the psychoeducational assessment of ELLs? served as a measure of coursework. Two questions asked about continuing education experiences and continuing education hours, and were combined into a measure of continuing education. When combining questions that did not share a common response format, they were converted to z scores to allow for comparable units of measurement for subsequent analyses. Two fill-in-the-blank questions regarding applied training asked about percentage of cases during practica and internship devoted to assessing ELLs. Four questions addressed bilingual supervision. Participants indicated how many settings in which they received bilingual supervision (i.e., practica, internship, work) and how many hours of supervision they received in each setting. Participants rated the quality of supervision they received about ELL assessments on a 5-point Likert scale (i.e., 1 = poor quality, 5 = high quality). They also indicated if they found their supervisors well informed about (a) laws, (b) ethical standards, (c) professional guidelines, and (d) cutting-edge assessment approaches with ELLs. The six questions about applied training and bilingual supervision were combined into a measure of applied training/bilingual supervision. The third section asked seven questions about language proficiency assessment practices. These questions asked what participants knew about discriminating between academic difficulties due to second language acquisition issues and those due to learning difficulties, who was primarily responsible for conducting language assessments, and the formal and informal methods used to collect language proficiency information. Two items asked how (a) knowledgeable, and (b) comfortable the participants felt performing such assessments using 5-point Likert scales (i.e., 1 = not at all, 5 = very). The seventh question asked, When conducting a language proficiency assessment with an ELL student, I acquire information regarding the student s..., with (a) expressive and receptive language; (b) oral, written, and reading skills; (c) language use in home; and (d) language use in academic and interpersonal contexts, as possible answers. Responses to this latter question were combined as a measure of language proficiency assessment practices.

6 English Language Learners 1023 The fourth section contained four questions about selection and use of assessment methods when the school psychologist shares and does not share a second language with the ELL. The questions asked the participants to indicate what best practice recommendations they consider when selecting an instrument (i.e., psychometric properties, research about the tool, standardization sample, child s test-taking experience, and any other factors) and the type of instruments used (e.g., curriculumbased, norm-referenced using norms representative of the student, norm-referenced instruments with standard norms, nonverbal, dynamic, criterion-referenced, and other ). These items were combined into two composite measures named factors considered when selecting instruments with ELLs who share a second language and factors considered when selecting instruments with ELLs who do not share a second language. The fifth section contained two questions about acculturation, the first about whether they assessed acculturation during an assessment and the second about how they do so (e.g., interviews, questionnaires, or other methods). The latter question was used for a measure named acculturation assessment practices. Finally, the sixth section asked 10 questions about using interpreters during assessments. Participants reported the percentage of cases in which they used interpreters during their training and while employed, how they were used (i.e., to translate permission forms, communicate with parents, perform simultaneous and consecutive translations), and the languages that interpreters used that they do not speak. They were asked if they use best practices when screening interpreters (i.e., evaluating the interpreter s proficiency in English and the student s native language; evaluating their knowledge of education, special education, ethics, the students cultural/linguistic background, and the legal requirements of nonbiased assessment) and while working with them (i.e., briefing interpreters prior to assessment, practicing with them, speaking slowly and clearly to ensure proper interpretation, and debriefing). The latter two questions were combined into a measure of practices in selecting and using interpreters. Participants were asked whether they used interpreters in ways discouraged in the professional literature (i.e., using untrained interpreters, friends, and/or family to interpret, without reporting their use, without receiving training in their use, to translate standardized measures, to perform on-the-spot translations of measures, to evaluate students language proficiency). They were also asked to rate on a 5-point Likert scale how comfortable they felt using interpreters (1 = not at all, 5 = very), how they located them, and what problems they encountered in their use. Three open-ended items were analyzed qualitatively using manifest content analysis (Leech & Onwuegbuzie, 2008). The items asked how academic difficulties due to second language acquisition issues were distinguished from those due to learning difficulties, how interpreters were located, and what the problems encountered in interpreter use were. Two researchers analyzed observable content and independently identified categories that emerged. Categories were agreed upon, and items were coded by category by each rater independently. Inter-rater agreement was calculated using the following formula: percentage agreement (number of agreements/number of agreement + number of disagreements). The average inter-rater agreement between the coders was 92.6% across the three items (i.e., 90.4%, 93.4%, and 94%). Procedure Dillman s (2000) Tailored Design method was used to recruit participants and included an initial contact (containing a cover letter, informed consent form, and the survey), a follow-up postcard that served either as a thank you note or a reminder, and (4 weeks after the initial contact for nonrespondents) a third contact containing a replacement survey. The University Institutional Review Board reviewed and approved the study methods and procedures.

7 1024 O Bryon and Rogers Data Analysis One of the two major aims of the study was to examine the relationships of education and training with the use of best practice approaches when assessing ELLs. Standard multiple regression analyses were performed with three independent variables (coursework, applied training/bilingual supervision, and continuing education) and each of the five dependent variables (language proficiency assessment practices, factors considered when selecting instruments with ELLs when sharing asecondlanguage,factorsconsideredwhenselectinginstrumentswithellswhennotsharingasecond language, acculturation assessment practices, and practices in screening and using interpreters). Survey items were combined as noted to represent the independent and dependent variables. If they did not share a common response format, they were converted to z scores to allow for comparable units of measurement. RESULTS What Are the Current Assessment Practices of Bilingual School Psychologists With ELL Students? Language Proficiency Assessment Practices. Participants reported speaking 24 languages other than English fluently, with 51.8% identifying Spanish as their second language. American Sign Language was second most common (10.1%), followed by French (5.4%), and several other second languages (21.4%; e.g., a Chinese dialect, etc.), including those who spoke more than two languages (10.5%). Most (51.4%) provide services to clients who speak only one language other than English, and 43% of clients spoke Spanish. On average, participants indicated that they had above average knowledge regarding secondlanguage acquisition (mean [M] = 4.03, standard deviation [SD] = 0.92) and were, on average, somewhat comfortable assessing an ELL s language proficiency (M = 3.63, SD = 1.32). Various school-based personnel were identified as being primarily responsible for conducting ELLs language proficiency assessments. Thirty-one percent indicated that the school s ELL specialist was responsible, 20.3% identified two or more professionals, 14.9% identified the speech language pathologist, 14.9% indicated they were, and 14.1% identified other professionals. When they perform them, they assess native and second language use in the ELL s home (72%), native and second language skills in interpersonal and academic settings (69%), expressive and receptive English language skills (68%), and oral, written, and/or reading skills in the ELL s native and second language (63%). On average, school psychologists assessed 3.7 of the aforementioned areas, and many (58%) acquired information in all four, suggesting that this subset engaged in a comprehensive approach to determining students first and second language proficiencies. Nearly 23% do not perform language proficiency assessments. Participants reported their use of formal and informal language proficiency assessment methods. Many (57.6%) use formal instruments, and, among those who offered examples, 25.7% used the Woodcock Muñoz Language Survey, 19.6% the Bilingual Verbal Ability Test, 9.1% the Woodcock Johnson III, and 6.9% the Batería III Woodcock-Muñoz. Most (83.3%) use informal methods. Among those offering examples, parent interviews were used most frequently (44.2%), followed by student interviews (43.1%), student observations (33.3%), teacher interviews (32.9%), review of records (9.1%), language samples (8.7%), work samples (7.9%), and checklists or questionnaires (6.5%). Participants indicated how they discriminate between academic difficulties due to second language acquisition issues and those due to learning difficulties when working with ELLs. Sample responses appear in Table 1. Besides those who were nonresponsive, the most frequent responses

8 English Language Learners 1025 Table 1 Qualitative Analysis of What do you know about how to discriminate between academic difficulties due to second language acquisition issues and academic difficulties due to learning issues? Category Samples n % 1. Look at skills in native and second language, distinguish between BICS and CALP, use language proficiency results 2. Review background, review file, review past educational experiences, check academic assessment results (e.g., CBM) Examine students language ability in English and their native language ; It takes 5 7 years to become proficient in another language and this affects reading and writing in that language Measure progress of academic skill acquisition over time compared to other ELLs and other students in general ; I check instructional match to rule out academic issues Nonresponsive I am quite knowledgeable, I teach it Consult with relevant professionals Co-evaluate with the speech language pathologist to rule out expressive/receptive language disorder Notes. N = 228. Inter-rater agreement = 90.4%. BICS: Basic Interpersonal Communication Skills; CALP: Cognitive Academic Language Proficiency; CBM: curriculum-based measurement. Percentages do not add to 100% due to multiple responses. included examining the results of language assessments in both languages (54.8%) and reviewing educational records, including academic assessment results (42.5%). Factors Considered in the Selection and use of Assessment Measures and Types Employed. Table 2 contains a breakdown of factors considered when determining the assessment measures to select and the types of measures used with shared language ELLs and with nonshared language ELLs. When working with shared language ELLs, they considered an average of 3.21 factors (SD = 1.15). When working with nonshared language ELLs, they considered an average of 3.10 factors Table 2 Factors Considered When Selecting an Assessment Instrument: When Sharing and Not Sharing a Second Language Factors Considered During Instrument Selection Process n % With ELLs with whom participant shares a language Standardization sample Psychometric properties of the measure Research conducted on the measure The child s test-taking experience Other a With ELLs with whom participant does not share a language Standardization sample Psychometric properties of the measure Research conducted on the measure The child s test-taking experience Other b a Sample responses included: consider reports from teachers, the child s history of instruction in native language, reason for referral. b Sample responses included: consider the student s language proficiency, conduct a file review, consider the availability of a translator.

9 1026 O Bryon and Rogers Table 3 Major Categories of Assessment Instruments Used with ELL Students Assessment Instruments Used n % With ELLs with whom participants share a language Nonverbal assessment instruments Norm-referenced instruments, using the standardized norms Norm-referenced instruments, using norms representative of the student Curriculum-based assessment Criterion-referenced assessment Dynamic assessment Other With ELLs with whom participants do not share a language Nonverbal assessment instruments Curriculum-based assessment Norm-referenced instruments, using standardized norms Norm-referenced instruments, using norms representative of the student Criterion-referenced assessment Dynamic assessment Other Note. N = 276. (SD = 1.19). A paired-samples t test did not detect statistical significance between the number of factors considered with shared language ELLs (M = 3.21, SD = 1.15) and with nonshared language ELLs (M = 3.10, SD = 1.19), t(207) = 1.46, p =.15 (two-tailed). Participants indicated what types of assessment measures (e.g., curriculum-based, etc.) they used with shared language ELLs and with nonshared language ELLs. Table 3 presents the breakdown of categories of measures. On average, they use 3.74 (SD = 1.47) different measures when working with shared language ELLs. When working with nonshared language students, they used an average of 3.28 (SD = 1.58) measures. A paired-samples t test found statistical significance between the number of measures used with shared language ELLs (M = 3.77, SD = 1.49) and with nonshared language ELLs (M = 3.31, SD = 1.59), t(201) = 5.75, p<.0005 (two-tailed). The eta squared statistic (.14) indicated a large effect size, suggesting that they report using more instruments with shared-language than with nonshared ELLs. Acculturation Assessment Practices. Most (84.1%) assess acculturation, 11.6% do not, and 4.3% did not respond. The majority (85%) use interviews to do so, 41.3% questionnaires, 32.9% other instruments (e.g., observations, record review), and 55% use multiple methods. Working with Interpreters. Participants provided information regarding their use of interpreters during assessments with ELLs. On average, during graduate preparation, interpreters were used in 5% of assessment cases. While employed, they used them in an average of 14% of cases, and 26.8% did not use them at all. Participants indicated how comfortable on a 5-point scale (not at all = 1, very = 5) they feel using interpreters. On average, they were somewhat comfortable (M = 3.24, SD = 1.16). Interpreters spoke 58 languages other than English, including Spanish (31.1%), Vietnamese (23.9%), and Cantonese (14.1%). Table 4 shows the context in which they used interpreters, with 66.3% using them to communicate with parents and 54.4% to perform consecutive translations. Table 4 also presents what participants revealed concerning practices discouraged in the professional literature, with 32.9% using untrained interpreters and a disquieting 31.8% using them to establish children s language

10 Table 4 Interpreter Uses: Encouraged Activities and Discouraged Activities English Language Learners 1027 Use of Interpreters n % Encouraged activities Communicating with parents Performing consecutive translations Translating permission forms for testing Performing simultaneous translations Discouraged activities Using an untrained interpreter Evaluating an ELL student s native language linguistic skills Using friends and/or family to interpret Translating standardized measures of intelligence, adaptive behavior, academic achievement, and behavior Using an interpreter without having received training in the use of interpreters Performing on-the-spot translations of published assessments Using an interpreter without highlighting their use in a psychological report Note: n = 276. proficiency. Table 5 shows participants professional practices when using an interpreter s services. Most common were speaking slowly and clearly to ensure proper interpretation (56.5%) and thoroughly briefing the interpreter prior to an assessment (52.5%). Participants reported how they find interpreters and the problems they experience with them. They were found through a district-provided list, by the school staff, or by a hospital (71.9%), an outside agency (18.4%), family or a religious group (5.3%), and colleagues or university contacts (4.9%). Inter-rater agreement for coding this question was 93.4%. The most common problems noted in using interpreters were that they did not perform translations verbatim or lacked knowledge of professional terms (69.7%), were hard to find (17.5%), and were not trained (15.9%). Inter-rater agreement for this question was 94%. Table 5 Professional Practices Employed When Using Interpreters Professional Practices n % Speaking slowly and clearly to ensure proper interpretation during the assessment Thoroughly briefing the interpreter prior to the assessment Evaluating the interpreter s proficiency in both English and the native language of the student being assessed Evaluating the interpreter s familiarity with the student s cultural and/or linguistic background Evaluating the interpreter s familiarity with the fields of education and special education Evaluating the interpreter s understanding of the ethical guidelines surrounding the assessment of ELLs Evaluating the interpreter s understanding of the legal requirements of nonbiased assessment Reviewing the interpreter s performance after the assessment Practicing with the interpreter Note. n = 276.

11 1028 O Bryon and Rogers Table 6 Summary of Multiple Regression Analysis Predicting Best Practice Approaches in Assessing ELL Language Proficiency Independent Variable B SE B β p sr 2 1. Coursework < Continuing education Applied training and bilingual supervision Notes. B: unstandardizedregressioncoefficient;se B: standard error of B; β: standardizedregressioncoefficient;sr 2 : semipartial correlation. F (3, 123) = 5.44, p =.002. R 2 =.12. p<.05, p =.052. Does Training Predict Use of Best Practice Approaches in the Psychoeducational Assessment of ELL Students? Participants took an average of one preservice course (SD = 1.5, range 0 7) about assessing ELLs and reported that an average of 25.5% (SD =.30) of practica cases and 32.4% (SD =.32) of internship cases involved ELLs. Although most (57.6%) have not worked with a bilingual supervisor, those who did rated the quality as satisfactory (M = 3.17, SD = 1.32). They engaged in an average of 3.2 continuing education efforts about assessing ELLs (SD = 2.4, range = 0 11) and, over the last 5-year period, devoted an average of almost 30 hours to such efforts. To analyze if education and training experiences predict the use of best practice approaches in the assessment of ELLs, standard multiple regression analyses were performed with the three independent variables and the five dependent variables. Table 6 shows that coursework, applied training/bilingual supervision, and continuing education accounted for 12% of the variance in the language proficiency assessment practices dependent variable. When the three independent variables were entered together, the results were significant (F (3,123) = 5.44, p =.002). Only continuing education made a significant contribution to the regression, suggesting that those learning to assess language proficiency through continuing education used multiple best practice approaches to do so (i.e., acquired information about oral, written, and reading skills in native and second language, etc.). Asecondmultipleregressionshowedthatcoursework,appliedtraining/bilingualsupervision, and continuing education accounted for 9% of the variance in best practices when selecting and using assessment measures with shared language ELLs. Table 7 shows that the regression was significant (F (3, 123) = 3.98, p =.010), with continuing education making a unique contribution, suggesting that those engaging in continuing education consider several important factors when determining Table 7 Summary of Multiple Regression Analysis Predicting Best Practice Approaches in Selection and Use of Assessment Instruments with ELLs Who Share a Language with Participants Independent Variable B SE B β p sr 2 1. Coursework < Continuing education Applied training and bilingual supervision <.001 Notes. B: unstandardizedregressioncoefficient;se B: standard error of B; β: standardizedregressioncoefficient;sr 2 : semipartial correlation. F (3, 123) = 3.98, p =.010. R 2 =.09. p<.05.

12 English Language Learners 1029 Table 8 Summary of Multiple Regression Analysis Predicting Best Practice Approaches in Assessing Acculturation Independent Variable B SE B β p sr 2 1. Coursework Continuing education Applied training and bilingual supervision Notes. B: unstandardizedregressioncoefficient;se B: standard error of B; β: standardizedregressioncoefficient;sr 2 : semipartial correlation. F (3, 120) = 3.804, p =.012. R 2 =.09. p<.05. Table 9 Summary of Multiple Regression Analysis Predicting Best Practice Approaches in Screening and Using Interpreters Independent Variable B SE B β p sr 2 1. Coursework Continuing education Applied training and bilingual supervision Notes. B: unstandardizedregressioncoefficient;se B: standard error of B; β: standardizedregressioncoefficient;sr 2 : semipartial correlation. F (3, 72) = 2.85, p =.043. R 2 =.11. p<.05. which measures to use and use many measures when working with shared language ELLs. Although not reported in Table 7, a separate regression equation using the three independent variables to predict selection and use of assessment measures with nonshared language ELLs was not significant (F (3, 106) = 2.14, p =.100). Afourthmultipleregressionrevealsthatcoursework,appliedtraining/bilingualsupervision,and continuing education efforts accounted for 9% of the variance in acculturation assessment practices, and was significant (F (3, 120) = 3.804, p =.012). Table 8 shows that applied training/bilingual supervision was the only significant contributor, offering 4% unique variance. This finding suggests that those with large caseloads of ELLs during their applied preservice training who received highquality supervision from bilingual supervisors were likely to assess acculturation using a multifaceted approach. As noted in Table 9, the results of a fifth multiple regression showed a significant prediction in the use of best practices in screening and using interpreters (F (3, 72) = 2.85, p =.043), accounting for 11% of the variance. Continuing education contributed significantly to the prediction, offering 7% unique variability. This finding suggests that continuing education was predictive of using recommended methods when screening and using interpreters. DISCUSSION This study builds on previous research examining assessment practices used by mostly monolingual school psychologists when working with ELLs (e.g., Ochoa et al., 2004) by focusing exclusively on NASP members who self-identify as bilingual and fluent in a second language. Several distinctive features highlight their assessment practices. First, although not usually the primary person responsible for conducting students language proficiency assessments, when they were, they usually took

13 1030 O Bryon and Rogers amultifacetedapproachbyassessingnativeandsecondlanguagesinmultipledomainsandusing formal and informal methods. Also, the finding that only approximately 15% of bilingual practitioners take the lead in performing language proficiency assessments stands in contrast to results from earlier studies. In Ochoa et al. (2004), 78% of their sample of school psychologists, and, in Lopes (2005), 51% of their sample of school psychologists, reported engaging in language proficiency assessments. In neither of these earlier studies, however, did those sampled identify as a bilingual practitioner fluent in a second language. The present results speak directly to the practices engaged in by the small number of bilingual practitioners who are fluent in multiple languages and use those linguistic skills professionally. Our study also found that bilingual practitioners reported above average knowledge regarding the process of second-language acquisition, appropriately identifying the importance of looking at students skills in their native and second language, looking at past educational experiences and academic achievement performance, and using language proficiency results to discriminate between academic difficulties due to second language acquisition issues and those due to learning issues. Still, they reported feeling only somewhat comfortable when performing language proficiency assessments, an observation perhaps reflecting the difficulty inherent in these types of complex assessments. Teasing apart the impact that emergent language skills are having on children s academic performance when that performance is faltering is at the heart of the referral question for most ELL cases. The findings suggest that, although most bilingual school psychologists who conduct language proficiency assessments use valuable, appropriate methods to gauge students language proficiencies, they do so with a level of discomfort, likely reflecting this difficulty. Also, it is possible that at least some of this discomfort may be due to gaps in training in performing language proficiency assessments as, surprisingly, not all used recommended formal and informal measures to do so. Another finding of importance concerns the selection of and types of assessment instruments used to assess ELLs. An important addition concerns the use of acculturation assessments among bilingual school psychologists. An awareness of the degree to which the ELL is assimilated into the mainstream culture allows assessment practices to be tailored to best meet the needs of the student. We found that 84% included acculturation measures and typically used several approaches, including interviews and questionnaires, as part of a multifaceted battery when assessing ELLs. Previous research of mostly monolingual school psychologists (Ochoa et al., 2004) showed that few (2.3%) assessed acculturation of ELLs during assessments for emotional disturbance. Clearly, bilingual school psychologists are mindful of the recommendations of noted assessment scholars (e.g., Rhodes et al., 2005; Sattler, 2001) who point out that ELLs experience distinct forms of acculturative stress and thus have distinct psychological and assessment needs. The study also provides new information about bilingual school psychologists decision making when determining how to perform assessments with ELLs. Whether they shared a second language with their ELL client or did not, some bilingual practitioners are not selecting measures on the basis of best practices. At most, only approximately 80% examined standardization samples, 76% considered psychometric properties, and 67% reviewed relevant research when selecting a measure. Although regarded as high priorities, these considerations have not gained widespread acceptance among this group of practitioners, suggesting a training need. In addition, unknown are the factors considered instead of the identified best practices. The data also show that they are using a range of assessment instruments including nonverbal, norm-referenced, curriculum-based, criterion-referenced, and dynamic assessment instruments. When they shared the second language with their client, however, they used more measures than when they did not, suggesting that sharing the second language may have heightened their confidence in using a broad selection of instruments. Perhaps when sharing a second language they feel freer to gather more information about student strengths and weaknesses

14 English Language Learners 1031 because they are confident in their ability to interpret the findings in a meaningful way a confidence they do not have when they do not share the second language. When bilingual school psychologists need to work with interpreters, the study yielded several interesting findings regarding their experiences locating, screening, and using interpreters as well as problems encountered in their use. Interpreters were found most frequently through the work setting through a school district provided list, the school staff, or a hospital, but locating welltrained interpreters was a vexing issue. When school psychologists are unable to find a competent interpreter through their own school resources, calling surrounding school districts or hospitals seems a reasonable first step. Only 5% reported that they learned how to use an interpreter during their graduate preparation, and few reported collaborating with interpreters when working, a finding related to their being only somewhat comfortable in their use of interpreters. Previous research (Ochoa, Gonzalez, Galarza, & Guillemard, 1996) found that 11% of the school psychologists they sampled were trained in interpreter use, suggesting that explicit attention in how to work with interpreters may be diminishing over time in training programs. On a positive note, many report engaging in recommended practices when working with interpreters by speaking clearly, thoroughly briefing the interpreter, and evaluating his or her native and second language skills. Of particular concern, however, are the many ways that interpreters are being used that have been discouraged by experts. The present sample used untrained interpreters, family and friends to interpret, and interpreters to translate standardized assessment instruments, among other discouraged practices, all actions that compromise the services delivered to ELLs, including decisions vital to students present and future academic services and overall well-being. Such practices lead to violations in the practitioners ethics (i.e., competence, confidentiality), are among the practices discouraged by the U.S. Office of Civil Rights (U.S. Department of Health and Human Services, Office of Civil Rights, 2005), compromise quality of care (Snowden, Masland, & Guerrero, 2007), and, as pointed out by Ochoa and colleagues (2004), are at odds with the standard that addresses how to assess ELLs (i.e., Standard 9.7) of the Standards for Educational and Psychological Testing (AERA et al., 1999). Moreover, approximately one third reported using interpreters to evaluate ELLs language proficiencies in English and their native language, which, combined with the practice of using untrained interpreters, is fraught with problems. As suggested by Lopez (2002), the profession of school psychology must become active in developing a set of recommended guidelines for best practice use of interpreters and disseminate them to practitioners and trainers to eliminate these persistent detrimental practices. Interestingly, of the various training experiences in which they participated, engaging in continuing education efforts (e.g., workshops, reading in professional outlets, etc.) proved to be most important in learning best practices when performing language proficiency assessments, selecting assessment measures, and working with interpreters. Once employed, the practitioners were active participants in continuing education activities, devoting an average of 30 hours over the last 5 years to an array of efforts about assessing ELLs. Their commitment to continuing education stands out in comparison to the psychologists in a study by Hansen and colleagues (2006) who reported that 42% of psychologists they sampled rarely or never developed a plan for developing their cross-cultural competencies through continuing education activities. Participating in continuing education usually requires special planning and extra motivation, a combination of qualities that seemed to distinguish our practitioners. Given that most did not attend a graduate program specializing in service delivery to bilingual populations, it speaks volumes that they sought out professional development in these forums and demonstrated their ongoing commitment to continued skill improvement. In terms of assessing acculturation, preservice applied experiences over other training activities carried the most weight. It appears that practica and internship training that involved working with ELL clients along with bilingual supervision provides specialized knowledge about how best to

15 1032 O Bryon and Rogers assess acculturation. A recent investigation underscores the importance of culturally responsive supervision. Burkard and colleagues (2006) found that supervisors who were open about discussing cultural and racial issues had positive effects for supervisees in helping them develop needed skills and for clients in facilitating positive outcomes. Identifying supervisors who are bilingual who follow best practices is critical when learning to provide services to bilingual clients. Altogether, these results highlight a need for increased availability of applied training supervised by bilingual supervisors, and increased access to professional development experiences about best practices in serving ELLs. Limitations and Concluding Remarks One limitation concerned the identification of bilingual school psychologists fluent in a second language who use that language professionally. Although participants were identified as bilingual by the NASP membership directory and were presumed to meet the criterion as a bilingual practitioner for the study s purposes, approximately one third did not respond to questions about their fluency nor did they identify the second language they spoke fluently. This failure to self-identify as fluent in asecondlanguagecreatedquestionsabouttheirstatusandidentificationasabilingualpractitioner. How they defined fluency likely varied, and they may have used a second language in their practice yet do not consider themselves fluent in that language. Those not self-identifying as fluent had to be eliminated from subsequent analyses. This elimination significantly affected sample size, although it should be noted that the sample size still met requirements for all subsequent analyses, and was comparable to the most recent related study published about assessment practices used with ELLs (i.e., 29.3% in the Ochoa et al., 2004, study). Future studies will need to define fluency and explicitly request information regarding use of a second language in professional practices. The generalizability of the results may also be affected by the sample characteristics. Participants were NASP members, and not all practicing school psychologists are NASP members. Also, approximately 10% reported using American Sign Language as a second language. Considerations for working with students who are deaf or hard of hearing may not be the same as those from other culturally and linguistically diverse backgrounds. As such, the practices of these participants may not generalize to the behavior of other bilingual school psychologists. The format of some survey questions was a limitation. For example, questions about the factors used when making decisions about measures to use to assess ELLs might better have been written as ratings of importance of the factors rather than as checklist items, the scoring of which had to be converted into z scores to perform the needed analyses. Although survey items were vetted prior to use, some would have been improved by using other formats to provide consistent measurement of variables. Overall, these findings advance our understanding of the practices of bilingual school psychologists when delivering assessment services to ELLs, and highlight various training needs as they address and overcome language barriers when working with ELL youth. This is a time of great anticipation within school psychology as increasing numbers of ELLs enter our nation s schools and the promise exists to improve service delivery to them. Also afoot are multiple collaboration, outreach, and educational efforts that address the needs, issues, resources, and experiences of linguistically diverse students and their families. Some efforts are sponsored by NASP, including the Cultural Competence link on the Web site, and the Directory of Bilingual School Psychologists, available online on the NASP Web site. The formation of the Bilingual School Psychology Interest Group in 2008 has added new vitality within the profession, and has marshalled together the resources, wisdom, and collective expertise of a uniquely talented subset of our professional group, representing bilingual school psychologists who speak different languages. Our path is an exciting one as we

16 English Language Learners 1033 work toward excellence in our practices with ELLs to ensure that the services they access, use, and receive meet the highest professional standards. REFERENCES Acevedo-Polakovich, I. D., Reynago-Abiko, G., Garriott, P. O., Derefinko, K. J., Wimsatt, M. K., Gudonis, L. C., et al. (2007). Beyond instrument selection: Cultural considerations in the psychological assessment of U.S. Latinas/os. Professional Psychology: Research and Practice, 38, American Educational Research Association (AERA), American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association. Bainter, T. R., & Tollefson, N. (2003). Intellectual assessment of language minority students: What do school psychologists believe are acceptable practices?, 40, Burkard, A. W., Johnson, A. J., Madson, M. B., Pruitt, N. T., Contreras-Tadych, D. A., Kozlowski, J. M., et al. (2006). Supervisor cultural responsiveness and unresponsiveness in cross-cultural supervision. Journal of Counseling Psychology, 53, Curtis, M. J., Lopez, A. D., Castillo, J. M., Batsche, G. M., Minch, D., & Smith, J. C. (2008). The status of school psychology: Demographic characteristics, employment conditions, professional practices, and continuing professional development. Communiqué, 36, Dillman, D. A. (2000). Mail and internet surveys: The tailored design method. New York: John Wiley & Sons. Fernandez, K., Boccaccini, M. T., & Noland, R. M. (2007). Professionally responsible test selection for Spanish-speaking clients: A four-step approach for identifying and selecting translated tests. Professional Psychology: Research and Practice, 38, Figueroa, R. A. (1990). Best practices in the assessment of bilingual children. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology II (pp ). Washington, DC: National Association of School Psychologists. Hansen, N. D., Randazzo, K. V., Schwartz, A., Marshall, M., Kalis, D., Frazier, R., et al. (2006). Do we practice what we preach? An exploratory survey of multicultural psychotherapy competencies. Professional Psychology: Research and Practice, 37, Kindler, A. L. (2002). Survey of the states limited English proficient students and available educational programs and services summary report. Washington, DC: National Clearinghouse for English Language Acquisition and Language Instruction Educational Programs. Klingner, J. K., & Harry, B. (2006). The special education referral and decision-making process for English language learners: Child study team meetings and placement conferences. Teachers College Record, 108, Leech, N. L., & Onwuegbuzie, A. J. (2008). Qualitative data analysis: A compendium of techniques and a framework for selection for school psychology research and beyond. School Psychology Quarterly, 23, Lopes, C. J. (2005). Bilingual school psychologists practices with students with limited English proficiency. Unpublished doctoral dissertation, University of Rhode Island. Lopez, E. C. (1995). Best practices in working with bilingual students. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology III (pp ). Washington, DC: National Association of School Psychologists. Lopez, E. C. (2002). Best practices in working with school interpreters to deliver psychological services to children and families. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology IV (pp ). Washington, DC: National Association of School Psychologists. National Association of School Psychologists. (2003). Position statement on student grade retention and social promotion. Retrieved on September 27, 2008, from nasp/pospaper graderetent.aspx National Center for Education Statistics. (2009). The condition of education Retrieved on June 22, 2009, from Ochoa, S. H., Gonzalez, D., Galarza, A., & Guillemard, L. (1996). The training and use of interpreters in bilingual psychoeducational assessment: An alternative in need of study. Diagnostique, 21, Ochoa, S. H., Riccio, C., Jimenez, S., Garcia de Alba, R., & Sines, M. (2004). Psychological assessment of English language learners and/or bilingual students: An investigation of school psychologists current practices. Journal of Psychoeducational Assessment, 22, Ochoa, S. H., Rivera, B., & Ford, L. (1997). An investigation of school psychology training pertaining to bilingual psychoeducational assessment of primarily Hispanic students: Twenty-five years after Diana vs. California. Journal of School Psychology, 35, Paredes Scribner, A. (2002). Best assessment and intervention practices with second language learners. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology IV (pp ). Washington, DC: National Association of School Psychologists.

17 1034 O Bryon and Rogers Rhodes, R. L., Ochoa, S. H., & Ortiz, S. O. (2005). Assessing culturally and linguistically diverse students: A practical guide. New York: Guilford Press. Roseberry-McKibbin, C., Brice, A., & O Hanlon, L. (2005). Serving English language learners in public school settings: A national survey. Language, Speech, and Hearing Services in Schools, 36, Sattler, J. M. (2001). Assessment of children: Behavioral and clinical applications (4th ed.). Austin, TX: Pro-Ed. Snowden, L. R., Masland, M., & Guerrero, R. (2007). Federal civil rights policy and mental health treatment access for persons with limited English proficiency. American Psychologist, 62, U.S. Department of Education. (2008a). Biennial report to Congress on the implementation of the Title III state formula grant program school years Office of English Language Acquisition, Language Enhancement, and Academic Achievement for Limited English Proficient Students. Washington, DC. Retrieved July 13, 2009, from U.S. Department of Education. (2008b). The condition of education 2008 (NCES ). Table Retrieved June 23, 2009, from U.S. Department of Health and Human Services, Office of Civil Rights. (2005). Guidance to federal financial assistance recipients regarding Title VI prohibition against national origin discrimination as it affects persons with limited English proficiency. Retrieved July 20, 2009, from