Benchmarking Teacher Practice in Queensland Transition Programs for Youth With Intellectual Disability and Autism

Transcription

1 J Spec Educ OnlineFirst, published on April 7, 2010 as doi: / Benchmarking Teacher Practice in Queensland Transition Programs for Youth With Intellectual Disability and Autism The Journal of Special Education XX(X) 1 15 Hammill Institute on Disabilities 2010 Reprints and permission: sagepub.com/journalspermissions.nav DOI: / Wendi Beamish 1, Denis Meadows 1, and Michael Davies 1 Abstract Extensive work has been done in North America to examine practices recommended to facilitate postschool transitions for youth with disabilities. Few studies in Australia, however, have investigated these practices. This study drew on the Taxonomy for Transition Programming developed by Kohler to benchmark practice at government and nongovernment schools throughout Queensland, Australia. A statewide survey was used to gather data from teachers and other influential staff (N = 104). Participants were asked to (a) indicate their level of agreement that each practice was a program quality indicator and (b) report on the current use of that practice in school programs. Transition practices in the areas of Family School Relationships, Student Development, and student-focused planning were strongly endorsed and frequently implemented. Lower levels of endorsement and implementation, however, were signaled in the areas of Interagency Collaboration and Program Structure. Recommendations for reform at the school, systems, and interagency levels are provided. Keywords postschool transition, secondary transition, secondary education, high school, best practice Increasingly, teachers around the globe are being asked to show evidence of meeting standards, benchmarks, and indicators related to quality practice and practice improvement (Otis-Wilborn, Winn, Ford, & Keyes, 2000, p. 20). Today, as in the past, inquiry into best practice is making an important contribution in this area. In best practice inquiry, sets of literature-based practices are identified and are then subjected to expert opinion, professional consensus, or social validation to become practice standards or indicators of program quality. Hence, research of this kind allows individuals within a service to consider their everyday practice and evaluate their performance against predetermined practice standards (Beamish, 2008; Everitt & Hardiker, 1996). This process of benchmarking, moreover, is not viewed to be the endpoint of activity. Often, benchmarking is seen to serve as a beginning point to direct efforts (individual and collective) toward professional development, program improvement, workplace reform, and systems change. Within the educational context, this approach presumes that teacher adoption of practice standards leads to improved teaching and learning and, ultimately, to improved student outcomes. Benchmarking teachers everyday practice against identified best or ideal practice is a well-established procedure in special education (Peters & Heron, 1993; Test et al., 2009). Perhaps the ongoing need for special educators to demonstrate educational effectiveness with their difficultto-teach student population has contributed to the ongoing popularity of best practice research in the field (Beamish, 2008). Appeal also may be related to the notion that benchmarking provides opportunity for participating teachers to reflect systematically on the complexity and status of their current teaching practice. As confirmed recently by Cook, Tankersley, and Landrum (2009), teaching practice is an aspect on which special educators have focused attention because it is one determinant of student outcomes over which they have always exercised primary control (p. 365). The first set of comprehensive best practice inquiries were undertaken in the area of severe disabilities. Pioneering U.S.-based work by Atkins, Campbell, and Hartley (1987) in Mississippi and by Williams, Fox, Thousand, and Fox (1990) in Vermont was aimed to appraise the quality of public educational programs for this particular student 1 Griffith University, Nathan, Australia Corresponding Author: Wendi Beamish, Griffith University, Faculty of Education, Nathan, Q 4111, Australia w.beamish@griffith.edu.au

2 2 The Journal of Special Education XX(X) Student-Focused Planning IEP development Student participation Planning strategies The Taxonomy for Transition Programming Family Involvement Family training Family involvement Family empowerment Student Development Life skills instruction Career & vocational Curricula Structured work experience Assessment Support services Program Structure Program philosophy Program policy Strategic planning Program evaluation Resource allocation Human resource development Interagency Collaboration Collaborative framework Collaborative service delivery Figure 1. The Taxonomy for Transition Programming (Kohler, 1996). population. Both inquiries drew on a best practice listing developed by Fox et al. (1986) to conduct statewide surveys of practitioner and parent stakeholders. Both inquiries reported that moderate to high levels of participant endorsement (level of agreement) were accompanied by lower levels of actual use (level of implementation) and that a substantial gap existed between endorsement of best practice (agreement) and actual practice use (implementation). Assorted benchmarking inquiries related to the education of this student population followed in the United States (e.g., Ayres, Meyer, Erevelles, & Park-Lee, 1994; Eichinger & Downing, 1992) and in Australia (e.g., Beamish, 1992; Chalmers, Carter, Clayton, & Hook, 1998; Stephenson & Carter, 2005). All studies furnished confirmatory agreement and implementation data that highlighted the need for schools and education systems to bridge the research to practice gap. Complementary inquiries also have been undertaken in the area of transition from school to adult life for students with disabilities. Over the past 15 years, best practices have been identified in both skills needed by students and processes needed to support transition (Lubbers, Repetto, & McGorray, 2008, p. 281). One of the most comprehensive and widely accepted frameworks of quality transition practice is the Taxonomy for Transition Programming, developed by Kohler and colleagues (Kohler, 1996; Kohler & Field, 2003). According to Test, Aspel, and Everson (2006), Kohler s taxonomy is the only research-based transition model in the literature. Figure 1 presents the Taxonomy for Transition Programming, with practices organized according to five key areas indicative of successful transition-focused programming and an outcome-oriented planning process. These areas are Student-Focused Planning, Student Development, Interagency Collaboration, Family Involvement, and Program Structure. Kohler s taxonomy has provided the foundation for extensive tool development in the United States across the past decade. For example, the taxonomy was the starting point for the well-known Transition Specialist Competencies generated by the Division on Career Development and Transition (2000) and the distinguished Standards for the Preparation of Transition Specialists published by Council for Exceptional Children (2001). By comparison, Anderson et al. (2003) used a supplementary competency-oriented tool (Kohler, 2000) and the Standards for the Preparation of Transition Specialist (Council for Exceptional Children, 2001) to generate the Secondary Transition Program Survey. This instrument was subsequently used in a national survey to examine the quality of personnel preparation in transition. More recently, Wandry et al. (2008) adapted the five areas of Kohler s taxonomy when constructing a section of a tool aimed to measure preservice teachers perceived levels of instruction and competence in transition-focused education. In the United States, these and many other researchbased transition initiatives have worked in concert with federal special education regulations to improve postschool student outcomes through boosting the effectiveness of

3 Beamish et al. 3 transition services and the competencies of transition staff. Statewide survey-based inquiries into transition practice have been particularly instrumental in providing feedback on the status of transition-focused programs and in providing stimulus for systems change. For example, McMahan and Baer (2001) reported general compliance with mandated policy and regulations but not widespread use of best practices among 105 transition teams in Ohio. On a smaller scale, Thoma, Held, and Saddler (2002) found that the majority of special educators (N = 84) in Nevada and Arizona demonstrated effective knowledge about particular transition assessment practices but did not use these practices to support student self-determination. On the other hand, Zhang, Ivester, Chen, and Katsiyannis (2005) concluded from their investigation into the quality of school and district transition services throughout South Carolina that schools had done only an adequate job (p. 23) in addressing crucial transition practices. Findings from a recent inquiry by Lubbers et al. (2008) reported that teachers (n = 492) and district transition contacts (n = 41) throughout Florida identified system and policy issues and information and training as key barriers to the transition process. In the United States, extensive research efforts and substantial policy reform have progressively led to improvements in transition programs and practices. By comparison, Australian inquiries into the transition from school to adult life for youth with disabilities have been sparse. Moreover, specific policy or rigorous guidelines related to transitionfocused education for students with disabilities have not been put in place by any educational system in this country. Furthermore, no nationally mandated set of transition practices in Australia compare to those found in the Individuals With Disabilities Education Act (1990, 1997) and its amendments (Individuals With Disabilities Education Improvement Act, 2004) in the United States. Australian transition literature, in journals and postings on government and nongovernment Web sites, is primarily composed of reports. In the main, these reports (e.g., Centre for Disability Research and Development, Edith Cowan University, 1995; Hudson, Carr, Dowsett, & Moroney, 2000; Laragy, 2004) provide descriptions of transition models, program content, and some recommendations for future action. They do not feature any hard data about actual transition practice or student outcomes. One notable exception has been the New South Wales transition initiative (for a comprehensive review, see Riches, 1996). This transition project, which began as a pilot in 1989, led to several reports that described program features and identified student outcomes (Parmenter & Riches, 1990; Riches, 1996; Riches & Parmenter, 1993; Riches, Parmenter, Fegent, & Bailey, 1993; Riches, Parmenter, & Robertson, 1996; Watters, Riches, & Parmenter, 1993). These reports provided commentary about the pivotal role that both government and educational sectors played in the development of a strategic policy and plan to support effective transition practices for students with disabilities in that state. Findings, however, were not translated into government policy or mandates. Taken together, the paucity of legislative frameworks and practice guidelines throughout Australian states and territories set the agenda for a benchmarking inquiry into postschool transition practices in Queensland, Australia. This current study was part of a larger study that investigated the relationship between outcomes for students with intellectual disability, autism, and dual diagnosis and the transition practices in place throughout Queensland state, independent, or Catholic education sectors (i.e., secondary schools, secondary special education units and classes, and special schools in the sectors). The study was restricted to this specific student population because the transition practices used with these groups of students were of particular interest to the government department funding the investigation. To date, Meadows (2009) has provided an overview of the total study and major recommendation and Davies and Beamish (2009) have reported the findings of a statewide parent survey. The present study sought to benchmark the current practice of teachers and other influential staff involved in the school to postschool transition for this specific student population. Two research questions framed the study: To what extent did transition staff in Queensland agree that the transition practices drawn from Kohler s Taxonomy for Transition Programming were indicators of program quality? To what extent did staff implement these transition practices? Method Participants and Procedure As in the parent survey (Davies & Beamish, 2009), this study used a statewide survey methodology. A direct fieldbased sampling procedure was employed but varied across sectors. For the government sector, this procedure involved using an electronic listing of schools to invite the participation of staff involved in transition programs. For the nongovernment sectors (independent and Catholic education), designated officers invited their respective schools to participate. In all, 198 schools (secondary, secondary education units and classes, and special schools) were requested to provide individual addresses of interested teachers and influential staff to the research team. In all, 220 potential participants were identified through this process:

4 4 The Journal of Special Education XX(X) 90 from special education units in state secondary schools, 90 from state special schools, 20 from independent schools, and 20 from catholic schools. All participants were subsequently ed instructions on how to access an online survey and given 1 month to respond. Within 2 weeks, many participants communicated that they were experiencing technical difficulties in completing the survey online. In response, a hard copy of the survey was mailed to all participants. A 3-week time frame for completion was requested. Incoming responses were collated, and two reminders were sent to participants. A total of 104 useable surveys (online and hard copy) were received, constituting a response rate of 47.3%. This return rate compared more than favorably to similar U.S.-based statewide transition studies (e.g., Lubbers et al., 2008; Thoma et al., 2002; Zhang et al., 2005). Table 1 presents demographic information on staff who completed the present benchmark survey. Staff were evenly spread between secondary and special schools, with a majority of schools (61%) located in an urban area. Slightly more than half of the participants were male. The majority of participants (63%) were classroom teachers, and 20% were heads of special education services. School transition officers (6%) and principals (5%) composed the remainder of the sample. Almost half (44%) of the sample possessed a special education qualification at the undergraduate or postgraduate level. The modal number of years of teaching experience was 15 years, with 32% having little experience (1 3 years) in the area of transition education, 35% a moderate amount (4 8 years), and 34% a great deal of experience (9 20 years). Of the participants, 52% reported that they had had previous experience in an industry other than education. The Survey Instrument The benchmark survey comprised 8 demographic items (see Table 1) and 46 practice items. Practice item statements were generated from a range of material developed by Kohler (2002, 2003a, 2003b) for staff reflection on practices recommended in her taxonomy. Items were clustered according to the taxonomy s five practice areas. Items were then refined and field tested using a three-step process. First, the research team considered the statements in terms of perceived levels of importance, measurability, and relevance to the Queensland context. Second, a highly experienced school transition officer reviewed the practice items and made several editorial suggestions. Third, a group of 10 experienced transition teachers from the state and Catholic school sectors reviewed the practices, item by item. This process resulted in further editorial revisions to the wording of particular practice items and the labeling of one practice area (i.e., Family Involvement was renamed Family School Table 1. Characteristics of the Sample Completing the Benchmark Survey Characteristic Number Percent Type of school Secondary schools Special education units Special schools Across schools School location Urban Regional Rural Gender Male Female Primary role Teachers Heads of special education services Principals School transition officers Highest qualification Dip teach (secondary) Dip teach (primary) BEd (primary) BEd (secondary) BEd spec ed Grad dip spec ed MEd Master of special education Years teaching experience Years teaching in transition Other industry experience Yes No N = 104. Missing data results in some percentages not adding up to 100%. Relationships because of practice item content). The need for a glossary of key terms (e.g., transition planning documents, work experience) also was identified, and this material was generated and embedded in the survey. A total of 46 practice items, with abbreviated descriptors, resulted from this process (see Table 2). Item distribution across practice areas was as follows: Student-Focused Planning (n = 9), Student Development (n = 10), Interagency Collaboration (n = 7), Family School Relationships (n = 7), and Program Structure (n = 13). Demographic and practice items were then incorporated into an online survey

5 Beamish et al. 5 Table 2. Items With Abbreviated Descriptors by Practice Areas Abbreviation Practice Student-Focused Planning 1.1 Postsecondary education goals Where appropriate, education and training goals in postsecondary education (e.g., TAFE or university) are specified in each student s transition planning documents. 1.2 Community access goals Education and training goals in community access and participation (e.g., travel training and shopping skills) are specified in each student s transition planning documents. 1.3 Work experience/day activity Education and training goals in work experience and/or day program activity are specified in goals each student s transition planning documents. 1.4 Independent living goals Education and training goals in independent living (e.g., cooking, budgeting) are specified in each student s transition planning documents. 1.5 Goal/program relationship A clear relationship exists between the education and training goals set in the transition planning documents and the program of instruction, learning, and work experiences. 1.6 Student takes active role in The student takes an active role in the transition program planning team. planning 1.7 Students taught to take active Students are taught how to express personal views in individual transition planning meetings. role 1.8 Planning for and use of adjustments When necessary, there is planning for and use of adjustments for students (e.g., the use of communication devices) so that student participation in transition planning meetings is facilitated. 1.9 Student and family needs Student and family needs and preferences are actively elicited during the transition planning elicited process. Student Development 2.1 Life skills training in social skills Life-skills instruction includes training in social skills (e.g., interpersonal skills). 2.2 Training for self-advocacy Life-skills instruction includes teaching the student to identify, plan, and advocate for their personal needs, preferences, and interests. 2.3 Data gathered in natural Data on student performance are gathered in natural environments (e.g., shopping skills in environments the local supermarket). 2.4 Adjustments in natural Educational adjustments are provided during instruction in natural environments (e.g., a selfmonitoring chart). environments 2.5 Instruction in work related Employment skills instruction includes information on work-related behaviors such as jobseeking skills, appropriate dress, and being punctual. behaviors 2.6 Work exp in last 3 4 years Work experience is spread across the last 3 4 years of schooling. 2.7 Work exp available in 3 settings Work experience in the community is available to most students at least 3 times during their transition program. 2.8 Work exp across different jobs Work experience in the community is available across different jobs. 2.9 Work exp is student preferred Work experiences in the community include placements that are identified or preferred by the student School assists with postschool jobs Interagency Collaboration 3.1 Formal collaborative structures exist 3.2 Roles/responsibilities clearly articulated 3.3 Meetings with postschool providers The school assists students to find their preferred postschool jobs or programs. Formal collaborative structures are established between school transition programs, work experience providers, and other service agencies. Roles and responsibilities (e.g., referral protocols, and task responsibilities) are clearly articulated with work experience providers and other service agencies. Teachers and providers of work experience and postschool services meet regularly (e.g., once every semester) to identify and address school- and community-level transition education and service issues. 3.4 Information passed to parents Students and their families are given information on community organizations that provide postschool programs at least 1 2 years prior to the student leaving school. 3.5 Funding for placements exists Funding is available so that schools can place students for periods of time in postschool services while they are still at school and follow up on school exit placements of past students. 3.6 Cross-departmental arrangements 3.7 Postschool providers attend IEP Funding is provided by relevant departments to establish a postschool coordinating/case management position to assist students and/or families (where necessary) when students leave school. When placement with an agency that provides postschool programs is being discussed, the transition planning team involves personnel from that agency. (continued)

6 6 The Journal of Special Education XX(X) Table 2. (continued) Abbreviation Practice Family School Relationships 4.1 Personalized relationships with Schools build a personalized relationship with families. families 4.2 Transition aims provided to Schools provide families with information about the aims of the transition program. families 4.3 Planning involves family The transition planning program team includes family members. members 4.4 Family members lead decision Family members are an integral part of the transition planning team and lead decision making. making 4.5 School/family negotiate roles The school and families negotiate their respective roles and responsibilities during the transition process. 4.6 Families presented with Family involvement is supported through the presentation and explanation of options during options 4.7 Location and timing of meetings Program Structure 5.1 Program philosophy/policy documented 5.2 Families contribute to overall planning the decision-making process in preparation for a student s life after school. Attention is given to the location and timing of transition planning meetings so that family members can participate. Transition-related philosophy, policies, and procedures adopted by schools are documented. Families are encouraged to contribute to the overall school transition program (planning, implementation and evaluation) in addition to the individual planning for their son or daughter. The school system supports the full access and participation of all students in the transition program in school to work and postschool activities. Personnel with specific responsibilities for transition education are identified at the school 5.3 School supports transition access 5.4 School transition personnel identified level. 5.5 Systemic transition personnel Personnel with specific responsibilities for transition education are identified at the school identified level. 5.6 Strategic planning at local level Strategic planning for school to postschool transition services is conducted at the local community level. 5.7 Strategic planning at the system Strategic planning for school to postschool transition services is conducted at systems levels level (statewide and regional). 5.8 Ongoing evaluation by the There is an ongoing evaluation by schools of their overall transition programs. school 5.9 Evaluation of each student s The school evaluates each student s transition program. program 5.10 Evaluation 12 months The school evaluates postschool outcomes for their students for a period of at least 12 postschool months Training available to teachers Teachers involved in transition programming receive preservice and in-service training activities that promote the implementation of recommended practice Training available to other Teaching assistants involved in transition programming receive in-service training activities school staff that promote the implementation of recommended practice Training available to families Families and community service providers involved in transition programming receive training and providers activities that promote the implementation of recommended practice. using a package developed at Griffith University called Surveymaker. The survey format involved three parts. Part 1 aimed to collect demographic information using fixed-response questions. Parts 2 and 3 provided a relevant glossary of terms before presenting each of the 46 practice item statements for a response using a six-category response format. In Part 2, participants were asked to indicate the extent to which they agreed that each of the transition practices was an indicator of program quality. This level of agreement was rated on a continuum consisting of completely disagree, disagree, inclined to disagree, inclined to agree, agree, and completely agree. In Part 3, participants were asked to indicate the extent to which they implemented each practice in their school programs. This level of implementation was rated on a continuum consisting of not at all, infrequently, not very much, on a few occasions, frequently, and very frequently. In addition, comments on each of the five practice areas on the survey were sought in Parts 2 and 3 and a final open-ended comment was invited at the end of Part 3.

7 Beamish et al. 7 Analyses The quantitative components of the benchmark survey were analyzed using SPSS. The first step in these analyses was to use a data-collapsing strategy to decrease the number of response categories. The systematic collapsing of categories is a well-established procedure in survey research methodology (e.g., Babbie, 1995; Goldberg & Williams, 1988) and in best practice research (e.g., Beamish, 2008; Chalmers et al., 1998; Odom, McLean, Johnson, & LaMontagne, 1995; Stephenson & Carter, 2005). In the present study, the collapsing of categories was aimed to facilitate inferences about decision making related to levels of agreement and implementation. For level of agreement, the two lowest levels (completely disagree and disagree) were combined and labeled disagreement. The two middle levels (inclined to disagree and inclined to agree) were combined and labeled uncertain, whereas the two highest ratings (agree and completely agree) were combined and labeled agreement. Similarly, for levels of implementation, the two lowest levels (not at all and infrequently) were combined and labeled nonimplementation. The two middle levels (not very much and on a few occasions) were combined and labeled uncertain, whereas the two highest ratings (frequently and very frequently) were combined and labeled implementation. These labels for the combined ratings are used as measures of agreement and implementation throughout this article. The rationale for combining the categories in this manner was that, for both agreement and implementation, the two lowest ratings were a clear indication of a lack of agreement with, or commitment to, the item as an indicator of program quality or of nonimplementation (Beamish, 2008). Conversely, the two highest ratings indicated either a strong agreement with or commitment to the item as an indicator of program quality or a high rating for implementation. In contrast, the two middle ratings indicated a degree of uncertainty about commitment to or implementation of the practice and so could be grouped together. The examination of levels of agreement and implementation used well-known benchmarking conventions. For level of agreement, both the 50% criterion level of agreement (Odom et al., 1995) and the more stringent 80% criterion level (Beamish, 2008; Williams et al., 1990) were used. For level of implementation, the traditional 50% criterion level for implementation (Beamish, 2008; Odom et al., 1995; Williams et al., 1990) was used. To provide an alternative approach to analyzing levels of agreement and implementation, the response categories were assumed to have interval value. Each category was allocated a numerical value for level of agreement (strongly disagree = 1, disagree = 2, inclined to disagree = 3, inclined to agree = 4, agree = 5, and completely agree = 6) and for level of implementation (not at all = 1, infrequently = 2, not very much = 3, on a few occasions = 4, frequently = 5, and very frequently = 6). This allocation of numerical values provided the opportunity to analyze responses using descriptive statistics for each item and for each practice area. Results Results are reported for only the quantitative components of the statewide survey. First, ratings (percentages and means) for agreement and implementation are presented for items within each practice area. Levels of agreement and implementation are examined according to the 50% and 80% benchmarking conventions (see the analyses section for details). Second, mean ratings are presented for each practice area. Student-Focused Planning Table 3 presents data on levels of agreement and implementation for each of the nine practice items in terms of percentage and mean ratings. The majority of participants (ranging from 77% to 94%) agreed with the practices, whereas a smaller proportion (ranging from 51% to 90%) reported implementing them. Mean ratings confirmed the relative percentage ratings across items. All items met the traditional 50% benchmark levels for agreement and implementation. In addition, the more stringent 80% benchmark level for agreement was met by seven (77.8%) of the items. A closer inspection of data revealed that for seven of the nine items, levels of agreement and implementation were similar. Items with the highest discrepancies between levels of agreement and implementation were 1.7 (students taught to take active role) and 1.8 (planning for and use of adjustments). These two items also displayed very high levels of uncertainty for implementation (43% and 34%, respectively) and high levels of uncertainty for agreement (22% and 17%, respectively). Student Development Percentage and mean ratings of agreement and implementation for each of the 10 practice items are presented in Table 4. Results in this practice area were similar to those reported for Student-Focused Planning. A majority (ranging from 78% to 98%) agreed with these practices, with a smaller proportion (ranging from 55% to 91%) signaling practice implementation. Mean ratings also supported the relative percentage ratings across items. All items met the traditional 50% benchmark levels for agreement and implementation. In addition, the more stringent 80% benchmark level for agreement was met by all but one (90%) of the items. As for the previous area, ratings

8 8 The Journal of Special Education XX(X) Table 3. Percentage and Mean Ratings for Student-Focused Planning Items Level of agreement Level of implementation Mean rating Student-Focused Planning item Agreement Uncertainty Disagreement Implementation Uncertainty Nonimplementation Agreement a Implementation b 1.9 Student and family needs elicited 1.3 Work experience/day activity goals 1.5 Goal/program relationship 1.2 Community access goals 1.4 Independent living goals 1.6 Student takes active role in planning 1.8 Planning for and use of adjustments 1.1 Postsecondary education goals 1.7 Students taught to take active role Agreement ratings are denoted in roman type and implementation ratings are in italics. Items are ranked according to level of agreement. a 6 = completely agree, 5 = agree, 4 = inclined to agree, 3 = inclined to disagree, 2 = disagree, 1 = completely disagree. b 6 = very frequently, 5 = frequently, 4 = on a few occasions, 3 = not very much, 2 = infrequently, 1 = not at all. Table 4. Percentage and Mean Ratings for Student Development Items Level of agreement Level of implementation Mean rating Student Development items Agreement Uncertainty Disagreement Implementation Uncertainty Nonimplementation Agreement a Implementation b 2.5 Instruction in work related behaviors 2.2 Training for selfadvocacy 2.3 Data gathered in natural environments 2.10 School assists with postschool jobs 2.1 Life skills training in social skills 2.7 Work experience available in 3 settings 2.6 Work experiences in last 3 4 years 2.8 Work experience across different jobs 2.4 Adjustments in natural environments 2.9 Work experience is student preferred Agreement ratings are denoted in roman type and implementation ratings are in italics. Items are ranked according to level of agreement. a 6 = completely agree, 5 = agree, 4 = inclined to agree, 3 = inclined to disagree, 2 = disagree, 1 = completely disagree. b 6 = very frequently, 5 = frequently, 4 = on a few occasions, 3 = not very much, 2 = infrequently, 1 = not at all. were similar for 8 of the 10 items. Items with the highest agreement implementation discrepancies were 2.2 (training for self-advocacy) and 2.4 (adjustments in natural environments). These two items also displayed high levels of uncertainty for implementation (22% and 37%, respectively).

9 Beamish et al. 9 Table 5. Percentage and Mean Ratings for Interagency Collaboration Items Level of agreement Level of implementation Mean rating Interagency Collaboration items Agreement Uncertainty Disagreement Implementation Uncertainty Nonimplementation Agreement a Implementation b 3.4 Information passed to parents 3.1 Formal collaborative structures exist 3.2 Roles/responsibilities clearly articulated 3.7 Postschool providers attend IEP 3.5 Funding for placements exists 3.6 Cross-departmental arrangements 3.3 Meetings with postschool providers Agreement ratings are denoted in roman type and implementation ratings are in italics. Items are ranked according to level of agreement. a 6 = completely agree, 5 = agree, 4 = inclined to agree, 3 = inclined to disagree, 2 = disagree, 1 = completely disagree. b 6 = very frequently, 5 = frequently, 4 = on a few occasions, 3 = not very much, 2 = infrequently, 1 = not at all. Interagency Collaboration Table 5 presents data on levels of agreement and implementation for each of the seven practice items in terms of percentage and mean ratings. In contrast with the previous two areas, a smaller majority of participants (ranging from 70% to 89%) agreed with these practices. Mean ratings again confirmed the relative percentage ratings across items. All items met the traditional 50% benchmark level for agreement, but only four of the items met the 80% benchmark. In this practice area, a wide variation in implementation (ranging from 21% to 88%) was reported. Four items met the 50% benchmark for implementation, and these were the items that met the stringent 80% agreement benchmark. Items with the highest agreement implementation discrepancies were 3.5 (funding for placements exists) and 3.6 (cross-departmental arrangements). Family School Relationships High and consistent percentage and mean ratings of agreement and implementation for each of the seven practice items are presented in Table 6. Almost all participants (ranging from 86% to 98%) agreed with the practices, whereas a slightly smaller proportion (ranging from 80% to 94%) reported implementing them. All items met three benchmarks: the 50% and 80% benchmarks for agreement and the 50% benchmark for implementation. In fact, the items also met an 80% level for implementation. These highly correlated levels of agreement and implementation resulted in low or nonexistent agreement implementation discrepancies and low levels of uncertainty across all items. Program Structure Table 7 presents data on levels of agreement and implementation for each of the 13 practice items in terms of percentage and mean ratings. As in the area of Interagency Collaboration, a similar majority of participants (ranging from 63% to 90%) agreed with the practices, and a similar wide variation in implementation (ranging from 21% to 85%) was reported. In general, mean ratings supported respective percentage ratings. All items met the traditional 50% benchmark level for agreement, with only six (46%) of the items meeting the 80% benchmark. A close inspection of the seven items that did meet this stringent benchmark revealed that all but one of them did not reach the 50% implementation benchmark. It followed that the majority (five of the seven items) had high agreement implementation discrepancies. Items with the highest discrepancies between agreement and implementation were 5.1 (program philosophy and policy documented), 5.2 (families contribute to overall planning), 5.5 (systemic transition personnel identified), 5.7 (strategic planning at the system level), 5.8 (ongoing evaluation by the school), 5.10 (evaluation at 12 months postschool), 5.11 (training available to teachers), 5.12 (training available to other school staff), and 5.13 (training available to families and providers). The majority of these items were also characterized by high levels of uncertainty. Summary of Quantitative Data To provide an overview of the quantitative data, mean levels of agreement and implementation were compared across

10 10 The Journal of Special Education XX(X) Table 6. Percentage and Mean Ratings for Family School Relationships Items Level of agreement Level of implementation Mean rating Family School Relationships Items Agreement Uncertainty Disagreement Implementation Uncertainty Nonimplementation Agreement a Implementation b 4.2 Transition aims provided to families 4.7 Location and timing of meetings 4.3 Planning involves family members 4.6 Families presented with options 4.1 Personalized relationships with families 4.5 School/family negotiate roles 4.4 Family members lead decision making Agreement ratings are denoted in roman type and implementation ratings are in italics. Items are ranked according to level of agreement. a 6 = completely agree, 5 = agree, 4 = inclined to agree, 3 = inclined to disagree, 2 = disagree, 1 = completely disagree. b 6 = very frequently, 5 = frequently, 4 = on a few occasions, 3 = not very much, 2 = infrequently, 1 = not at all. Table 7. Percentage and Mean Ratings for Program Structure Items Level of agreement Level of implementation Mean rating Program Structure items Agreement Uncertainty Disagreement Implementation Uncertainty Nonimplementation Agreement a Implementation b 5.3 School supports transition access 5.4 School transition personnel identified 5.1 Program philosophy/ policy documented 5.8 Ongoing evaluation by the school 5.9 Evaluation of each student s program 5.11 Training available to teachers 5.2 Families contribute to overall planning 5.5 Systemic transition personnel identified 5.6 Strategic planning at local level 5.12 Training available to other school staff 5.13 Training available to families and providers 5.10 Evaluation 12 months postschool 5.7 Strategic planning at the system level Agreement ratings are denoted in roman type and implementation ratings are in italics. Items are ranked according to level of agreement. a 6 = completely agree, 5 = agree, 4 = inclined to agree, 3 = inclined to disagree, 2 = disagree, 1 = completely disagree. b 6 = very frequently, 5 = frequently, 4 = on a few occasions, 3 = not very much, 2 = infrequently, 1 = not at all.

11 Beamish et al. 11 Percent of agreement & implementation Student focused planning Student development Interagency collaboration Agreement Family school relations Program structure Implementation Figure 2. Comparison of mean levels of agreement and implementation across the five practice areas. the five practice areas. Comparisons focus on mean percentages and then on mean ratings. Both comparisons furnish complementary patterns of agreement and implementation. A graphic comparison of mean percentages is presented in Figure 2. Taken as a whole, high levels of agreement were accompanied by lower levels of implementation. For level of agreement, all practice areas met the 50% international benchmark, with three of the five areas meeting the more stringent 80% criterion (Family School Relationships, Student Development, and Student-Focused Planning). Likewise, for level of implementation, all practice areas met the 50% criterion. The areas of Interagency Collaboration and Program Structure, however, only just met this benchmark and were the lowest on agreement compared to the other practice areas. In contrast, the area of Family School Relationships showed a position of prominence by exceeding a 90% level of agreement and a slightly lower level of implementation. A corresponding comparison of mean ratings is presented in Table 8. The average ratings for agreement ranged from 5.07 to 5.54, reflecting a high level of agreement across practice areas. On the other hand, average ratings for implementation ranged from 3.81 to These data confirmed the relative positions of the areas of Family School Relationships and Program Structure as reported above and illustrated in Figure 2. Table 8. Practice Areas With Number of Items by Mean Ratings of Agreement and Implementation Practice area Discussion Number of items Mean agreement rating a Mean implementation rating b Student-Focused Planning Student Development Interagency Collaboration Family School Relationships Program Structure a 6 = completely agree, 5 = agree, 4 = inclined to agree, 3 = inclined to disagree, 2 = disagree, 1 = completely disagree. b 6 = very frequently, 5 = frequently, 4 = on a few occasions, 3 = not very much, 2 = infrequently, 1 = not at all. Data from this benchmark survey produced an initial view of practice throughout Queensland transition programs for youth with intellectual disability and autism. This view of practice was provided by a substantial cohort of teachers and other influential staff (N = 104) who were asked to provide data related to ideal practice (agreement that a specific transition practice was an indicator of program quality) and actual practice (everyday implementation of that transition practice in their particular school program). Overall, three key findings emerged from this benchmarking of practice (ideal and actual). First, the level of practice agreement was sufficiently high to socially validate the North American practices within the Australian context. Second, Family School Relationships emerged as an area of practice strength. Third, high agreement implementation discrepancy patterns in the areas of Interagency Collaboration and Program Structure provided cause for concern. Validation of Transition Practices Participant endorsement of transition practices drawn from Kohler s work was a key feature of this study. All practices (N = 46) met the 50% benchmark criterion for level of agreement, which signified that participants identified each and every practice to be an indicator of program quality. Moreover, consensus among participants was so strong that 33 of the practices (72%) met the more stringent 80% benchmark criterion. Allocation of such high levels of agreement provided a Queensland validation of the North American practices. This outcome confirms Ashman s (1990) assertion that the Australian practice base for working with individuals with intellectual impairment draws

12 12 The Journal of Special Education XX(X) substantially on that established in North America. It also is testament to the recommendation that there is nothing wrong with basing our efforts on accumulated knowledge. The error (if there is one) is not validating this knowledge in our own political, legal, and service provision structures (Ashman, 1990, p. 180). Family School Relationships Practices within Family School Relationships were the most highly valued and most frequently implemented compared to those in other practice areas. This finding is not surprising given the familiarity with and expectation of collaborative family teacher relationships that both teachers and family members experience as a result of the individualized education program meeting process. It was highly likely, therefore, that participants had sound knowledge and competence in this practice area, and as such they perceived these practices as ones over which they have substantial control (Knott & Asselin, 1999; Wandry et al., 2008). Studies from the United States report that the involvement of parents in the transition planning process for students with disabilities is vital (Ankeny, Wilkins, & Spain, 2009), is a common practice (Kraemer & Blacher, 2001), and is viewed as a key factor in improving student outcomes (Defur, Todd-Allen, & Getzel, 2001). In this Queensland study, family involvement in planning processes and meetings and providing families with transition aims and options were the most strongly endorsed family school practices. However, the more empowering practices related to families leading decision making and negotiating roles with families were less supported and were accompanied by levels of greater uncertainty. This trend is consistent with North American findings. For example, Defur et al. (2001) reported that parental views on the transition process were not highly valued or listened to and that despite a high level of parental attendance at meetings, attendance did not always translate into meaningful collaboration. It would appear that participants in this Queensland study were reticent about empowering parents to be equal or leading collaborative partners in the transition planning process. Interagency Collaboration and Program Structure By comparison, practices within the areas of Interagency Collaboration and Program Structure were assigned the lowest overall ratings by participants for both agreement and implementation and were also characterized by the largest agreement implementation discrepancies. For Interagency Collaboration, this pattern of poor endorsement and implementation of transition practices may have been related to postschool agencies not being funded to engage with school students and their teachers. On the other hand, teachers may have had little experience in working with community agencies and may have been unsure about their role in partnering with these staff. Moreover, Queensland teachers, in common with their North American counterparts, may have limited knowledge of the postschool sector (Blanchett, 2001; Knott & Asselin, 1999). Whatever the reasons for the low practice ratings, the current relationships between school staff and postschool agencies in Queensland are problematic. More formal collaborative structures between sectors need to be developed locally between schools and agencies and at the cross-departmental systems level. For Program Structure, almost half of the practices were characterized by very low levels of implementation and high levels of uncertainty. These practices focused not only on staff and parent training but also on program policy, program evaluation, and strategic planning at local and state levels. This correlation of practice ratings suggests a lack of commitment to and support for transition-focused education at the school and systems levels. Teachers in Florida recently identified similar issues and proposed clarity on policy and systemic planning as a primary solution in their efforts to improve transition practice (see Lubbers et al., 2008). A search of Web sites across Queensland education sectors reveals little information and no specific policy on transition-focused education. This lack of policy and strategic direction may account for the reported confusion about staff roles and responsibilities in this overarching practice area. Given that good program practice requires a sound policy base, there is an urgent need in Queensland for government and nongovernment sectors to position transitionfocused education within their established policy and procedures frameworks. Limitations and Implications for Future Research Findings from this study provide an initial benchmarking of transition practice in Queensland, Australia. Because transition-focused education for students with disabilities across Australia is not driven by nationally determined policies, transition standards, or accountability measures, these findings may apply in varying degrees to other states. Findings, however, should be considered with a number of limitations kept in mind. First, sampling bias was evident in participation selection, with only interested teachers and other influential staff at government and nongovernment secondary and special schools completing the survey. Bias was further magnified by problems experienced in the online survey. The views expressed by the 104 participants who persevered and contributed data to the study may not necessarily represent the views of the majority of staff involved in transition programs throughout Queensland. Second, the present study relied on participants to indicate

13 Beamish et al. 13 practice agreement and implementation using (a) predetermined practice statements and (b) a range of fixed-response categories. These self-reported data may not have adequately captured genuine beliefs about ideal practice for certain educational communities. Although anonymous, self-reported data also may not always provide an accurate snapshot of actual practice at particular transition programs, such field-based research is crucially important in the planning of research and evaluation as it relates to internal and statistical conclusion validity. The former form of validity relates to the effectiveness of programs or interventions, whereas the latter refers to the degree to which conclusions reached about relationships in data are reasonable (see Gravetter & Forzano, 2009). The benchmarks identified as a result of this initial study point to the need for further Australian research at a number of levels. At a district level, in-depth cases studies involving individual teachers and schools could provide a better understanding of practice implementation and identify solution-focused strategies for practice improvement (see, e.g., Lubbers et al., 2008). At a state level, a longitudinal inquiry across schools and educational sectors could investigate the effect of specific policy and strategic directions on transition practice and student outcomes if such initiatives were put in place at a systems level (see, e.g., Zhang et al., 2005). At a national level, the current study could be replicated to provide an initial comparison of transition practice across states (see, e.g., Thoma et al., 2002). These data could then be used to develop Australian benchmarks for transition practice, which in turn could be incorporated into standards of practice (at a professional level) and a national evaluation tool (at a systems level). Moreover, other countries may consider replicating elements of this study to develop their distinct transition benchmarks, standards of practice, and evaluation tools. Conclusion Findings from the present study signal an urgent need for action at the teacher, school, and systems levels. Much could be achieved if teachers adopted a more proactive stance and partnered differently with significant others in the transition process. As teacher leaders (Hook & Vass, 2002, p. 10), they have the capacity to create situations where students and parents can meaningfully contribute in decision making about postschool life, families can be empowered to press for better postschool options, and postschool providers can feel more valued. Moreover, at a school level, they can team with administration and other school staff to build a more responsive school culture that can generate documents and structures related to local school philosophy, transition policy, and interagency cooperation. Furthermore, this bottom-up approach can extend its zone of influence through schools and districts to put pressure on the wider system to commit to transition-focused education for students with disabilities. System commitment should translate into the formulation of specific transition policies and procedures, the establishment of cross-departmental management structures to assist school leavers and their families, and the provision of specific training opportunities for school staff and parents. Teachers, schools, and systems all have a responsibility to ensure that transition policies, structures, and practices are in place to support the achievement of positive postschool pathways for youth with intellectual disability and autism in Queensland. Declaration of Conflicting Interests The author(s) declared no potential conflicts of interests with respect to the authorship and/or publication of this article. Funding The author(s) disclosed receipt of the following financial support for the research and/or authorship of this article: This research was supported by the Queensland Department of Education and the Arts (Grant CO 13/2005). References Anderson, D., Kleinhammer-Tramill, J., Morningstar, M. E., Lehmann, J., Bassett, D., Kohler, P., &... Wehmeyer, M. (2003). What s happening in personnel preparation in transition? A national survey. Career Development for Exceptional Individuals, 26, Ankeny, E. M., Wilkins, J., & Spain, J. (2009). Mothers experiences of transition planning for their children with disabilities. Teaching Exceptional Children, 41(6), Ashman, A. F. (1990). Setting a research agenda for the study of intellectual disability in Australia. Australia and New Zealand Journal of Developmental Disabilities, 16, Atkins, K., Campbell, R., & Hartley, V. (1987, October). A comprehensive analysis of Mississippi public school educational programs for school age children and youth with severe handicaps. Paper presented at the 14th annual conference of the Association for the Severely Handicapped, Chicago, IL. Retrieved from ERIC database. (ED294334) Ayres, B. J., Meyer, L. H., Erevelles, N., & Park-Lee, S. (1994). Easy for you to say: Teacher perspectives on implementing most promising practices. Journal of the Association for Persons With Severe Handicaps, 19, Babbie, E. R. (1995). The practice of social research (7th ed.). Belmont, CA: Wadsworth. Beamish, W. (1992). Levels of acceptance and implementation of basic indicators of best educational practice for students with severe handicaps: A Queensland study. Unpublished master s thesis, Griffith University, Brisbane, Australia.

14 14 The Journal of Special Education XX(X) Beamish, W. (2008). Consensus about program quality: An Australian study in early childhood special education. Saarbrucken, Germany: VDM. Blanchett, W. J. (2001). Importance of teacher transition competencies as rated by special educators. Teacher Education and Special Education, 24, Centre for Disability Research and Development, Edith Cowan University. (1995). Report on the evaluation of the 1994 Western Australian Transition Support Program for students with disabilities. Perth, Australia: Edith Cowan University. Chalmers, S., Carter, M., Clayton, M., & Hook, J. (1998). Education of students with high support needs: Teachers perceptions of possible best practices, reported implementation and training needs. Australasian Journal of Special Education, 22, Cook, B. G., Tankersley, M., & Landrum, T. J. (2009). Determining evidence-based practices in special education. Exceptional Children, 75, Council for Exceptional Children. (2001). Standards for the preparation of transition specialists. Arlington, VA: Author. Davies, M., & Beamish, W. (2009). Transition from school for young adults with intellectual disability: Parental perspectives on life as an adjustment. Journal of Intellectual and Developmental Disabilities, 34, Defur, S. H., Todd-Allen, M., & Getzel, E. E. (2001). Parent participation in the transition planning process. Career Development for Exceptional Individuals, 24, Division on Career Development and Transition. (2000). Transition specialist competencies (Fact sheet). Reston, VA: Council for Exceptional Children, Division on Career Development and Transition. Eichinger, J., & Downing, J. (1992). An administrator and teacher perspective on program quality indicators for students with severe disabilities. Journal of the Association for Persons With Severe Handicaps, 17, Everitt, A., & Hardiker, P. (1996). Evaluating for good practice. London, UK: Macmillan. Fox, W., Thousand, J., Williams, W., Fox, T., Towne, P., Reid, R., &... Calcagni, L. (1986). Best educational practices 86: Educating learners with severe handicaps. Burlington: University of Vermont, Center for Developmental Disabilities. Goldberg, D., & Williams, P. (1988). A user s guide to the General Health Questionnaire. Windsor, UK: Nfer-Nelson. Gravetter, F. J., & Forzano, L.-A. B. (2009). Research methods for the behavioural sciences. Belmont, CA: Wadsworth Cengage. Hook, P., & Vass, A. (2002). Teaching with influence. London, UK: David Fulton. Hudson, M., Carr, L., Dowsett, J., & Moroney, J. (2000). Startright final report: Transition from school to post school options for students with a disability. Adelaide: South Australia Department of Education, Training, and Employment. Retrieved from Individuals With Disabilities Education Act, Pub. L. No , 104 Stat (1990). Individuals With Disabilities Education Act, Pub. L. No , 111 Stat. 37 (1997). Individuals With Disabilities Education Improvement Act, Pub. L. No , 118 Stat (2004). Knott, L., & Asselin, S. B. (1999). Transition competencies: Perception of secondary special education teachers. Teacher Education and Special Education, 22, Kohler, P. D. (1996). Preparing youth with disabilities for future challenges: A Taxonomy for Transition Programming. In P. D. Kohler (Ed.), Taxonomy for Transition Programming: Linking research to practice (pp. 1 62). Champaign, IL: University of Illinois at Urbana Champaign, Transition Research Institute. Kohler, P. D. (2000). Tools for incorporating transition-related competencies into teacher education. Kalamazoo: Western Michigan University, Department of Educational Studies. Kohler, P. (2002). Taxonomy for Transition Planning. Worksheet for student-focused planning and student development practices. Retrieved from index.html Kohler, P. (2003a). Taxonomy for Transition Planning. Worksheet for family involvement practices. Retrieved from homepages.wmich.edu/~kohlerp/index.html Kohler, P. (2003b). Taxonomy for Transition Planning. Worksheet for interagency collaboration practices. Retrieved from homepages.wmich.edu/~kohlerp/index.html Kohler, P. D., & Field, S. (2003). Transition-focused education: Foundation for the future. The Journal of Special Education, 37, Kraemer, B. R., & Blacher, J. (2001). Transition for young adults with severe mental retardation: School preparation, parent expectations, and family involvement. Mental Retardation, 39, Laragy, C. (2004). Self-determination within Australian school transition programmes for students with a disability. Disability and Society, 19, Lubbers, J. H., Repetto, J. B., & McGorray, S. P. (2008). Perceptions of transition barriers, practices, and solutions in Florida. Remedial and Special Education, 29, McMahan, R., & Baer, R. (2001). IDEA transition policy compliance and best practice: Perceptions of transition stakeholders. Career Development for Exceptional Individuals, 24, Meadows, D. (2009). Where have all our students gone? School to post-school transition in Australia. Australasian Journal of Special Education, 33, Odom, S. L., McLean, M. E., Johnson, L. J., & LaMontagne, M. J. (1995). Recommended practices in early childhood special education: Validation and current use. Journal of Early Intervention, 19, Otis-Wilborn, A., Winn, J., Ford, A., & Keyes, M. (2000). Standards, benchmarks, and indicators: Designing a framework for

15 Beamish et al. 15 professional development of preservice and practicing teachers. Teaching Exceptional Children, 32(5), Parmenter, T. R., & Riches, V. C. (1990). Establishing individual transition planning for students with disabilities within the NSW Department of School Education. Sydney, Australia: Macquarie University, School of Education. Peters, M., & Heron, T. (1993). When the best is not good enough: An examination of best practice. Journal of Special Education, 26, Riches, V. (1996). A review of transition from school to community for students with disabilities in NSW, Australia. Journal of Intellectual and Developmental Disability, 21, Riches, V. C., & Parmenter, T. R. (1993). NSW TAFE transition courses for students with disabilities: A research and development report. Sydney, Australia: Macquarie University, School of Education. Riches, V., Parmenter, T., Fegent, M., & Bailey, P. (1993). Secondary education: A follow-along study of students with disabilities in NSW (Report). Sydney, Australia: Macquarie University, School of Education. Riches, V., Parmenter, T., & Robertson, G. (1996). Youth with disabilities in transition from school to community (Report). Sydney, Australia: Macquarie University, School of Education, Unit for Community Integration Studies. Stephenson, J., & Carter, M. (2005). Student teachers perceptions of best practice in practicum with learners who have high support needs. Australasian Journal of Special Education, 29, Test, D. W., Aspel, N. P., & Everson, J. M. (2006). Best practices and future issues in transition. In Transition methods for youth with disabilities (pp ). Upper Saddle River, NJ: Pearson. Test, D. W., Fowler, C. H., Richter, S. M., White, J., Mazzotti, V., Walker, A. R., &... Kortering, L. (2009). Evidence-based practices in secondary transition. Career Development for Exceptional Individuals, 32, Thoma, C. A., Held, M. F., & Saddler, S. (2002). Transition assessment practices in Nevada and Arizona: Are they tied to best practices? Focus on Autism and Other Developmental Disabilities, 17, Wandry, D. L., Webb, K. W., Williams, J. M., Bassett, D. S., Asselin, S. B., & Hutchinson, S. R. (2008). Teacher candidates perceptions of barriers to effective transition programming. Career Development for Exceptional Individuals, 31, Watters, M., Riches, V., & Parmenter, T. (1993). NSW Transition Project: Secondary vocational education job coach service. Sydney, Australia: Macquarie University, School of Education. Williams, W., Fox, T., Thousand, J., & Fox, W. (1990). Level of acceptance and implementation of best practices in the education of students with severe handicaps in Vermont. Education and Training in Mental Retardation, 25, Zhang, D., Ivester, J. G., Chen, L., & Katsiyannis, A. (2005). Perspectives on transition practices. Career Development for Exceptional Individuals, 28, About the Authors Wendi Beamish, PhD, currently conducts research in teacher practice related to transitions, early childhood intervention, positive behavioral support, and inclusive education at Griffith University. Denis Meadows, PhD, currently conducts research in postschool transitions for individuals with developmental disabilities at Griffith University. Michael Davies, PhD, currently conducts research in transitions for individuals with developmental disabilities and inclusive education at Griffith University.