RESEARCH. Assessing School Leader and Leadership Programme Effects on Pupil Learning. Conceptual and Methodological Challenges

RESEARCH Assessing School Leader and Leadership Programme Effects on Pupil Learning Conceptual and Methodological Challenges Kenneth Leithwood Professor OISE/University of Toronto Ben Levin Deputy Minister of Education Government of Ontario Research Report RR662

Research Report No 662 Assessing School Leader and Leadership Programme Effects on Pupil Learning Conceptual and Methodological Challenges Kenneth Leithwood Professor OISE/University of Toronto Ben Levin Deputy Minister of Education Government of Ontario The views expressed in this report are the authors and do not necessarily reflect those of the Department for Education and Skills. Kenneth Leithwood and Ben Levin 2005 ISBN 1 84478 527 0

Table of Contents Summary...3 Introduction...6 The Meaning of Leadership...6 Leadership Effects...7 Leadership Programme Effects...10 A General Framework to Guide Research on Programme and Leader Effects on Pupil Learning...12 Leadership Practices: The Independent Variables...14 Leadership Effects: The Dependent Variables...18 Mediating Variables for Leadership Effects Research...23 Variables Moderating Leadership Effects...29 The Antecedents of Leaders Practices...33 Evaluating Leadership Programme Effects...35 Methodological Challenges...40 Conclusions and Recommendations...44 2

Summary Leadership is widely considered a variable critical to school improvement; considerable evidence now justifies the claim that leadership has important effects on pupil learning. While largely indirect, evidence summarized in this paper indicates that such effects explain as much as a quarter of the variation in pupil learning across schools accounted for by school factors. These leadership effects, furthermore, are usually greatest where they are needed most; leadership acts as a catalyst unleashing the potential of other factors contributing to the improvement of pupil learning. Given the key role of leadership in school improvement, the Department for Education and Skills (DfES) commissioned this paper to identify the pathways by which school leadership programmes and practices influence, or positively impact on, pupil learning and to inform future research and development in this field. More precisely, our goals were to develop a model or framework of variables and relationships clarifying how leadership programmes and practices impact on pupil outcomes. Our paper describes challenges associated with conceptualizing the relationship between leadership programmes, changes in leader practices and the effects of such changed practices on schools and pupils. We also grapple with some of the methodological challenges facing evaluators and researchers with an interest in programme and leader effects, offering suggestions about how these challenges might be addressed. Conceptual Challenges Four conceptual challenges are addressed in our paper: How to most usefully frame the relationship between leadership programmes, leadership practices and pupil outcomes? We offer a relatively generic response to this question, suggesting that any comprehensive framework would include independent, dependent, antecedent, moderating and mediating categories of variables. A review of research illustrates what we know about each category of variables and the relationships among them. Frameworks guiding a handful of current and recently completed studies illustrate variations on the generic framework; How the nature of leadership practices might be conceptualized and what intellectual resources are available to assist in such conceptualization? We review a wide range of alternative leadership models to assist in thinking about this challenge; these are models developed in both school and non-school organizational contexts; What can leadership theory and research developed outside of schools contribute to our understanding of school leadership practices and their effects? Building on our earlier description of leadership models in non-school contexts, we identified perspectives on leadership yet to be explored in school-based leadership research but with potential. How to define and what do we know about - dependent, moderating and mediating variables in school leader and leadership programme effects studies? We illustrate the state-of-the-art of knowledge about these variables through a review of leadership research carried out exclusively in U.K. contexts. It is clear from this review that 3

recently published U.K. leadership research is almost exclusively small scale and qualitative in nature. Methodological Challenges Although not as extensive as our treatment of conceptual challenges, the paper also responds to a series of methodological challenges typical of field based leadership research; these challenges are addressed more fully in a subsequent paper 1. One of the problems we take up in this paper is the measurement of leadership practices, examining a small set of instruments commonly used for this purpose. We also discuss such common difficulties in conducting both programme and leadership effects research as the narrow and unreliable nature of commonly used student assessment instruments and how to deal with missing data for individual schools. The paper presents evidence that, while the school is the unit of analysis in much leadership effects research, there is typically greater within - than across - school variation in measures of leadership an important, largely unaddressed, challenge for future research. Promising approaches to the evaluation of leadership programme effects are also outlined. Conclusions and Recommendations Our conclusions and recommendations bear on selected aspects of future DfES - sponsored research that might be carried out on both leadership, and leadership programme, effects. Future leadership effects research should: measure a more comprehensive set of leadership practices than has been included in most research to date: these measures should be explicitly based on coherent images of desirable leadership practices. Such research is likely to produce larger estimates of leadership effects on pupil outcomes than has been provided to date; measure an expanded set of dependent (outcome) variables: these are variables beyond just short-term pupil learning including, as well, longer term effects. Examples of such long - term effects include pupil success in tertiary education, employment and commitment to learning over the life span; systematically describe how leaders successfully influence the condition of variables mediating their effects on pupils: we now have considerable evidence about what are the most potentially powerful variables mediating school leader effects but we know much less about how leaders influence these mediators; attend more systematically to variables moderating (enhancing, reducing) leadership effects: lack of attention to this category of variables seems likely to be a major source of conflicting findings in the leadership research literature. Furthermore, when studies do attend to moderators, their choice has often been difficult to justify and largely atheoretical. 1 See Leithwood, K., Levin, B. (2005). Assessing Leadership Effects on Pupil Learning: Methodological Issues. Toronto: Report submitted to the DfES, January. 4

reflect greater methodological variety than is evident in recently published U.K. leadership research: this will be essential if a robust body of context - relevant knowledge is to be developed. Future leadership programme effects research should: be guided by conceptual frameworks similar to those we have recommended for leadership effects research: our review suggests that the vast majority of previous efforts to evaluate leadership programme effects, in most parts of the world, have not generated the type and quality of evidence required to confidently answer questions about their organizational or pupil effects. This problem could be addressed by introducing, into the conceptual frameworks we have suggested for leadership effects studies, leadership programmes conceptualized as one category of antecedent variables stimulating changes in leadership practices. provide comparative information about programme effects: formal programmes are just one of many influences on leaders practices; to fully appreciate the value of such programmes, their effects need to be compared to the effects of such other antecedents as on-the-job learning, leaders traits and early family experiences, for example. Such information would inform not only programme improvement efforts but leadership selection processes, as well. Information of this sort would also assist with cost-effectiveness judgements in the context of planning for leadership development. be funded at levels consistent with the expectations for what is to be accomplished: few documented programme evaluations provide the type of comprehensive data we call for here and funding is part of the reason. If future leadership programme evaluations are to assess the direct and indirect effects of such programmes on pupil learning as well as leaders practices, then a different level of funding will be required than has been typical to date. Implementing these recommendations will require considerable attention to a significant number of conceptual and technical issues at the core of designing and conducting high quality, high impact research and evaluation. Just doing more research of the type that is typically being done in the country now - at least as it is reflected in the published literature - seems unlikely to significantly advance our understandings about how leadership programmes and leaders most productively improve pupil learning. 5

Introduction Purposes and Methods The Department for Education and Skills (DfES) commissioned this paper to identify the pathways by which school leadership programmes and practices influence, or positively impact on, pupil learning and to inform future research and development in this field. More precisely, our goals were to develop a model or framework of variables and relationships clarifying how leadership programmes and practices impact on pupil outcomes. In this paper we describe alternative approaches to conceptualizing the relationship between leadership programmes, changes in leader practices and the effects of such changed practices on schools and students. We also identify some of the methodological challenges facing evaluators with an interest in programme and leader effects and offer some suggestions about how these challenges might be addressed. Our methods for developing this paper entailed systematic analyses of current research and theory. In the case of each challenge or issue taken up in the paper, we aimed to reflect the best of current thinking and to be quite comprehensive about the relevant literatures to which we paid attention; at least a sample of our sources are cited throughout the paper. We have grappled with many of these issues in our own recent research, as well, and our commentary also reflects that experience. The Meaning of Leadership We begin by reflecting on the thorny question of Just what is leadership, anyway? not because we are able to provide a precise answer but because the concept at least deserves some preliminary exploration in a paper with the focus of this one. It is perfectly reasonable to ask How could we understand leadership effects if we don t have a clear notion of what produced those effects? That said, we raise this question in the face of many claims that there is no agreed definition (e.g., Antonakis, Cianciolo & Sternberg, 2004). Indeed, the leadership literature contains literally hundreds of at least slightly different conceptions of the concept. One way often used to clarify the meaning of leadership is to compare it to the concept of management. Some of these comparisons seem largely unhelpful, as in Bennis and Nanus (1985) claim that management is doing things right and leadership is doing the right things. More helpful, we think, is a distinction offered by Kotter (1990). According to this source, management is about producing order and consistency, whereas leadership is about generating constructive change. Adopting this perspective, the primary effect of organizational leadership would be significant change in a direction valued by the organization. In practice, of course, distinguishing between leadership and management behaviours can be extremely difficult. This is because the distinction rests not on the nature of the behaviour but its effects. If behaviour produces order and consistency then it must be management; if it produces change in a valued direction it must be leadership. 6

Most conceptions of leadership do associate it with productive change. And at the core of most of these conceptions are two functions generally considered indispensable to its meaning: Direction-setting: helping members of the organization establish a widely agreed on direction or set of purposes considered valuable for the organization; and Influence: encouraging organizational members to act in ways that seem helpful in moving toward the agreed on directions or purposes. Each of these functions can be carried out in different ways, such differences distinguishing many models of leadership from one another. As Yukl notes, leadership influences: the interpretation of events for followers, the choice of objectives for the group or organization, the organization of work activities to accomplish objectives, the motivation of followers to achieve the objectives, the maintenance of cooperative relationships and teamwork, and the enlistment of support and cooperation from people outside the group or organization (1994, p. 3). One must assume, in the case of this conception, that the objectives being referred to entail some sort of change. This is it not a very precise way of defining leadership and may be vulnerable to the occasionally-heard charge that such lack of precision severely hampers efforts to better understand the nature and effects of leadership. But leadership is a highly complex concept. Like health, law, beauty, excellence and countless other equally complex concepts, efforts to define it too narrowly are more likely to trivialize than help bring greater clarity to its meaning. Leadership Effects Policy makers aiming to improve schools on a large scale in today s context invariably assume that the success with which their policies are implemented has much to do with the nature and quality of local leadership, especially leadership at the school level (e.g., Caldwell, 2000; Murphy & Datnow, 2003; Young, Peterson & Short, 2001). This has not always been the case. As David Day (2001) reminds us, it was popular several decades ago, at least in academic circles, to claim that the effects of those in organizational leadership roles were substantially outweighed by the effects of other organizational actors and conditions. Conducted in non-school organizations, for example, Meindl s (1995) research about the romance of leadership argued that leadership was simply the easiest and simplest explanation for organizational effects that were actually the result of a host of more complex and harder to understand relationships and conditions. Evers and Lakomski (2000) have mounted similar arguments about school leadership, in particular. However, more recently arguments that leadership does not matter have been overtaken by empirical evidence indicating that it matters a great deal. We summarize this evidence 7

here very briefly with an exclusive focus on pupil effects. It is important to note that most of this evidence has come from research on school-level leaders, especially heads, deputy heads or their equivalent in schools outside of the United Kingdom (U.K.). Local Education Authority (LEA) or district leadership effects on pupils have, until recently, been considered too indirect and complex to sort out, and research on teacher leadership has rarely inquired about pupil effects (Barr & Duke, 2004). 2 Claims about the important effects of leadership on pupils are justified by five quite different types of research evidence: Large-scale quantitative studies of overall leadership effects on pupil test scores: evidence of this type reported between 1980 and 1998 (approximately four dozen studies across all types of schools) has been reviewed in a series of papers by Hallinger and Heck (1996a, 1996b, 1998). These reviews conclude that the combined direct and indirect effects of school leadership on pupil test scores (primarily math and language scores) are small but educationally significant. While leadership explains only three to five percent of the variation in pupil test scores across schools (not to be confused with the very large within-school effects that are likely), this is actually about one quarter of the total across-school variation (10 to 20 percent) explained by all school-level variables, after controlling for pupil intake or background factors (Townsend, 1994; Creemers & Reetzig, 1996). The quantitative school effectiveness studies providing much of these data indicate that classroom factors explain more than a third of the variation in pupil test scores. Overall leadership effects on pupil engagement in schools: using designs comparable to the leadership effects studies summarized above, a second source of evidence about leadership effects on pupils replaces test scores with pupil engagement in school as the criterion variable. Building on Finn s (1989) early work, participation is the behavioural component of engagement and includes students actions in the classroom and school. Identification with school is the psychological component of engagement, its meaning captured in such terms as affiliation, involvement, attachment, bonding and commitment. Two overlapping research programmes on leader effects, one in Canada (e.g., Leithwood & Jantzi, 1999), the other in Australia (e.g. Silins & Mulford, 2002) have used engagement as their dependent measure and both have found significant indirect effects of transformational approaches to leadership on student engagement. Evidence from these programmes and others (Fredricks, Blumenfeld & Paris, 2004) also suggests significant relationships between engagement and both retention and a wide range of achievement outcomes in elementary and secondary schools. Effects of specific leadership practices on pupil test scores: a third source of evidence about leadership effects on pupils is also large-scale and quantitative in nature. However, instead of examining overall leadership effects, it inquires about the effects of specific leadership practices on pupil test scores. Evidence of this sort can be found 2 This very comprehensive review of empirical evidence was able to locate only five studies assessing the effects of teacher leadership on pupils. Only four of these studies actually included direct measures of pupil learning and three of the four found no effects of teacher leadership on such learning. In light of how popular teacher leadership has become as a strategy for school reform, this is an astonishing demonstration of just how susceptible are schools to evidence-free claims about what they ought to do. 8

sporadically in the research alluded to above, but a recent meta-analysis by Waters, Marzano and McNulty (2003) has significantly extended this type of research. This study identifies 21 leadership responsibilities and calculates an average correlation between each and whatever measures of pupil achievement were used in the original studies. From this data, estimates are calculated of the effects on pupil test scores (e.g. a 10 percentile point increase in pupil test scores resulting from the work of an average principal who improved her demonstrated abilities in all 21 responsibilities by one standard deviation (p. 3). 3 Primarily qualitative case study evidence: studies providing this type of evidence typically are conducted in exceptional school settings (e.g. Gezi, 1990; Reitzug and Patterson, 1998 ). These are settings believed to be contributing to pupil learning significantly above or below normal expectations as, for example, effective schools research based on outlier designs. Such studies usually report very large leadership effects not only on pupil learning but on an array of school conditions, as well (e.g. Mortimore, 1993; Scheurich, 1998). What is lacking from this evidence, however, is external validity or generalizability. Research on leader succession is the final and, arguably, most compelling source of evidence about leadership effects. This evidence demonstrates, for example, that few school improvement initiatives survive a change in principal leadership and that important attitudes toward leadership influence are significantly and negatively influenced by frequent changes in such leadership. Teachers come to ignore or otherwise inoculate themselves against the influence of their schools administrative leaders when these leaders are perceived to rotate through the school every two or three years (Hargreaves, Moore, Fink, Brayman, & White, 2003; Macmillan, 1996). Thus inoculated, it takes considerable time and a prodigious effort on the part of a committed leader prepared to stay the course to recover the cooperation of her school s staff in the interests of school improvement. Similarly, leader succession research in non-school organizations provides powerful evidence of very large leader effects, especially the effects of leaders at the apex of their organization s hierarchy. Evidence from Day and Lord (1988) and Thomas (1988) indicates, for example, that: When properly interpreted and methodologically sound, the research on leadership succession---has demonstrated a consistent effect for 3 While this quantitative synthesis of research produces clearly interesting data, estimates from such data to principal effects on pupil learning in real world conditions must be treated with considerable caution. First of all, the data are correlational in nature, but cause and effect assumptions are required for the extrapolated effects of leadership improvement on pupil learning. Second, the illustrative effects on pupil achievement described in the study depend on a leaders improving their capacities across all 21 practices at the same time, an extremely unlikely occurrence! Some of these practices are dispositional in nature, or rooted in deeply held beliefs unlikely to change much, if at all, within adult populations (e.g., ideals/beliefs, flexibility). And just one of the 21 practices, increasing the extent to which the principal is knowledgeable about current curriculum, instruction and assessment practices would be a major professional development challenge, by itself. Nonetheless, this line of research is a useful addition to other lines of evidence which justify a strong belief in the contributions of successful leadership to pupil learning. 9

leadership that explained 20 to 45 percent of the variation in relevant organizational outcomes (Day, 2001). These five sources of evidence add up to powerful support for the importance widely attributed to leadership by policy makers and the public at large. However, without minimizing the considerable progress that has been made over the past 15 years, we still have a great deal to learn about: the nature of school leadership practices that are successful in improving pupil learning; under what conditions or circumstances are some forms of school leadership more successful than others? how successful leadership practices are connected to changes in the school organization and eventually to improvements in pupil learning; and how successful leadership is best developed. Leadership Programme Effects Because effective leadership is so critical to school improvement, massive amounts of energy and considerable resources have been devoted to its initial development and ongoing improvement over the past 15 years, in particular. The National College of School Leadership is a highly visible manifestation of this attention in England. Formal training programmes appear to be the most direct strategy for building leadership capacity but they are by no means uniformly effective. Indeed, calls for more rigorous programme evaluation have become ubiquitous. These calls have escalated significantly in the past decade in response to an outcomes-oriented accountability policy environment for education and a widely shared belief in the importance of effective leadership. Programme evaluators have struggled to keep up with these calls for more rigorous evaluation for several reasons. Most obviously, first of all, the funds available for leadership programme evaluation rarely match the ambition of the expectations. Second, the most ambitious expectations to assess leadership programme effects on pupil learning require highly sophisticated theoretical frameworks, frameworks that lie outside the theoretical repertoire of those typically charged with programme evaluation responsibilities. Finally, and closely related to this second reason, such highly sophisticated frameworks (called indirect effects or mediated models) potentially include all of the variables at the school and classroom level that are themselves the focus of independent lines of active research with the usual debates and uncertainties about their effects on pupil learning. Examples of such variables at the school level include school culture, school improvement planning processes, and shared decision-making. At the classroom level, such variables include, for example, classroom instructional processes, opportunity to learn and class size. In a recent analysis of leadership preparation programmes across the United States, McCarthy (1999) concluded that we do not actually know whether, or the extent to which, such programmes actually achieve the goal of producing effective leaders who create school environments that enhance pupil learning? (p. 133). This gap in our 10

knowledge is not because leadership preparation programmes are never evaluated; rather, the vast majority of such evaluations do not provide the type and quality of evidence required to confidently answer questions about their organizational or pupil effects. Most evaluations are limited to assessing participants satisfaction with their programmes and sometimes their perception of how such programmes have contributed to participants work in schools (McCarthy, 2002). 11

A General Framework to Guide Research on Programme and Leader Effects on Pupil Learning Most efforts to conceptualize the relationships among leadership programmes, leaders practices and pupil learning has assumed that the effects of leaders on pupil learning are largely indirect. Based on this well-justified assumption, then, one of the primary challenges for research on programme and leaders effects is to locate the most defensible set of variables mediating and moderating programme and leaders effects. A second significant challenge is to uncover the nature of the relationships among these variables and between leaders and such variables. Figure 1, reflecting these assumptions, is the general framework guiding our account of how to better understand programme and leader effects. This figure indicates that leadership practices (overt behaviours - or properties of the organization - aimed at direction setting and influence) have direct effects on potentially a wide range of variables; they stand between or mediate the effects of leadership, particularly when those effects are conceptualized as pupil learning. Below we discuss some defensible alternatives to pupil outcomes as potential dependent variables in leadership effects research but also assume that in the current U.K. educational environment pupil learning is likely to be at least among dependent variables necessarily included in DfES or the National College for School Leadership (NCSL) endorsed studies of educational leadership effects. Figure 1 also includes a set of moderating variables. As we explain more fully below, these are features of the organizational or wider context in which leaders work that interact with the dependent and/or mediating variables. These interactions potentially change the strength or nature of relationships between, for example, the independent and mediating variables or the mediating and dependent variables. For example, if previous evidence suggested that male and female teachers respond differently to the same set of headteachers leadership behaviours, then teachers gender would be a promising moderating variable to include in a study of headteacher effects. Some of the antecedents variables in Figure 1 are internal to the leader, including for example, leaders traits, values, cognitions, and emotions. There are external antecedents, as well. These would include leadership programmes, of course, but also such variables as LEA relationships, government educational policies, and leader family and socialization experiences (Popper & Mayseless, 2002). The next several sections of the text address issues in the conceptualization and measurement of each of the variables in Figure 1. 12

Antecedents Moderating Variables e.g. family background family culture gender formal education reward structure Independent Variables leadership practices Mediating Variables school conditions class conditions individual teachers professional community Dependent Variables short term long term Figure 1: A General Framework for Guiding Leadership Effects Research 13

Leadership Practices: The Independent Variables Direction-setting and influence, as we argued above, is the generic meaning of leadership. Both of these functions can be exercised in many different ways and more or less successfully. In this section we illustrate the range of ways in which the successful or effective exercise of leadership has been conceptualized both within and outside of the education sector; these are alternative leadership models or theories. We also identify some important challenges in the measurement of successful or effective leadership. Leadership Models Developed in the Education Sector In their extensive review of research, Leithwood and Duke (1999) developed a five-fold classification of leadership models reflected in the educational literature. These are summarized along with related theory and research published since 1999. Instructional leadership: focuses on the behaviours of teachers as they engage in activities directly effecting the learning of pupils. The more fully developed models in this category (e.g., Hallinger, 2003) also include attention to broader sets of organizational variables, such as school culture or climate, thought to influence teachers classroom practices. Transformational leadership: focuses on the commitments and capacities of organizational members, as well as their willingness to engage in extra effort on behalf of their organizations. While the bulk of the evidence about this approach to leadership has been collected in non-school contexts (e.g., Avolio & Yammarino, 2002), educational researchers have recently begun to redress this imbalance (e.g., Nguni, 2004; Lunenburg, 2004). Moral leadership: is concerned with the ethics and values of those exercising leadership. Specifically, it aims to clarify the nature of the values used by leaders in their decision making and how conflicts among values are best adjudicated (Begley & Leonard, 1999; Begley & Johansson, 2003). A strand within this approach to leadership specifically aims to promote democratic values and the empowerment of a large proportion of organizational members (e.g. Starratt, 2003: Johansson, 2003). Participative leadership: shines a spotlight on group decision-making processes. Educational research inquiring about this approach builds on a strong foundation of research in other sectors dating back to seminal studies in the early 1930 s (e.g., Mayo, 1933) about increases in organizational effectiveness associated with greater participation of employees in meaningful decisions about their work. The extensive body of research on teacher participation in decision making reasonably can be viewed as part of the body of evidence about this model of leadership (e.g., Conley, 1991). Rapidly growing literatures on both teacher leadership (York-Barr & Duke, 2004; Harris & Chapman, 2002) and distributed leadership (Gronn, 2002; Spillane, Halverson & Diamond,2002) are the most recent evolutions of this approach. Managerial and strategic leadership: encompass a range of tasks or functions found in the classical management literature (reviewed in Rost, 1991), including tasks such as coordination, planning, monitoring and the distribution of resources. Educational literature from the United Kingdom reflects a far greater interest in this form of leadership than does the North American literature. Also addressed much more 14

extensively in the UK than the North American literature 4 is the entrepreneurial, creative and change oriented strategic leadership sometimes thought to be the exclusive purview of those occupying senior levels of the organizational hierarchy (Yukl & Lebsinger, 2004). Contingent leadership: emphasizes the need for leaders to be responsive to the unique demands of their organizations and the contexts in which those organizations function. While this approach is quite mature in both education and non-education sectors (e.g., Blake and Mouton, 1964), its original conception was limited to a very small number of dimensions along which leadership styles could vary in response to context (primarily the initiation of structure and demonstrations of consideration for employees). Current leadership research continues to call for more sensitivity to the context in which leaders work and greater flexibility on the part of leaders across a much larger number of dimensions (Yukl & Lepsinger, 2004). Leadership Models Developed in Other Sectors With the exception of instructional leadership, all of the approaches to leadership explored by the educational research community are also active areas of research in other sectors. But academic leadership research 5 in these other sectors reflects an additional range of approaches or models. The classification of this research offered by Dansereau, Yammarino & Markham (1995) illustrates this range. We provide only a cursory description of their categories here, our aim being to simply alert readers to this larger field of work. A total of 13 approaches to leadership appear in the classification system of Dansereau and his colleagues and these are nested within four superordinate categories: Classical approaches: including several contingency-oriented, as well as participative, approaches to leadership. This category assumes that leaders can and ought to change their styles over time in response to the circumstances in which they find themselves. Contemporary approaches: with a more explicit focus on both the leaders and the development of their followers (p. 254), includes charismatic and transformational approaches to leadership. This superordinate category also includes Leader-Member Exchange theory which argues that leaders have unique relationships with individual organizational members depending on such factors as trust, perceived competence and the like. 6. Alternative approaches: expand the focus of attention beyond either the individual leader or followers to relationships and interactions. Included in this superordinate 4 See, for example, the special issue of School Leadership and Management (2004, vol. 24, no. 1) edited by Brent Davies. 5 We use this term academic leadership research in reference to systematic, theoretically informed, empirical inquiry about leadership - as distinct from the highly popular genre of leadership literature which is autobiographical, anecdotal and/or exclusively case based. 6 For one of the few studies of this model of leadership in a school context, see Devereaux s (2004) recent dissertation. 15

category are information processing, substitutes for leadership, and romance of leadership models. New wave approaches: this category includes a quite eclectic set of leadership models held together as much by their recent emergence as anything more conceptually coherent. Self leadership, a multiple linkages model, multi-level theory and individualized approaches to leadership are included. Measuring Leadership Practices Some of the education sector leadership models have been specified in detail and tested with instruments which are quite well developed. This is the case, for example, with Hallinger s instructional leadership model (see Hallinger & Murphy, 1985), Leithwood s transformational school leadership model (Leithwood & Jantzi, 2000) and Marks and Printy s (2003) synthesis of both of these forms of leadership. Several versions of Bass (1985) Multifactor Leadership Questionnaire have been used extensively to study transformational leadership effects primarily in non-school organizations, but in schools and districts, as well (e.g., Nguni, 2004). Many other instruments are available for measuring leadership, especially for doing so in non-education organizations (Clark & Clark, 1990). Without diminishing their many other contributions, one limitation of some of the most widely cited reviews of leadership effects on pupil learning is that they confound estimates of such effects by failing to distinguish among alternative approaches to, or models of, leadership (e.g., Hallinger & Heck, 1996b). Other reviews seem to infer comprehensive assessments of leadership effects while actually limiting themselves to the behaviours associated with a particular model of leadership (e.g., Witzier, Bosker & Kruger, 2003). Furthermore, some original studies of leadership effects use secondary data sources originally created for other purposes, typically estimating the effects of potentially incoherent or incomplete models of leadership. Virtually all large-scale quantitative leadership effect studies in education restrict their attention to only part of what it is that leaders do. Such studies are usually guided by a leadership model which is intentionally reductionist in nature. The aim of this work is to assess the explanatory power of a particular set of leadership behaviours, which are associated with the chosen model. But, of course, real leaders do much more in their schools than provide, for example, instructional or transformational leadership. On occasion, almost all leaders demonstrate the use of practices associated with strategic leadership, moral leadership and the like. Such outside-the-model practices are often captured in the much wider net of qualitative case study research and this is one of the reasons for the discrepancy between the size of leadership effects reported in qualitative and quantitative studies. A review of research (Bell, Bolam & Cubillo, 2003) prepared for the British Evidencebased Policy and Practice Centre (EPPI) started with more than 4000 potential references but found that only eight met the criteria of a well-specified leadership model and empirical evidence on pupil outcomes. In addition, many leadership programme 16

evaluations neither specify nor measure the leadership practices which they aim to improve, electing instead for more global measures of participant satisfaction with the contribution of the programme to participants personal and implicit leadership efforts or espoused leadership theories. These shortcomings in the actual measurement of leadership point to the importance of clearly specifying those leadership practices which are hypothesized to effect pupil outcomes. Failure to do this arises from both practical and conceptual sources. Practically, available resources will often press researchers and evaluators to rely on existing evidence, evidence that is an imperfect match for their purposes. Conceptually, a major source of the problem is lack of agreement about the definition of leadership, as we discussed earlier. These challenges notwithstanding, subsequent improvements in our understanding of leadership effects on pupils depends, to a significant degree, on the use of reliable leadership measures explicitly based on clearly defined and conceptually coherent images of desirable leadership practices. 17

The Big Picture Leadership Effects: The Dependent Variables David Day s (2001) views on the assessment of leadership outcomes in the business sector, are useful stimulants for thinking more comprehensively about the outcomes of educational leadership. He identifies three levels of leadership outcomes which he terms financial performance, operational performance, and organizational effectiveness. Financial performance is the first and narrowest level, the equivalent in the education sector to pupil test results. The next broadest level is operational performance. Subsuming financial performance, operational performance in the business sector is a function of, for example, market share, product quality, and measures of technological efficiency. Research on educational leadership has typically conceptualized outcomes of this sort as mediating variables (discussed more full below) or mechanisms through which the influence of leaders eventually impact on pupils. Finally, at what Day terms the broadest and most subsuming level (2001, p. 388) is organizational effectiveness, the primary indicator being the health or survival of the organization. In Day s words, For those leaders at the highest organizational levels in which a system perspective is imperative for successful performance [survival] may be primarily a function of the organization s identity, image, and reputation (2001, p. 389). In the remainder of this section, we describe the educational equivalent of Day s first and third levels of leadership outcomes, leaving our treatment of the second level (mediating variables) to the next section. Level One Effects in Education: Pupil Learning In this section we consider some of the major challenges associated with determining and measuring pupil outcomes in inquiries about leadership effects. Academic achievement, as it is typically measured, is just one of several indicators of pupil outcomes. Others of a more long-term nature include graduation rates, drop out rates, and engagement in school, for example. These two sets of outcomes are quite different. Achievement measures reflect pupils skills and knowledge in a specific curriculum domain. Secondary school graduation rates, however, reflect not only specific curricular goals but also course selection decisions, course load, exam difficulty and the like. Achievement test scores are necessarily informed by pupils entire previous school careers, as well as their personal lives. Indeed, there is a strong case to be made that the important outcomes of education are, in fact, the broader and longer-term measures such as participation in further or higher education, employment, and other measures of social participation. Many people care far more about these kinds of outcomes than they do about, for example, science test scores at age 15. Moreover, broader measures tend to present fewer data problems. 18

As these arguments make clear, our current preoccupation with pupil test scores, as the dependent measure of choice in inquiries about leadership effects, is open to serious challenge. That said, the preference for assessing leadership effects on pupil test scores is not likely to go away anytime soon. In fact, Hallinger and Heck (1996a) decried the extent of use of such measures in studies of leadership a decade ago. But the press to use them has grown rather than diminished in the interim. So what are the challenges associated with this measure of pupil outcomes? While purpose-built achievement measures could be used by researchers and evaluators (although they would have their own limitations), in practice, both levels of funding and national, state or LEA policies mean that most research studies and programme evaluations end up using existing measures. These measure are typically part of national, state or LEA pupil testing programmes which have three well-known limitations as estimates of leadership effects: a narrow focus, questionable or unknown reliability, and the questionable accuracy with which they are able to estimate change over time. We have encountered a handful of less pervasive, practical limitations in some of our own recent work which we also identify in this section. Narrow focus. First, most large-scale testing programmes confine their focus to maths and language achievement with occasional forays into science. Only in relatively rare cases (e.g. Kentucky) have efforts been made to test pupils in most areas of the curriculum not to mention cross-curricular areas such as problem-solving or teamwork. Technical measurement challenges, lack of resources and concerns about the amount of time for testing explain this typically narrow focus of large-scale testing programmes. But this means that evidence of leaders effects on pupil achievement using these sources is evidence of effects on pupils literacy and numeracy. Because improving literacy and numeracy are such pervasive priorities in so many schools at the moment, this is a limitation that will not concern many researchers and evaluators. There is evidence, however, that leadership effects are of a different magnitude for even these two areas of achievement. The lesson for researchers and programme evaluators is that the size and significance of leadership effects on other areas of achievement cannot be assumed or extrapolated. Reliability. Lack of reliability at the school level is a second limitation of many large-scale testing programmes. Most of these programmes are designed to provide reliable results only for large groups of pupils. So results aggregated to national, state or LEA levels are likely to be reliable. But as the number of pupils diminishes, as in the case of a single school or even a small district or region, few testing systems claim to even know how reliable their results are (e.g., Wolfe, Childs & Elgie, 2004). The likelihood, however, is that they are not very reliable, thereby challenging the accuracy of judgments about leadership effects. Researchers and programme evaluators would do well to limit analysis of achievement to data aggregated above the level of the individual school or leader. Estimating change. Conceptually speaking, monitoring the extent to which a school improves the achievement of its pupils over time is a much better reflection of a school s 19

(and leader s) effectiveness than is its annual mean achievement scores. Technically speaking, however, arriving at a defensible estimate of such change is difficult. Simply attributing the difference between the mean achievement scores of this year s and last year s Key Stage Two pupils on the country s literacy test to changes in a school s (and/or leader s) effectiveness overlooks a host of other possible explanations: Cohort differences: This year s pupils may be significantly more or less advanced in their literacy capacities when they entered the cohort. Such cohort differences are quite common, as any teacher will attest; Test differences: While most large-scale assessment programmes take pains to ensure equivalency of test difficulty from year to year, this is an imperfect process and there are often subtle and not-so subtle adjustments in the tests that can amount to unanticipated but significant differences in scores; Test conditions differences: Teachers are almost always in charge of administering the tests and their class s results on last year s tests may well influence the nature of how they administer this year s test (more or less leniently) even within the guidelines offered by the testing agency; External environment differences: Perhaps the weather this winter was more severe than last winter and pupils ended up with six more snow days - six fewer days of instruction, or a teacher left half way through the year, or was sick for a significant time; Regression to the mean: this is a term used by statisticians to capture the highly predictable tendency for extreme scores on one test administration to change in the direction of the mean performance on a second administration. So schools scoring either very low or very high one year can be expected to score less extremely the second year, quite aside from anything else that might be different. Linn(2003) has demonstrated that these challenges to change scores become less severe as change is traced over three or four years. It is the conclusions drawn from simply comparing this year s and last year s scores that are especially open to misinterpretation. Unfortunately, it is the year over year comparisons that are most commonly made by those who report achievement results. The lesson here for researchers and programme evaluators is to use, as measures of their dependent variable, changes in pupil achievement over relatively long periods of time (three or more years). Two further limitations. While the three limitations, reviewed above, of national or state achievement data challenging researchers and evaluators have attracted considerable recent attention, we encountered two others in the course of conducting a recent leadership programme evaluation (Leithwood et al., 2003) using U.S. state (Louisiana) achievement data as the measure of programme and leadership effects. One of these limitations was changing measures over time. Ours was a five year longitudinal evaluation which was complicated by changes in the state s achievement measures. Given the frequency of policy shifts in pupil assessment practices in many jurisdictions, it may become impossible to maintain a consistent set of data over several years. Not only can the tests change, but scoring rubrics and cut-offs may also be modified. 20

Missing or incorrect data may also be a problem for researchers and evaluators. Indeed, school records of achievement data, should they be used, are quite likely to be inadequate for research purposes. Data may be missing, or coded incorrectly, or simply misplaced over the years. In our own recent evaluation, achievement data for a few schools were available for one year but not subsequent years, even though the grades to which the tests were administered were included in the schools. This reduced the number of schools available for comparison across years. A few schools changed the grade levels tested further complicating the comparability issue by the lack of data for the same grade(s) each year. Level Three Effects In Education: Toward Organizational Survival Level three effects are the long-term outcomes of schooling. Such effects reflect fairly closely, a number of the important reasons for society s heavy investment in public education. These reasons, as we have argued elsewhere (Leithwood, Aitken & Jantzi, 2002), concern both the individual welfare of students and the general public good. Public investment in primary and secondary schooling is typically justified by the assumption that it prepares people for life after secondary school employment, terteriary education and commitments to learning over the life span. So two important effects suggested by this justification are student s commitment to further education and preparation for work. The public s investment in schooling is also justified by the contributions to communities that accrue from a well educated population. These aggregate rather than individual effects include, for example, contributions to the community s economic productivity. Contributions of this sort might be gleaned from employers perceptions of how well graduates are prepared for the workplace and from claims on the part of employers that their attraction to the community and their decisions to stay can be at least partly attributed to the quality of its schools. Economic productivity is not the only indicator of schools contributions to their communities. Such contributions may also take the form of: physical facilities with non-school community organizations; sponsoring non-credit courses for adults; participation in community projects; and contributing time and skill to the political life of the community. Attitudes toward public education that accrue through the long term effects described here go some distance toward shaping public support for educational investments and through such support, influence the very survival of schooling as an institution, the ultimate effect or definition of organizational effectiveness in Day s (2001) treatment of leadership effects. The demise or at least the serious decline of public schooling was not viewed by many as a serious possibility even a short decade ago and so survival of the institution would not have been considered, at that time, a relevant long-term conception of a 21

leadership effect. But that was before the complete capture of the public sector by an ideology and ethic of accountability, one which has proven extraordinarily hostile to the monopolistic provision of public services. By now, competition from private education alternatives is quite common and public schools on both sides of the Atlantic failing to meet their performance targets are sometimes closed and reconstituted, after being placed in special measures, often with new leadership and staff. Less obvious but perhaps even more pervasive is the growth of private providers of a wide range of services to public schools and LEAs, services formerly provided within the public school system itself. This trend is viewed by its advocates as a highly desirable development within the public school sector allowing it to focus on its core competences and mission. But it could as easily be viewed as death by a thousand cuts. Wherever this trend takes us in the future, there can be little doubt that leaders in public education systems are faced with a challenge to the survival of their systems. So institutional survival is a rational long term criterion by which to judge their effectiveness. 22

Mediating Variables for Leadership Effects Research The next three sections concern leadership mediators, moderators and antecedents (see Figure 1). Evidence used in these sections largely came from three sources: a recent review of research undertaken for another project; U.K. evidence reported in the ERIC data base and two U.K. leadership journals; and evidence reported in a prestigious leadership journal reporting research carried out primarily in non-education contexts. 7 What Are They? It is commonly claimed that the effects of leadership on pupil learning are largely indirect and Figure 1 was based on this. This is most obviously the case for leadership outside the classroom the leadership provided by headteachers, for example. In order for these sources of leadership to effect pupil learning they must exercise some form of positive influence on the work of other colleagues such as teachers, as well as the status of key conditions or characteristics of the organization (school culture, for example) that have a direct influence on pupils. These people and conditions are the mediating variables in Figure 1. Leaders potentially have a direct relationship or influence on these variables and these variables, in turn, have a direct influence on pupil learning. As Baron and Kenny explain, mediating variables: represent the generative mechanisms through which the focal independent variable[e.g., leadership] is able to influence the dependent variable of interest [e.g., pupil learning] (1986, p.1173). Examples of Mediating Variables As we mentioned in the Introduction, one of our own recent reviews of literature (Leithwood, Louis, Anderson & Wahlstrom, 2004) summarized an extensive body of empirical evidence collected largely in North America about promising mediating variables for school leadership effects studies. Using results of this review as a starting 7 First, we re-examined evidence summarized in an extensive review of literature carried out for another major research project (Leithwood, Seashore Louis, Anderson & Wahlstrom, 2004). The recentness and comprehensiveness of this review allowed us to assume that it adequately represented the current state of knowledge from North American research sources. Second, in order to reflect information reported in the U.K. context about variables both mediating and moderating successful leadership, we conducted a search of relevant empirical evidence - both qualitative and quantitative - published over the past 12 years using the international ERIC data base (1976-2004) for U.K. sources only, Education-line (an exclusively British data base) and the Web of Science (U.K sources only). We also reviewed all empirical studies conducted in the U.K. relevant to leadership effects, very broadly defined, published in Educational Management, Administration and Leadership between 2000 and 2004. Finally, to augment our search for variables which moderate leadership effects, we analyzed empirical studies reported between 2000 and 2004 in The Leadership Quarterly, one of the most widely respected outlets for research on leadership conducted in non-school organizations. While limited to only one journal, we assumed that this source would include most, if not all, of the relevant variables that would have surfaced had an expanded number of comparable journals been included in our analysis. This assumption was confirmed by scanning chapters in a small number of quite recent edited texts in this field. 23

point for our present work, we then supplemented the results with evidence from the U.K. sources described above. Table 1 lists the mediating variables found in these studies, as a whole. It should be stressed that these variables were often not always conceptualized by the authors of the U.K. studies as we have conceptualized them for this paper. One of the reasons for this is that by far the majority of the U.K. studies we were able to locate were qualitative in nature and carried out using case study methodologies. Research of this sort is not often designed to explore causal relationships in as direct a manner as the overall framework for this paper indicates was our intention. Nonetheless, the decision to include a variable in Table 1 was based on evidence either that leadership had influenced it or that it had influenced pupil learning. We found no studies that had systematically tested both sets of relationships. Indeed, we found no U.K. leadership effects research explicitly guided by a framework approximating Figure 1. The left hand column of Table 1 names each of the mediating variables (four categories) and the middle column cites published evidence about each mediator collected in UK contexts. We provide here a brief explanation of what is included in each category of mediating variables. The School Conditions category of mediating variables in Table 1 includes: school structures: this variable includes school size and reflects evidence about the value of small structures in building a sense of identity with the school and increased chances for meaningful relationships between teachers and pupils. Also included is the extent to which governance is centralized or decentralized and the degree to which there are opportunities for staff to participate in school-level decision making. school culture: this mediator encompasses evidence about the impact on students of a school-wide sense of community, as well as initiatives (such as anti-racist policies) aimed at ensuring practices within the community that are equitable for all students; school philosophy or ethos: something close to the vision held by professionals in the school for what it is they aspire to for both students and the nature of their organization; organizational learning processes: these are largely self reflective and critical reflective processes on the part of teaching staff in the literature in which they appeared; school improvement processes: how school professionals go about trying to accomplish their shared goals for the school; community culture/ownership: the extent to which the community surrounding the school identifies with and feels some ownership in the school; relationships with families: the extent to which parents are considered partners with schools in the education of pupils; instructional policies and practices: concerned directly with the school s core technology, this variable encompasses school-wide policies shaping decisions about student retention and promotion, the coherence of instructional programmes, and the nature and availability of extra-curricular programmes; 24

human resources: how teacher time is allocated, as well as a wide range of working conditions that support the instructional work of teachers are part of this variable; extra-curricular activities: the types of activities in which students engage outside of classrooms but within the sponsorship of the school. Teacher recruitment and retention practices. Classroom Conditions, as Table 1 indicates, encompasses 6 specific mediators: class size: suited especially to leadership studies at the primary level, this variable reflects evidence of the effects on learning in the primary grades of small classes (approximating 15 students) when teachers adapt their instruction to take advantage of smaller classes; teaching loads: suited especially to leadership studies in secondary schools, this variable reflects evidence of a significant relationship between pupil achievement and reduced numbers of pupils for which a teacher is responsible. teaching in areas of formal preparation: also especially promising for secondary school leadership studies, this variable assumes that pupil learning is significantly associated with teachers subject matter knowledge; homework: although difficult to implement in a widely acceptable manner, this variable is built on considerable evidence of the positive effects of homework on learning, especially in combination with coordinated parent involvement; student grouping: a long-standing and extensive line of research provides the foundation for this variable; this evidence clearly favours heterogeneous over homogeneous grouping strategies; curriculum and instruction: this variable reflects the impact, on the learning of most students, of a rich curriculum, one that is organized around powerful ideas, encourages deep understanding and builds the meta-cognitive skills pupils need to become autonomous learners. There are two categories of mediating variables in Table 1 directly bearing on teachers. One concerns teachers as individuals, the other concerns teachers working together as part of a professional community. Mediating variables concerned with the individual teacher include: basic skills: especially important to measure are teachers literacy skills in primary schools; subject matter content knowledge: especially useful to include in studies of secondary school leadership effects; pedagogical content knowledge and skill; this is knowledge and skill about how to teach particular ideas or concepts; classroom experience: there is evidence underpinning this mediator of a curvilinear relationship between teachers length of experience and the effectiveness of their teaching. Experience is associated with increased effectiveness up to about five years. Effectiveness often levels off after this length of experience and may drop in the middle or later stages of a career, depending on many individual differences and circumstances; motivation: the nature and extent of commitment staff have toward pupils and their school. 25

Specific mediating variable concerning teachers professional community include: shared norms and values among teachers concerning student and teaching standards; a focus on pupil learning as the criterion for virtually all teacher decisions; deprivatized practice: an openness to colleagues viewing one s teaching and a willingness to learn from their feedback; reflective dialogue: places a high priority on inquiry and collaborative study of one s own and colleagues practices; collective professional development: the nature and extent to which professional development is pursued in collaboration by teachers; distribution of leadership; the nature and extent to which leadership in the school is shared. 26

Table 1: A Summary of U.K. Evidence about Variables which Mediate Leaders Effects on Pupil Learning Mediating Variables School Conditions Structure School culture (e.g., shared norms and values, shared decision making, collaboration) U.K. Research Barker (2001) Nicolaidou, Anscow (2002) Colwell, Hammersley-Fletcher (2004) Morris (2000) Harris, et al (2003) School philosophy or ethos Morris (2000) Organizational learning processes Devos, Verhoeven, (2003) School improvement processes Rutherford (2002) Wilson, McPake (2000) Community culture/ownership Barker (2001) Jones (2003) Harris et al (2003) Relationships with families Jones (2003) Caddell (1996) Harris et al (2003) Instructional policies/practices James, Colebourne (2004) (e.g., performance targets) Jennings, Lomas (2003) Resources (human, financial) Wilkins (2002) Extra-curricular activities Milgram (2003) Teacher recruitment and retention Farrell, Morris (2004) Classroom conditions Class size Teaching loads Formal preparation Homework Curriculum & Instruction Teachers: Individual Basic skills Content knowledge Pedagogical Content Knowledge Poulson (2001) Experience Harris et al (2003) Professional development Motivation Farrell, Morris (2004) Teachers: Collective Shared norms Focus on students Deprivatized practice Reflective dialogue Collective Pro Development Penny (2003) Durrant (2003) Distribution of leadership Wallace (2001) Adey (2000) 27

We do not claim to have uncovered all potential mediators or all of the relevant evidence about those we have identified. But we do believe that most mediators conceptually suited for leadership effects studies are evident in Table 1 and there is sufficient empirical evidence about the influence of each on pupil learning to warrant their consideration in the design of leadership effects research. 28

Variables Moderating Leadership Effects What Are They? The direct effects of leadership on mediating variables, as well as the indirect effects of leadership on pupils, is depressed, neutralized, or enhanced by key features of the situation or context in which leadership is exercised. For example the same leadership practice may have quite different effects on teachers depending on their gender, age, amount of experience or levels of stress; so these are potential moderating variables. As Baron & Kenny explain: Moderator variables are typically introduced into a study when] a relation holds in one setting but not another, or for one subpopulation but not another (1986, p. 1173). Since moderating variables help explain how or why certain effects will hold, the careful selection of moderating variables is a key step in designing leadership effects research and one that has been badly neglected in educational leadership research to date. The consequences of this neglect can be easily illustrated. Suppose, for example, that we design a study to assess the effects of leaders goal-setting practices on teachers organizational commitment. Reflecting evidence from previous research, we might decide that teachers sense of professional self-efficacy is a key mediator of the effects of goal-setting practices on such commitment. So we measure it. But suppose goal-setting practices only influence teachers sense of efficacy in a positive direction when trust in leaders is high. Unless we measure trust in leaders, as well, it is likely that at least some subsequent studies will disconfirm our results. They will do so because the different teacher samples in those subsequent studies will vary in unknown ways with respect to their levels of trust in leaders. As this example makes clear, inadequate attention to moderating variables is one of the more plausible explanations for contradictory research findings and the general scepticism that often follows about the potential of research to provide clear guidance for policy and practice. This charge, it should be noted, is in no way unique to large-scale quantitative research, which at least has a tradition of worrying about such matters. Case study, qualitative, research is especially well suited to the task of surfacing promising moderators for subsequent consideration. 8 A final point of clarification about moderators concerns the basis on which a variable is assigned moderator status. Indeed, the same variable might be defined moderator, mediator or dependent variable status depending entirely on the theory or framework used to guide a leadership effects study. For example, employee trust was used above as an example of a moderating variable in research concerned with the effects of 8 Qualitative research should not be considered exempt from the obligation of providing evidence about moderating variables, even though some of its advocates eschew even the idea of causal relationships and the concept of dependent, independent and mediating variables. 29

leadership on teachers efficacy. But trust is viewed as a dependent measure in leadership studies when researchers are curious about the forms of leader behaviours which promote its development (e.g., Kouze & Posner, 1995). Trust is also conceived of as a mediating variable in studies, for example, concerned with the effects of leader behaviours on employees acceptance of decisions (Tyler & Degoey, 1996). The theory- driven nature of moderator designation means that the examples of moderating variables outlined next in this section should be understood as an illustration of what is of theoretical interest to leadership researchers not as a category of unique variables. Examples of Moderating Variables The most frequently used moderating variables in studies of leadership effects on pupils are pupil and family background characteristics. Indicators of wealth (e.g., pupil eligibility for free or reduced lunch at schools) and/or socio-economic status (e.g., parental occupation) and/or minority status are typically used to represent these variables. But other moderating variables are evident in recent educational leadership effects research. For example, Leithwood and his colleagues used family educational culture as a moderating variable on the grounds that it is the feature of pupil and family backgrounds that most directly influences the learning of pupils (Leithwood, Jantzi & Steinbach, 1999; Leithwood & Jantzi, 1999). Marks and Printy (2003) incorporated into their study, as moderators, classroom compositional variables related to pupil gender and ethnicity. Another possible moderator is, school stakeholders including, for example, unions, community and business groups, and the media. Parents might also be included here, depending on the role they assume. A recent review of evidence about stakeholders suggested that:... many stakeholder groups have a direct or indirect interest in schools and school leadership, and commission reports on the state of education have lamented the lack of involvement of stakeholders in decisions that affect them... There is, however, little research on how these groups affect the work of superintendents and principals. Certain themes are evident in the practitioner literature, most of which look at the ways in which stakeholders block or impede the work of school leaders, or point to ways in which their volunteer energy can be corralled to improve the work of schools (Leithwood, et al., 2004, p. 23) While evidence about other stakeholder effects is quite limited, we believe that closer attention to this moderator is warranted in future efforts to assess leaders impact on pupils. The external accountability requirements that are part of the current policy context for most educators seem likely to have made them especially sensitive to the views of such stakeholders. These views seem likely to influence, for example, the conditions teachers create in their schools and classrooms and the resources that are available to them, quite apart from the initiatives of those in leadership roles. 30

To extend our illustrative menu of possible moderators for future research beyond those typically found in our original North American sources we reviewed the same published U.K. sources used to identify mediating variables. In addition, we analyzed all issues of the 2000 to 2004 volumes of The Leadership Quarterly; this journal reports research conducted mostly in non-school contexts. As Table 2 indicates, our review of empirical studies carried out in the U.K. and studies published in The Leadership Quarterly identified five categories of moderators (twentyone specific moderators) that may warrant additional attention in future studies of educational leadership effects on pupils. The first category of moderators, Pupils, includes a range of features associated with pupils family background, family educational culture and pupils gender and ethnicity. In the second category, Teachers 9, is included gender, formal education and tenure (age and experience have been included here). Also part of this category are teachers ethnicity, their beliefs and values, morale, trust and the confidence they have in their leaders and their leadership prototypes. 10 Two characteristics of Leaders themselves are identified in Table 2; their gender and their level in the organizational hierarchy Table 2 identifies five features of the Organization which may moderate responses to leadership practices, including school size, what it is people are rewarded for, and opportunities for job enrichment. The difficulty of the tasks people are expected to accomplish, the interpersonal dynamics among people in the organization, and the availability and use of information in decision making are also identified as moderators in this category. Finally, in the category Organizational context, the nature of other stakeholders in the school or district and their relationships with the school, as well as the policy environment in which the organization finds itself, moderate the amount and nature of leaders influence. 9 We exclude from this category teachers in leadership roles. In the non-school leadership research literature, the equivalent category label would be employees or followers. 10 The citation in Table 2 to Lord and Maher is not found in the issues of The Leadership Quarterly that we reviewed. But leadership prototypes have gained extensive attention in other publications and should not go unnoticed, in our view. Leadership prototypes are the mental models people have developed for their meaning of leadership. Such prototypes serve as the source of criteria used by people in judging whether or not someone is exercising leadership and the extent to which that leadership is desirable. 31

Table 2: A Summary of U.K. and The Leadership Quarterly Evidence About Variables Which Moderate Leaders Effects Moderating Variables U.K Research The Leadership Quarterly Pupils Background, mobility, social identity, class Mac an Ghaill (199) Jones (2003) Strand (2000) Smith, Hardman, Mroz (1999) Jones (2003) Harris et al (2003) Family educational culture Caddell (1996) Jones (2003) Gender Ethnicity Gillborn (1997) Graham, Robinson (2004) Teachers (or followers ) Gender Moyo-Robbins (2002) Neubert, Taggar (2004) Vecchio, Boatwright (2002) Antonakis, Avolio & Sivasubramaniam (2003) Study 1&2 Formal education Vecchio, Boatwright (2002) Tenure/age/experience McLay, 2003 Vecchio, Boatwright (2002) Moyo-Robbins (2002) Ethnicity Moyo-Robbins (2002) Beliefs &values Jones (2003) McLay (2003) Morale Trust/confidence in leader Oduro (2004) Leaders prototypes Lord, Mayer (1993) Leaders Gender Neubert, Taggar (2004) Vecchio, Boatwright (2002) Antonakis, Avolio & Sivasubramaniam, (2003) Study 1&2 Hierarchical level Organization Size Antonakis, Avolio & Sivasubramaniam, (2003) Study 1&2 Day, Hadfield, Harris Koene et al (2002) (1999) Reward structure Moyo-Robbins (2002) Kahai, Sosik & Avolio (2003) Job enrichment opportunities Whittington, Goodwin & Murray, (2004) Goal or task difficulty Whittington, Goodwin & Murray(2004) Interpersonal dynamics/norms Johnston (1997) Pepin (2000) Availability/use of Olson, Davidson (2003) information technology Organizational context Stakeholders Selwyn (2001) Policy environment Radnor, Ball, Vincent (2002) Leithwood et al (2004) Antonakis, Avolio & Sivasubramaniam (2003) Study 1&2 32

The Antecedents of Leaders Practices What Are They? Antecedents may be internal or external to the leader and these two sets of antecedents are interdependent. By this we mean that the extent and nature of influence of an external antecedent depends on what sense is made of it internally and the importance the leader attaches to it as a stimulus for their own behaviour. The influence of external antecedents, in other words, is constructed from the internal cognitive and emotional resources of the individual leader. Put simply, what leaders do depends on what they think and how they feel. The dotted line joining antecedent and moderating variables in Figure 1 acknowledges that one study s antecedents may be another study s moderators; these may be theoretically defensible differences. Policy context is an example of a variable that might be either an antecedent or a moderator. If the research question is: What is the impact of accountable policy contexts on the frequency with which principals display transformational leadership behaviours? then policy context is an antecedent. But if the question is: To what extent do accountable policy contexts enhance or depress the impact of transformational leadership behaviours on the development of collaborative school cultures? then policy context is a moderator. The theory-driven nature of how variables are classified is a point worth a bit more attention here. The same variable actually might be assigned antecedent, moderator, mediator or dependent variable status depending entirely on the theory or framework used to guide a leadership effects study. Employee trust is an example of a moderating variable included in research about the effects of leadership on teachers efficacy, for example. However, trust is viewed as a dependent measure in leadership studies when researchers are curious about the forms of leader behaviours which promote its development (e.g., Kouze & Posner, 1995). Trust is also conceived of as a mediating variable in studies, for example, concerned with the effects of leader behaviours on employees acceptance of decisions (Tyler & Degoey, 1996). Examples of Antecedents While our main interest in this paper is in external antecedents, especially leadership programmes, others external antecedents for which there is evidence include, for example, on-the-job learning, socialization processes and early family experiences. A recent review of empirical research on transformational school leadership (Leithwood & Jantzi, 2005) also found evidence that organizational bureaucracy, organizational values, and school reform initiatives influenced the development of transformational leadership practices, as well as formal training experiences. For example, three studies provided information about how school reform initiatives influence transformational leadership practices. Creating uncertainty and introducing competition into an otherwise conservative organizational culture, school-based management was the reform initiative examined by Eyal & Kark (2004) using data from teachers in 140 Israeli elementary 33

schools. The highest levels of transformational leadership were associated with moderate (rather than vigorous or conservative) levels of organizational innovativeness prompted by increased competition. This study also demonstrated a significant relationship between principal proactivity and the use of transformational practices. Both Ross (2004) and Leithwood et al (2004 b) found evidence that two substantially different formal leadership experiences had significant effects on the development of transformational leadership behaviours among school principals. The training programme in Ross s study extended over four sessions (one full day and three half days), was conducted with principals and a team of their teachers and aimed to indirectly improve students reading and writing achievement by directly changing teachers assessment practices. Leithwood et al s study was based on a five-year longitudinal evaluation of the effects of a leadership centre programme which provided a variety of sometimes intense experiences for principals over several years both inside and outside their schools. Both studies reported significant positive effects of the training experiences on principals transformational leadership as well as student achievement. 34

Evaluating Leadership Programme Effects Building from Figure1, and concerned specifically with leadership programmes, Figure 2 captures the range of possible expectations for theoretically framing the evaluation of programme effects. As this figure indicates, there are six discrete models for such evaluation, along with many additional hybrids. Each of these models is distinguished by its choice of dependent variable and the number of additional variables mediating the effects of leadership programmes and leadership practices on that dependent variable. This framework is very consistent with Guskey s (2000) general model for evaluating professional development programmes. Guskey s model consists of five levels of data to be collected: participants reactions; participants learning; organization support and change; participants use of new knowledge and skills and pupil learning outcomes. Models 1, 2 and 3 are direct effects models. They propose no mediating variables between a leadership programme and either participants satisfaction (model 1) or participants internal processes (knowledge, skill, dispositions model 3); in the case of model 2, the qualities and features of the programme are assessed against a set of ideal features drawn from previous research and/or professional judgment. Model 1 is the simplest, least valuable, but most commonly used of the six alternatives. Model 2, a relatively recent alternative, has emerged with accumulations of evidence about the characteristics of effective programmes; both the University Council for Educational Administration (Peterson, 2001) and the Texas Principal Preparation Network (Eddins, 2002). Reflecting UCEA s perspective, for example, Peterson (2001) argues that programmes must have; a clear mission and purpose linking leadership to school improvement; a coherent curriculum that provides linkage to state certification schemes; and an emphasis on the use of information technologies. He also suggests that programmes should be continuous or long-term rather than one-shot, and that a variety of instructional methods should be used rather than relying on one or a small set of delivery mechanisms. 35

2 Qualities of Effective Programs Participant Satisfaction Leadership Preparation Experiences Changes in Knowledge, Skills, Dispositions Changes in Leadership Practices in Schools Improved Student Outcomes 1 Participant Satisfaction 3 Altered Classroom Condition 4 5 6 Figure 2: Alternative Frameworks to Guide the Evaluation of Leadership Programs Evaluations guided by models 3 and 4 are, in many respects, highly defensible responses to the outcomes usually demanded of education programmes in most fields. That is, programmes change the capacities and/or actual practices of participants. Indeed, it is reasonable to argue that, given the methodological difficulties associated with models 5 and 6, that this ought to be viewed as the near-term standard for evaluating leadership preparation programmes. Very few examples of such evaluation can be found, at present. One such example is Leithwood et al s (1996) summative evaluation of 11 universitybased programmes sponsored by the Danforth Foundation. With Wallace Foundation support, Darling-Hammond (2004), and her colleagues are just now beginning a second such example. These first three models may be parts of the more complex models 4 through to 6 and model 6 potentially subsumes all the others. In case of model 4, the criterion variable for judging leadership programme effects is change in programme participants actual practices in their organizations. Model 5 requires, in addition, evidence that such leadership practices lead to desirable changes in school and classroom conditions. Finally, model 6 expects everything the criterion for judging a leadership programme successful is that the students in the participants schools learn more. By mixing and matching, other hybrid models are possible, although we do not have much to say about them in this paper. For example, one could easily create a direct 36

effects model in which only leadership programmes and/or leadership practices and student outcomes were linked. But the likelihood of such a model detecting changes in pupils learning due to either experience in a leadership programme or only changes in leaders practices is remote (Hallinger & Heck, 1996). School Leadership Center Programs Participants Internal Processes Participants Leadership Practices in their Schools School and Classroom Conditions! Mission/Goals! Culture! School Planning! Instruction! Structure/ Organization! Information collection/ decision making! Policies and procedures Student Participation and Engagement Student Achievement Figure 3: Framework Guiding the Evaluation of the Greater New Orleans School Leadership Center Programmes Three recent studies with which we are associated provide more specific illustration of the range of alternatives within model 6. Figure 3 summarizes the framework used to guide a recently completed, five-year evaluation of an annual series of development initiatives provided for a selected set of practicing school principals in the greater New Orleans, Louisiana, region of the United States (Leithwood et al, 2003). In this case, internal processes were primarily cognitive in nature and a transformational model (after Leithwood, Jantzi, & Steinbach, 1999) was used to conceptualize leadership practices. Variables in the school and classroom, influenced by organizational design theory, were derived from evidence reported in Leithwood and Aitken (1995). Two sets of student outcomes served as dependent variables in this evaluation student participation and engagement in class and school, and achievement as measured by the state s annual tests. 37

1 State Leadership, Policies and Practices e.g. standards testing funding 3 Student/Family Background e.g. family educational culture 6 School Conditions e.g. culture/community school improvement planning 2 District Leadership, Policies and Practices e.g. standards curriculum alignment use of data 4 School Leadership 7 Teachers e.g. individuals capacity professional community 10 Student Learning 9 Leaders Professional Learning Experiences e.g. socialization mentoring formal programmes 5 Other Stakeholders e.g. unions community groups business media 8 Classroom Conditions e.g. content of instruction nature of instruction student assessment Figure 4: Framework for Wallace-supported Leadership Study A second illustration is a Wallace Foundation supported study presently in the second of its five-year duration (Leithwood, Seashore Louis, Wahlstrom & Anderson, 2003). This is a research rather than evaluation project and the framework guiding data collection is summarized in Figure 4. This framework is based on an extensive review of policyoriented literature by the principal investigators linking district and school leadership to pupil learning. The study itself inquires about the contribution of leaders formal and informal professional learning experiences to their leadership practices, among many other things. 38

In this study, both school and district leadership is conceptualized as a combination of (potentially distributed) practices useful in all contexts, as well as additional practices especially helpful in both the contexts of outcomes-oriented school accountability policies and highly diverse school communities (Leithwood & Riehl, 2002). This framework explicitly accounts for student and family background variables, as well as the influence of other stakeholders on school leaders practices. Mediating school leader effects on pupil learning are a series of classroom and school conditions not unlike those found in the New Orleans study. However, teachers (their capacities, communities, etc) are distinguished as a separate set of variables in this study. State and district tests will be among the measures of student achievement, the dependent variable in this study. A third illustration is embedded within the recently completed external evaluation of England s literacy and numeracy strategies (Earl et al, 2003). This was a sub-study of school leadership effects on changes in classroom practices and pupil learning (Leithwood & Jantzi, in press); Figure 5 summarizes the framework for this sub-study. As in the two previous examples, a transformational model was used to conceptualize leadership practices which might be distributed across several sources within the school. Mediating leadership effects on classroom practices and learning were a set of teacherrelated variables - motivation, capacity, work setting - based on a model of workplace performance from industrial psychology (Rowan, 1996) but adapted and extended for use in this study (Leithwood, Jantzi, & Mascall, 2003).This study was not intended to address issues of leadership preparation but the framework easily extended for that purpose, illustrates yet another model 6 possibility. Teachers Motivation School Leadership Practices Teachers Capacity Teachers Classroom Practices Student Learning Teachers Work Setting Figure 5: Framework Guiding UK External Evaluation 39

Methodological Challenges Earlier sections of this paper have identified some of the challenges of measuring both leadership and student outcomes. Evaluators of leadership programmes, and researchers inquiring about leadership effects, face additional thorny methodological challenges in their efforts to demonstrate programme and leader effects on pupil outcomes, in particular. We describe six such challenges in this section of the paper. The first four methodological challenges are based on the direct experience of Leithwood and his colleagues (2003) during the evaluation of the New Orleans programme described above. These challenges arose from the attempt to use state test data as the basis for assessing programme and leader effects on student achievement; data of this sort often will be the cheapest and most accessible achievement measures for evaluators and researchers. Leithwood et al (2003) describe their four challenges to illustrate what they believed ought to be the appropriate standards of evidence for evaluating leadership programme effects on pupil learning plausible evidence of effects - along with a convincing explanation for such effects. Certainty of effects, they claimed, is an unrealistic standard for programme evaluations. In the case of the New Orleans evaluation, while achievement data available was available through state sources, at least five important limitations on their use had to be addressed. School Organisation Considerable variation in the organisation of programme participants schools meant that there were very small numbers of students available for analysis within any one type of organization. Few schools had identical sets of data. Across participants schools, at least eleven different grade configurations could be found. None of these configurations included enough schools to carry out detailed analyses by configuration using the same measures. For example, a school mean for the Iowa achievement test was calculated to develop a gain score from 2000 to 2002. The number of grades contributing to the school mean varied across schools from one to five. This raised questions about the comparability of, for example, a mean score for grades 3 and 5, a mean score for a single grade 3, or a mean score for a grade 6 and 7 combination. But limiting comparisons only to identical grades would have virtually ruled out most analyses, particularly for schools within the same participants cohort. School Type A mix of state or public (82%) and parochial schools (18%) in the New Orleans sample meant that approximately one fifth of the schools did not have results for two of the main tests used by the state. Data for parochial schools could only be obtained directly from the schools, some of which are not able/willing to provide data without more encouragement than the external evaluator managed to provide. However, even if data had been provided, the small number of such schools would have permitted only the simplest form of analyses. 40

Changing Measure Over Time Comparison across the five years in the longitudinal New Orleans evaluation (this includes the baseline measures before the project began) was complicated by the lack of consistency in the achievement measures used and reported on the state department of education web site. Missing Data for Individual Schools. Achievement data for a few schools were available for one year but not subsequent years, even though the grades to which the tests were administered were included in the schools. This reduced the number of schools available for comparison across years. A few schools that changed the grade levels tested complicated the comparability issue by not having data for the same grade(s) each year. Determining the Unit of Analysis The New Orleans evaluation used the school as the unit of analysis. However, determining what the unit of analysis should be is a fundamental challenge for any social science research, one that is often not thought through adequately in study design. In one sense, leadership development experiences are expected to have an impact on individual leaders who in turn have an impact on other individuals and on entire organizations. Impact on individual leaders is relatively easy to assess. Usually this is done through attitude or opinion measures administered to leaders themselves, a largely unsatisfactory form of evidence. But this evidence tells us little or nothing about their practices or their impact since we know that people s self-reports of behaviour may be quite different from the way their behaviour is perceived by others. Alternatively, it is certainly possible to assess leaders acquisition of knowledge and skill in variety of quite direct ways using tests, case problems, simulations and the like (e.g., Leithwood & Steinbach, 1995). Whatever the form of evidence about changes in leader practices, however, such change, as Figure 2 makes clear, is only part of the chain of variables measured through more sophisticated evaluation designs. Improving leadership only matters, some would argue, if the result is improved organizational performance. As well, if one embraces a distributed model of leadership then a focus on individuals may hide important contributions to development in the school made by others. Some analysts argue that we focus too much on people in positions of authority and fail to see the way in which key leadership functions in schools may be exercised by many different people. Most survey-based research on the nature of school leadership uses data collected from teachers about their perceptions of the leadership provided by administrative leaders (e.g., Day et al, 2002; Leithwood & Jantzi, 2002). Such evidence is a considerable improvement on leader self-report data and is consistent with the theoretical claim that leadership is an attribution. 41

There are also problems with using entire schools as the unit of analysis for leadership impact. A school model assumes that leadership impacts are direct and evenly distributed, but as our earlier discussion shows, neither of these assumptions is necessarily correct. Take the issue of staff mobility. One reasonable expectation for good leadership is that staff stability should be high, since many changes in staff are likely to make it more difficult to improve learning and teaching. However changes in leadership are associated with increased turnover of staff as leaders recruit new staff who are more aligned with the leader s values. A further difficulty with using school-level results as the measure of leadership effectiveness relates to the situational nature of leadership. If we want to assess the impact of leadership in situ, then presumably we need a definition of leadership that is precise enough so that we can recognize it. Yet we also know that many factors shape school outcomes, and that these factors also may affect the nature of leadership. For example, presumably one important task of school leaders is to recruit and retain good teachers, but that task may be much more difficult in settings that also face other constraints to good outcomes, such as geographical isolation, high poverty or high ethnic diversity. Many other aspects of leader activity, such as strategies for involving parents, relations with students and approaches to curriculum may also vary considerably within schools. If this is so, how is it possible to assess leadership impact across settings? Two strategies are possible. The most commonly used strategy is simply to rely on an average impact score. While this doesn t tell us much about the areas of low and high impact in a school, it does tell us the overall level of impact which may be sufficient for some evaluation purposes; indeed, most of the quantitative evidence we now have of leadership impact is based on this strategy. A second strategy used quite extensively in leadership research, is to start with schools judged to be exemplary on the basis of student outcomes and then determine what leaders do. Even better is a design that compares outliers exceptionally high vs low scoring schools. While these designs have some value as research strategies, they are of no value for programme evaluation purposes. One cannot pick leaders and schools for evaluation; the programme picks them for you. Research design Statistical models used in quantitative research on leader effects typically some path analytic technique (e.g. LISREL) are capable of measuring both leaders direct effects on pupil learning, as well as their indirect effects through influence on school and classroom conditions which, in turn, also influence pupil learning. Statistical models of this type are extremely useful in the evaluation of leadership programmes, beyond models 1 to 3, in those vast majority of evaluations in which only data related to participants, their schools and students can be collected. 42

The use of experimental or quasi-experimental designs would create other opportunities, something we discuss at some length in our third paper. Essentially, such opportunities consist of tradeoffs: for example, tradeoffs between the theoretical complexity (and explanatory power) of a mediated effects model and the difficulties of either random assignments in real world conditions or locating comparison groups sufficiently similar to the leadership preparation group to rule out competing hypotheses to the programme experience as explanation for any observed differences in dependent measure. 43

Conclusions and Recommendations Our paper described challenges associated with conceptualizing the relationship between leadership programmes, changes in leader practices and the effects of such changed practices on schools and pupils. We also grappled with some of the methodological challenges facing evaluators and researchers with an interest in programme and leader effects, offering suggestions about how these challenges might be addressed. Four conceptual challenges were addressed in the paper: How to most usefully frame the relationship between leadership programmes, leadership practices and pupil outcomes? We offer a relatively generic response to this question, suggesting that any comprehensive framework would include independent, dependent, antecedent, moderating and mediating categories of variables. A review of research illustrates what we know about each category of variables and the relationships among them. Frameworks guiding a handful of current and recently completed studies illustrate variations on the generic framework; How the nature of leadership practices might be conceptualized and what intellectual resources are available to assist in such conceptualization? We review a wide range of alternative leadership models to assist in thinking about this challenge; these are models developed in both school and non-school organizational contexts; What can leadership theory and research developed outside of schools contribute to our understanding of school leadership practices and their effects? Building on our earlier description of leadership models in non-school contexts, we identified perspectives on leadership yet to be explored in school-based leadership research but with promising potential. How to define and what do we know about - dependent, moderating and mediating variables in school leader and leadership programme effects studies? We define and illustrate the state-of-the-art of knowledge about these variables through a review of leadership research carried out exclusively in U.K. contexts. It is clear from this review that most recently published U.K. leadership research is almost exclusively small scale and qualitative in nature. Although not as extensive as our treatment of conceptual challenges, the paper also responded to a series of methodological challenges typical of field - based leadership research. One of problems we took up in the paper is the measurement of leadership practices; we examined a small set of instruments commonly used for this purpose. We also discussed such common difficulties in conducting both programme and leadership effects research as the narrow and unreliable nature of commonly used student assessment instruments and how to deal with missing data for individual schools. The paper presents evidence that, while the school is the unit of analysis in much leadership effects research, there is typically greater within - than across - school variation in measures of leadership an important, largely unaddressed, challenge for future research. Promising approaches to the evaluation of leadership programme effects are also outlined. 44

Our conclusions and recommendations bear on selected aspects of future DfES - sponsored research and evaluation that might be carried out in the United Kingdom on both leadership, and leadership programme, effects. Future leadership effects research should, in our view: measure a more comprehensive set of leadership practices than has been included in most research to date: these measures should be explicitly based on coherent images of desirable leadership practices. Such research is likely to produce larger estimates of leadership effects on pupil outcomes than has been provided to date; measure an expanded set of dependent (outcome) variables: these are variables beyond just short-term pupil learning including, as well, longer term effects. Examples of such long - term effects include pupil success in tertiary education, employment and commitment to learning over the life span. systematically describe how leaders successfully influence the condition of variables mediating their effects on pupils: we now have considerable evidence about what are the potentially most powerful variables mediating school leader effects but we know much less about how leaders influence these mediators; attend more systematically to variables moderating (enhancing, reducing) leadership effects: lack of attention to this category of variables seems likely to be a major source of conflicting findings in the leadership research literature. Furthermore, when studies do attend to moderators, their choice has often been difficult to justify and largely atheoretical. reflect greater methodological variety than is evident in recently published U.K. leadership research: this will be essential if a robust body of context - relevant knowledge is to be developed. Future leadership programme effects research should: be guided by conceptual frameworks similar to those we have recommended for leadership effects research: our review suggests that the vast majority of previous efforts to evaluate leadership programme effects, in most parts of the world, have not generated the type and quality of evidence required to confidently answer questions about their organizational or pupil effects. This problem could be addressed by introducing, into the conceptual frameworks we have suggested for leadership effects studies, leadership programmes conceptualized as one category of antecedent variables stimulating changes in leadership practices. provide comparative information about programme effects: formal programmes are just one of many influences on leaders practices; to fully appreciate the value of such programmes, their effects need to be compared to the effects of such other antecedents as on-the-job learning, leaders traits and early family experiences, for example. Such information would inform not only programme improvement efforts but leadership selection processes, as well. Information of this sort would also assist with cost-effectiveness judgements in the context of planning for leadership development. be funded at levels consistent with the expectations for what is to be accomplished: few documented programme evaluations provide the type of comprehensive data we call for here and funding is part of the reason. If future leadership programme evaluations are to assess the direct and indirect effects of such programmes on pupil 45

learning as well as leaders practices, then a different level of funding will be required than has been typical to date. Implementing these recommendations will require considerable attention to a significant number of conceptual and technical issues at the core of designing and conducting high quality, high impact research and evaluation. Just doing more research of the type that is typically being done in the country now - at least as it is reflected in the published literature - seems unlikely to significantly advance our understandings about how leadership programmes and leaders most productively improve pupil learning. 46

References Adey, K. (2000). Professional development priorities: The views of middle managers in secondary schools. Educational Management and Administration, 28(4), 419-431. Antonakis, J., Avolio, B., & Sivasubramaniam, N. (2003). Context and leadership: An examination of the nine-factor full-range leadership theory using the Multifactor Leadership Questionnaire. The Leadership Quarterly, 14(3), 261-295. Antonakis, J., Cianciolo, A. T., & Sternberg, R. J. (Eds.). (2004). The nature of leadership. Thousand Oaks, CA: Sage Publications. Avolio, B., & Yammarino, F. (2002). Reflections, closing thoughts, and future directions. In B. Avolio & F. Yammarino (Eds.), Transformational and charismatic leadership: The road ahead. Oxford: Elsevier Science Ltd. Barker, B., & Busher, H. (2001). The nub of leadership: Managing the culture and policy contexts of educational organizations. Paper presented at the British Educational Research Association Annual Conference, Leeds. Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic and statistical considerations. Journal of Personality and Social Psychology, 51(6), 1173-1182. Bass, B. M. (1985). Leadership and performance beyond expectations. New York: The Free Press. Begley, P., & Johansson, O. (Eds.). (2003). The ethical dimensions of school leadership. Dordrecht: Kluwer Academic Publishers. Begley, P., & Leonard, P. E. (Eds.). (1999). The values of educational administration. London: Falmer Press. Bell, L., Bolam, R., & Cubillo, L. (2003). A systematic review of the impact of school headteachers and principals on student outcomes. Retrieved March 20, 2004, from http://eppi.ioe.ac.uk/eppiwebcontent/reel/review_groups/leadership/ lea_rv1/lea_rv1.pdf Bennis, W., & Nanus, B. (1985). Leaders: The strategies for taking charge. New York: Harper & Row. Blake, R. R., & Mouton, J. S. (1964). The managerial grid. Houston, TX: Gulf. Brown, D., & Lord, R. (1999). The utility of research in the study of transformational and charismatic leadership. The Leadership Quarterly, 10(4), 531-539. Bryman, A. (2004). Qualitative research on leadership: A critical but appreciative review. The Leadership Quarterly, 15(6), 729-769. Caddell, D. (1996). Roles, responsibilities and relationships: Engendering parental involvement. Paper presented at the Scottish Educational Research Association Conference, Dundee. Caldwell, B. J. (2000). Leadership in the creation of world-class schools. In K. A. Riley & K. S. Louis (Eds.), Leadership for Change and School Reform. London: Routledge Falmer. Campbell, D., & Stanley, J. (1963). Experimental and quasi-experimental designs for research. Boston, MA: Houghton Mifflin. Chatterji, M. (2004). Evidence on "what works": An argument for extended-term mixed method (ETMM) evaluation designs. Educational Researcher, 33(9), 3-13. 47

Clark, K. E., & Clark, M. B. (Eds.). (1990). Measures of Leadership. West Orange, NJ: Leadership Library of America, Inc. Clark, M., Walter, H., & Madaus, G. (2000). High stakes testing and high school completion. NBETPPS Statements, 1(3).Chestnut Hill, MA: National Board on Educational Testing on Public Policy. Collins, D. (2002) Performance-level evaluation methods used in management development studies from 1986 to 2000. Human Resource Development Review 1(1), 91-1110. Colwell, H., & Hammersley-Fletcher, L. (2004). The emotionally literate primary school. Paper presented at the British Educational Research Association Annual Conference, Manchester. Cook, T.D. (1991). Clarifying the warrant for generalized causal inferences in quasiexperimentation. In M. McLaughlin & D. Phillips (Eds.), Evaluation and education at quarter century (pp. 115-144). Chicago: National Society for the Study of Education. Conley, S. (1991). Review of research on teacher participation in school decision making. In G. Grant (Ed.), Review of research in education. Washington, DC: American Educational Research Association. Creemers, B. P. M., & Reezigt, G. J. (1996). School level conditions affecting the effectiveness of instruction. School Effectiveness and School Improvement(7), 197-228. Dansereau, F., & Yammarino, F. (Eds.). (1998). Leadership: The multi-level approaches. Stamford, CT: JAI Press. Dansereau, F., Yammarino, F., & Markham, S. (1995). Leadership: The multiple-level approaches. The Leadership Quarterly, 6(3), 251-263. Davies, B. (Ed.). (2004). School Leadership and Management, 24(1). Day, C., Hadfield, M., & Harris, A. (1999). Leading schools in times of change. Paper presented at the European Conference on Educational Research, Lahti, Finland. Day, C, Harris, A, Hadfield, M. (2001)(a). Grounding knowledge of schools in stakeholder realities: a multi-perspective study of effective school leaders, School Leadership and Management, 21, 1, 19-42. Day, C, Harris, A, Hadfield, M. (2001)(b). Challenging the orthodoxy of effective school leadership, International Journal of Leadership in Education, 4, 1, 39-56. Day, D. V. (2001). Assessment of leadership outcomes. In S. J. Zaccaro & R. J. Klimoski (Eds.), The nature of organizational leadership: Understanding the performance imperatives confronting today's leaders (pp. 384-410). San Francisco: Jossey- Bass. Day, D. V., & Lord, R. G. (1988). Executive leadership and organizational performance: Suggestions for a new theory and methodology. Journal of Management, 14, 453-464. Devereaux. (2004). Merging role-negotiation and leadership practices that influence organizational learning. University of Toronto, Toronto. Devos, G., & Verhoeven, J. C. (2003). School self-evaluation - Conditions and caveats: The case of secondary schools. Educational Management and Administration, 31(4), 403-420. Dror, Y. (1986). Policymaking under adversity. New Brunswick, NH: Transaction Press. 48

Earl, L. Watson, N., Levin, B., Leithwood, K. Fullan, M., and Torrance, N. (2003). Watching and learning 3: Final report of the OISE/UT evaluation of the implementation of the National Literacy and Numeracy Strategies. Prepared for the Department for Education and Skills, England. Toronto: OISE/University of Toronto. www.standards.dfes.gov.uk/literacy/publications. Evers, C., & Lakomski, G. (2000). A plea for strong practice. Educational Leadership, 61(3), 6-10. Durrant, J. (2003). Teachers leading learning. Paper presented at the British Educational Research Association Annual Conference, Edinburgh. Farrell, C., & Morris, J. (2004). Resigned compliance: Teacher attitudes towards performance-related pay in schools. Educational Management Administration and Leadership, 21(1), 81-104. Finn, J. D. (1989). Withdrawing from school. Review of Educational Research, 59(2), 117-143. Fredricks, J., Blumenfeld, P., Paris, A., (2004). Student Engagement: Potential of the Concept, State of Evidence, Review of Educational Research, 74, 1, 59-109. Gezi, K. (1990). The role of leadership in inner-city schools. Educational Research Quarterly, 12(4), 4-11. Gillborn, D. (1997). Ethnicity and educational performance in the United Kingdom: Racism, ethnicity and variability in achievement. Anthropology and Education Quarterly, 28(3), 375-393. Graham, M., & Robinson, G. (2004). The silent catastrophe: Institutional racism in the British educational system and the underachievement of black boys. Journal of Black Studies, 34(5), 653-671. Gronn, P. (2002). Distributed leadership. In K. Leithwood & P. Hallinger (Eds.), Second international handbook of educational leadership and administration (pp. 653-696). Dordrecht: Kluwer Academic Publishers. Gronn, P. (2000). Distributed properties: A new architecture for leadership. Educational Management and Administration 28(3), 317-338. Guskey, T. R. (2000). Evaluating Professional Development. Thousand Oaks, CA: Corwin Press. Hallinger, P. (2003). Reflections on the practice of instructional and transformational leadership. Cambridge Journal of Education. Hallinger, P., & Heck, R. (1996). The principal's role in school effectiveness: An assessment of methodological progress, 1980-1995. In K. Leithwood & et al. (Eds.), International handbook of educational leadership and administration (pp. 723-783). Dordrecht: Kluwer Academic Publishers. Hallinger, P., & Heck, R. (1996). Reassessing the principal's role in school effectiveness: A review of empirical research, 1980-1995. Educational Administration Quarterly, 32(1), 5-44. Hallinger, P., & Murphy, J. (1985). Assessing the instructional management behavior of principals. Elementary School Journal(86), 217-247. Hallinger, P., & Heck, R. H. (1996a). Reassessing the principal s role in school effectiveness: A review of empirical research, 1980-1995. Educational Administration Quarterly, 32(1), 5-44. 49

Hallinger, P., & Heck, R. H. (1996b). The principal s role in school effectiveness: A review of methodological issues, 1980-1995. In Leithwood, K, Chapman, J., Corson, D., Hallinger, P, & Weaver-Hart, A. (Eds.), International handbook of educational leadership and administration (pp.723-784). New York: Kluwer. Hallinger, P., & Heck, R. H. (1998). Exploring the principal s contribution to school effectiveness: 1980-1995. School Effectiveness and School Improvement, 9, 157-191. Hargreaves, A., Moore, S., Fink, D., Brayman, C., & White, R. (2003). Succeeding leaders? A study of principal rotation and succession. Toronto, ON: Ontario Principals' Council. Harris, A., & Chapman, C. (2002). Democratic leadership for school improvement in challenging contexts. Paper presented at the International Congress of School Effectiveness and Improvement, Copenhagen. Harris, A., Muijs, D., Chapman, C., Stoll, L., & Russ, J. (2003). Raising attainment in schools in former coalfield areas. London: Department for Education and Skills. Heck, R., Marcoulides, G. (1996). School culture and performance: testing the invariance of an organizational model, School Effectiveness and School Improvement, 7, 1, 76-95. Hesketh, B. (1997). Dilemmas in training for transfer and retention. Applied Psychology: An International Review 46(4), 339-361. James, C., & Colebourne, D. (2004). Managing the performance of staff in LEAs in Wales. Educational Management Administration and Leadership, 32(1), 45-65. Jennings, K., & Lomas, L. (2003). Implementing performance management for headteachers in English secondary schools. Educational Management and Administration, 31(4), 369-383. Johansson, O. (2003). School leadership as a democratic arena. In P. Begley & O. Johansson (Eds.), The ethical dimensions of school leadership. Dordrecht: Kluwer Academic Publishers. Johnston, J. (1997). Primary school teachers' perceptions of dynamic process in working together: A case study. Paper presented at the Educational Research Association Annual Conference, York, UK. Jones, S. (2003). School leadership in disadvantaged communities. Paper presented at the British Educational Research Association Annual Conference, Edinburgh. Kahai, S. S., Sosik, J. J., & Avolio, B. J. (2003). Effects of leadership style, anonymity, and rewards on creativity-relevant processes and outcomes in an electronic meeting system context. The Leadership Quarterly, 14(4-5), 499-524. Koene, B. A. S., Vogelaar, A. L. W., & Soeters, J. L. (2002). Leadership effects on organizational climate and financial performance: Local leadership effect in chain organizations. The Leadership Quarterly, 13(3), 193-215. Kotter, J. (1990). A force for change: How leadership differs from management. New York: The Free Press. Kouzes, J. M., & Posner, B. Z. (1995). The leadership challenge: How to keep getting extraordinary things done in organizations (Revised ed.). San Francisco: Jossey- Bass. 50

Leithwood, K., Levin, B. (2004 a). Approaches to the evaluation of leadership effects and leadership programmes. Toronto: OISE/UT: Paper prepared for the U.K. Department for Education and Skills, February. Leithwood, K., Levin, B. (2004 b). Understanding leadership effects on pupil learning. Toronto: OISE/UT: Paper prepared for the U.K. Department for Education and Skills, December. Leithwood, K., Levin, B. (2005). Assessing leadership effects on pupil learning. Part 2: Methodological issues. Toronto: OISE/UT: Paper prepared for the U.K. Department for Education and Skills, March. Leithwood, K., Aitken, R., & Jantzi, D. (2001). Making schools smarter: A system for monitoring school and district progress (2nd ed.). Thousand Oaks, CA: Corwin Press. Leithwood, K., & Duke, D. (1999). A century's quest to understand school leadership. In J. Murphy & K. Seashore Louis (Eds.), Handbook of research on educational administration (pp. 45-72). San Francisco: Jossey-Bass. Leithwood, K., & Jantzi, D. (1999). The relative effects of principal and teacher sources of leadership on student engagement with school. Educational Administration Quarterly, 35(Supplemental), 679-706. Leithwood, K., & Jantzi, D. (2000). The Transformational School Leadership Survey. Toronto: OISE/University of Toronto. Leithwood, K., Jantzi, D., & Steinbach, R. (1999). Changing leadership for changing times. Buckingham, UK: Open University Press. Leithwood, K., & Levin, B. (2004). Approaches to the evaluation of leadership programmes and leadership effects: OISE/UT and University of Manitoba. Leithwood, K., Riedlinger, B., Bauer, S., & Jantzi, D. (2003). Leadership programme effects on pupil learning: The case of the Greater New Orleans School Leadership Center. Journal of School Leadership and Management, 13(6), 707-738. Leithwood, K., Jantzi, D., Earl, L., Watson, N., Levin, B., & Fullan, M. (2004). Strategic leadership for large-scale reform: The case of England's National Literacy and Numeracy Strategies. Journal of School Leadership and Management, 24(1), 57-80. Leithwood, K., Seashore-Louis, K., Anderson, S., & Wahlstrom, K. (2004). How leadership influences pupil learning: A review of research for the Learning from Leadership Project. New York: The Wallace Foundation. Leithwood, K., Jantzi, D., Coffin, G., & Wilson, P. (1996). Preparing school leaders: What works? Journal of School Leadership, 6(3), 316-342. Leithwood, K., & Steinbach, R. (1995). Expert problem solving: Evidence from school and district leaders. New York: SUNY. Leithwood, K., & Levin, B. (2004). Approaches to the evaluation of leadership effects and leadership programmes: UK Department of Education and Skills. Levin, B. (in press). Students at-risk: A review of research. Paper prepared for The Learning Partnership, Toronto. Linn, R. (2003). Accountability: responsibility and reasonable expectations. Educational Researcher, 32(7), 3-13. Lord, R. G., & Maher, K. J. (1993). Leadership and information processing. London: Routledge. 51

Lunenburg, F. (2004). Transformational leadership: Factor structure of Bass and Avolio's MLQ in public school organizations. Paper presented at the annual meeting of the University Council for Educational Administration, Kansas City, MO. Mac an Ghaill, M. (1996). Class, culture and difference in England: Deconstructing the institutional norm. International Journal of Qualitative Studies in Education, 9(3), 297-309. Macmillan, R. B. (1996). The relationship between school culture and principal's practices at the time of succession. University of Toronto, Toronto. Marks, H., & Printy, S. (2003). Principal leadership and school performance: An integration of transformational and instructional leadership. Educational Leadership Quarterly, 34(3), 370-397. Mayo, E. (1933). The human problems of an industrial civilization. Boston, MA: Harvard Business School. McCarthy, M. M. (1999). The evolution of educational leadership preparation programs. In J. Murphy & K. S. Louis (Eds.), Handbook of Research on Educational Administration 2 nd ed. San Francisco: Jossey-Bass. McCarthy, M. M. (2002). Educational leadership preparation programs: a glance at the past with an eye toward the future. Leadership and Policy in Schools, 1(3), 201-221. Meindl, J. R. (1995). The romance of leadership as a follower-centric theory: A social constructionist approach. 6(3), 329-342. Milgram, R. M. (2003). Challenging out-of-school activities as a predictor of creative accomplishments in art, drama, dance and social leadership. Scandinavian Journal of Educational Research, 47(3), 305-315. Morris, A. (2000). Charismatic leadership and its after-effects in a Catholic school. Educational Management and Administration, 28(4), 405-418. Mortimore, P. (1993). School effectiveness and the management of effective learning and teaching. School Effectiveness and School Improvement, 4(4), 290-310. Murphy, J. & Datnow, A. (2003). The development of comprehensive school reform. In J. Murphy & A. Datnow (Eds.), Leadership Lessons from Comprehensive School Reforms. Thousand Oaks, CA: Corwin Press. Moyo-Robbins, M. (2002). Early careers of primary school teachers: Age, gender, ethnicity in graduates' job destinations and career development. Paper presented at the British Educational Research Association Annual Conference, Exeter. Nicolaidou, M., & Ainscow, M. (2002). Understanding 'failing' schools: The role of culture and leadership. Paper presented at the British Education Research Association Conference, Exeter. Nguni, S. (2004). Transformational leadership in Tanzanian education. Neubert, M. J., & Taggar, S. (2004). Pathways to informal leadership: The moderating role of gender on the relationship of individual differences and team member network centrality to informal leadership emergence. The Leadership Quarterly, 15(2), 175-194 Oduro, G. K. T. (2004). Distributed leadership in schools: What English headteachers say about the "pull" and "push" factors. Paper presented at the British Educational Research Association Annual Conference, Manchester, UK. 52

Olson, M., & Davidson, J. (2003). School leadership in networked schools: Deciphering the impact of large technical systems on education. International Journal of Leadership in Education, 6(3). Penny, R. (2003). Transforming schools, transforming learning: Integrating CPD and knowledge management strategies to build collegiate practice. Paper presented at the British Educational Research Association Annual Conference, Edinburgh. Pepin, B. (2000). Cultures of didactics: Teachers' perceptions of their work and their role as teachers in England, France and Germany. Paper presented at the European Conference on Educational Research, Edinburgh. Peterson, K. D. (2001). The professional development of principals: Innovations and opportunities. Paper commissioned for the first meeting of the National Commission for the Advancement of Educational Leadership Preparation, Rancine, WI. Popper, M., & Mayseless, O. (2002). Internal world of transformational leaders. In B. Avolio & F. Yammarino (Eds.), Transformational and charismatic leadership: The road ahead. Oxford: Elsevier Science Ltd. Poulson, L. (2001). Paradigm lost? Subject knowledge, primary teachers and education policy. British Journal of Educational Studies, 49(1), 40-55. Radnor, H. A., Ball, S. J., & Vincent, C. (2002). Local educational governance, accountability, and democracy in the United Kingdom. Educational Policy, 12(1-2). Reitzug, U., & Patterson, J. (1998). "I'm not going to lose you!" Empowerment through caring in an urban principal's practice with pupils. Urban Education, 33(2), 150-181. Roland, E., Galloway, D. (2004). Professional cultures in schools with high and low rates of bullying, School Effectiveness and School Improvement, 15, 3-4, 241-260. Ross, J. (2004). Effects of a running records assessment on early literacy achievement: Results of a controlled experiment. Journal of Educational Research, 97, 4, 186-194. Rost, J. C. (1991). Leadership for the twenty-first century. New York: Praeger Publishers. Rutherford, D. (2002). Changing times and changing roles: The perspectives of primary headteachers on the senior management teams. Educational Management and Administration, 30(4), 447-459. Scheurich, J. J. (1998). Highly successful and loving, public elementary schools populated mainly by low-ses children of color: Core beliefs and cultural characteristics. Urban Education, 33(4), 451-491. Selwyn, N., & Fitz, J. (2001). The politics of connectivity: The role of big business in UK education technology policy. Policy Studies Journal, 29(4). Shadish, W., Cook, T., & Campbell, D. (2002). Experimental and quasi experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin. Shavelson, R., & Towne, L. (Eds.). (2002). Scientific research in education. Washington, DC: National Academy Press. Silins, H., & Mulford, B. (2002). Schools as learning organizations: The case for system, teacher and pupil learning. Journal of Educational Administration, 40, 425-446. 53

Silins, H., & Mulford, B. (2004). Schools as learning organizations - effects on teacher leadership and student outcomes. School Effectiveness and School Improvement, 15(3-4), 443-466. Smith, F., Hardman, F., & Mroz, M. (1999). Evaluating the effectiveness of the National Literacy Strategy: Identifying indicators of success. Paper presented at the European Conference on Educational Research, Lahti, Finland. Spector, P. (1981). Research designs. Beverly Hills, CA: Sage Publications. Spillane, J., Halverson, R., & Diamond, J. (2000). Toward a theory of leadership practice: A distributed leadership perspective. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA. Spillane, J., Halverson, R. & Drummond, J. (2001). Investigating school leadership practice: A distributed perspective. Educational Researcher 30(3), 23-28. Stake, R. (1997). Case study methods. In R. Jaeger (Ed.), Complementary methods for research in education (pp. 4001-4421). Washington, DC: American Educational Research Association. Starratt, R. J. (2003). Democratic leadership theory in late modernity: An Oxymoron or ironic possibility? In P. Begley & O. Johansson (Eds.), The ethical dimensions of school leadership. Dordrecht: Kluwer Academic Publishers. Thomas, A. B. (1988). Does leadership make a difference to organizational performance? Administrative Science Quarterly, 33, 388-400. Townsend, T. (1994). Goals for effective schools: the view from the filed. School Effectiveness and School Improvement, 5(2), 127-148. Tyler, T. R., & Degoey, P. (1996). Trust in organizational authorities: The influence of motive attributes on willingness to accept decisions. In R. M. Kramer & T. R. Tyler (Eds.), Trust in organizations: Frontiers of theory and research. Thousand Oaks, CA: Sage Publications. Vecchio, R. P., & Boatwright, K. J. (2002). Preferences for idealized styles of supervision. The Leadership Quarterly, 13(4), 327-342 Viadero, D. (2004). Math program seen to lack a research base. Education Week, 24(1), 1. Wallace, M. (2001). Sharing leadership of schools through teamwork: A justifiable risk? Educational Management and Administration, 29(2), 153-167. Waters, T., Marzano, R. J., & McNulty, B. (2003). Balanced leadership: What 30 years of research tells us about the effect of leadership on pupil achievement. A working paper: McREL. Whittington, J. L., Goodwin, V. L., & Murray, B. (2004). Transformational leadership, goal difficulty, and job design: Independent and interactive effects on employee outcomes. The Leadership Quarterly, 15(5), 593-606. Wilkins, R. (2002). Linking resources to learning: Conceptual and practical problems. Educational Management and Administration, 30(3), 313-326. Wilson, V., & McPake, J. (2000). Managing change in small Scottish primary schools. Educational Management and Administration, 28(2), 119-132. Witzier, B., Bosker, R., & Kruger, M. (2003). Educational leadership and pupil achievement: The elusive search for an association. Educational Administration Quarterly, 34(3), 398-425. 54

Wofford, J. (1999). Laboratory research on charismatic leadership: Fruitful or futile? The Leadership Quarterly, 10(4), 523-529. Wolfe, R., Childs, R., & Elgie, S. (2004). Final report of the external evaluation of the EQAO's assessment process. Toronto: OISE/University of Toronto. York-Barr, J., & Duke, K. (2004). What do we know about teacher leadership? Findings from two decades of scholarship. Review of Educational Research, 74(3), 255-316. Young, M. D., Peterson, G. J. & Short, P. M. (2001). The complexity of substantive reform: a call for interdependence among key stakeholders. Paper commissioned for the first meeting of the National Commission for the Advancement of Educational Leadership Preparation, Rancine, WI. Yukl, G. (1994). Leadership in organizations (3rd ed.). Englewood Cliffs, NJ: Prentice- Hall. Yukl, G., & Lepsinger, R. (2004). Flexible leadership: Creating value by balancing multiple challenges and choices. San Francisco: Jossey-Bass. Zaccaro, S. J., & Klimoski, R. J. (2001). The nature of organizational leadership: An introduction. In S. J. Zaccaro & R. J. Klimoski (Eds.), The nature of organizational leadership: Understanding the performance imperatives confronting today's leaders (pp. 3-41). San Francisco: Jossey-Bass. 55

Copies of this publication can be obtained from: DfES Publications P.O. Box 5050 Sherwood Park Annesley Nottingham NG15 0DJ Tel: 0845 60 222 60 Fax: 0845 60 333 60 Minicom: 0845 60 555 60 Oneline: www.dfespublications.gov.uk Kenneth Leithwood and Ben Levin 2005 Produced by the Department for Education and Skills ISBN 1 84478 527 0 Ref No: RR662 www.dfes.go.uk/research