THE IMPACT OF THE SCHOOL LIBRARY RESOURCE CENTRE ON LEARNING

Transcription

1 THE ROBERT GORDON UNIVERSITY ABERDEEN THE IMPACT OF THE SCHOOL LIBRARY RESOURCE CENTRE ON LEARNING Report on research conducted for Resource: The Council for Museums, Archives and Libraries Library and Information Commission Research Report 112 by Professor Dorothy Williams Caroline Wavell 2001 Research Report 9 School of Information and Media FACULTY OF MANAGEMENT THE ROBERT GORDON UNIVERSITY

2 THE IMPACT OF THE SCHOOL LIBRARY RESOURCE CENTRE ON LEARNING A report on research conducted for Resource: The Council for Museums, Archives and Libraries by Professor Dorothy Williams and Caroline Wavell 2001 Resource: The Council for Museums, Archives and Libraries The opinions expressed in this report are those of the authors and not necessarily those of Resource: The Council for Museums, Archives and Libraries. Library and Information Commission Research Report 112 RE/037 ISBN ISSN Library and Information Commission Research Report 112 is published by The Robert Gordon University for Resource: The Council for Museums, Archives and Libraries and copies may be purchased from The School of Information and Media, The Robert Gordon University, Garthdee Road, Aberdeen, AB10 7QE, UK.

3 ABSTRACT This report describes research into the impact of the school library resource centre (SLRC) on learning, funded by Resource: The Council for Museums, Archives and Libraries. The study was conducted in Scottish secondary schools between August 1999 and February The research was undertaken at a time of increasing presssure to measure performance in all educational establishments, including libraries and information services (ILS) and to ensure the service is not only efficient but also effective. The study looked at learning in its broadest sense, including motivation, progression, independence, and interaction. The first phase of the study used focus group discussions with teachers and pupils and interviews with librarians to identify their perceptions of how the SLRC can contribute to learning. The second phase used case studies of specific examples of SLRC activity to establish whether this perceived impact on learning was indeed taking place; to identify indicators which might be used as evidence of learning; and to examine the process of assessing the impact of the SLRC on learning. The report identifies: stakeholder groups perceptions of the wide range of possible learning experiences which might be affected by the SLRC; the broad range of potential impacts on the learning experience which were observed in practice; examples of indicators of learning which can provide a starting point for individual teachers and librarians to examine the relationship between their own SLRC and learning; examples of techniques used to monitor the impact on learning; a range of factors which influence the effectiveness of learning in the SLRC, and which will help practitioners make decisions about aims, objectives and activities to optimise the learning potential. AUTHORS Dorothy Williams is Professor at the School of Information and Media, The Robert Gordon University, Aberdeen and has considerable experience in research at the interface between education and information, in particular information skills development, information management and the impact of information on decision-making in education. Caroline Wavell is a Research Assistant at the School of Information and Media, The Robert Gordon University, Aberdeen and has worked in the museum and school library sectors developing a keen interest in education in its broadest sense.

4 ACKNOWLEDGEMENTS The project team would like to thank all those who have contributed to this research project. We are grateful to the following members of our Advisory Group for their support and advice throughout the project: Rhona Arthur Mike Baughan Caroline Beaton Barbara Buckley Margaret Croucher Graham Legge Clare Nankivell Kay Wilson Assistant Director, Scottish Library Association Chief Executive, Learning & Teaching Scotland Curriculum Support Team, Perth & Kinross Education Department Research Programme Manager, Library & Information Commission Research Programme Manager, Resource - The Council for Museums, Archives and Libraries The Rector, Kemnay Academy Research Associate and CIRT Manager, School of Information Studies, University of Central England Network Librarian, Alford Academy & Community Education Centre We thank the Heads of School Library Services in Scottish local authorities for their help in identifying potential schools to help with the project. We thank the Directors of Education in Scottish local authorities for giving us permission to approach schools. Finally, we are particularly grateful to all those librarians, teachers, head teachers and pupils who kindly gave their time to participate in this study. This report represents the views of the authors and not necessarily those of Resource.

5 CONTENTS page Abstract Authors Acknowledgements Contents List of appendices List of tables List of figures Executive Summary i-iv 1. Introduction Background The SLRC and learning Other studies 4 3. Assessing Impact Indicators of impact Scope of impact assessment Sources of evidence The process of impact assessment Methodology Selection of schools Focus groups Case study design Focus Group & Interview Findings Findings from teachers' focus groups Findings from pupils' focus groups Findings from the librarians' interviews Discussion of findings Case Study Findings School C: S1 English School A: S1 Modern languages School I: S2 Geography School K: Sixth form biology School F: S3 Skills course School F: Reading for pleasure School H: Pupil librarians Special events and ad hoc use of the SLRC Discussion Motivation Progression Independence Interaction Revised framework Value of monitoring techniques 123

6 8. Conclusion Research in context Learning in the SLRC Monitoring and evaluating impact on learning Planning for effective learning through the SLRC Bibliography List of appendices Appendices 1-5 Instruments for focus groups 1. Flier and questionnaire sent out to heads of school library service 2. Questions for librarians' interviews 3. Letter of introduction and SLRC use sheet sent to participating teachers 4. Learning outcome grid completed by teachers and librarians 5. SLRC use sheet completed by pupils Appendix 6-16 Instruments for case studies 6. Procedure Matrix 7. Interview with teacher 8. Plan for use of the SLRC 9. Example of completed impact framework grid for case study Observation schedule 11. Example of questionnaire, reading for enjoyment 12. Checklist for evidence in work 13. Questionnaire used for attempted gathering of ad hoc data 14. Librarian log 15. Teacher and librarian interview at end of project 16. Teacher's reflection on pupil librarians Appendix Case study data 17. Case study 1, animal project questionnaire 18. Case study 2, French project questionnaire 19. Extract from observation notes, case study Case study 3, geography investigation questionnaire 21. Case study 4, biology investigation questionnaire 22. Case study 5, reading for enjoyment questionnaire: control class D, 23. individualised class E, 24. database class F 25. Analysis of reading material, case study 6

7 List of tables page 1 Distribution of responding and participating schools 14 2 Composition of focus groups 21 3 Completion rates of SLRC use sheets and learning outcome grids 22 4 Pupil use of the SLRC 26 5 Role of the librarian 28 6 Librarians' objectives for pupil use 29 7 Comparisons between group perceptions of impacts of SLRC 30 on learning 8 Initial classification of learning impacts 31 9 Summary of the conditioning factors influencing the impact 33 on learning 10 Comments on self-evaluation sheets completed by 79 observation class (case study 5) 11 Breakdown of self-evaluation comments from 80 all the classes (case study 5) 12 Motivation from the impact framework Progression from the impact framework Independence from the impact framework Interaction from the impact framework Summary of conditioning factors observed from the case studies 123 List of figures 1. Aspects of learning areas upon which the SLRC can impact Initial impact framework Planning and evaluation cycle Blank framework Revised impact framework

8 THE IMPACT OF THE SCHOOL LIBRARY RESOURCE CENTRE ON LEARNING EXECUTIVE SUMMARY This report presents the findings of a recent study which investigated the impact of the school library resource centre (SLRC) on learning. The study had two phases: 1. Focus group discussions with teachers and pupils and interviews with librarians identified teachers', pupils' and librarians' perceptions of how the SLRC can contribute to learning. 2. Case study research, using the framework developed from the first phase, identified learning experiences that the SLRC impacted upon; investigated indicators to be used as evidence of learning; and examined the process of assessing the impact of the SLRC on learning. The research was conducted in secondary schools in Scotland between August 1999 and February The study was commissioned by the Library and Information Commission, which was absorbed into Resource - The Council for Museums, Archives and Libraries in April Perceptions of learning impacts (Section 5) The teachers represented in the focus groups covered a wide cross-section of subject departments as well as members of senior management. Their accumulated perceptions of the impact of the SLRC on learning covered a broad range of potential learning experiences but it was clear that their views were based on broad expectations of what the impact might be, rather than any in-depth analysis or monitoring of actual learning impact. The pupils participating in the focus group sessions found the concept of how the SLRC might contribute to learning more difficult to grasp but were able to identify a range of potential learning impacts. The librarians interviewed were aware of what they were trying to achieve but were not sure whether their objectives were being met. The collective perceptions of impact were expressed as: The acquisition of information and wider general knowledge; Skills development in the areas of finding and using information, ICT skills and reading skills and their cross-curricular use and potential for equal opportunities; Higher achievement in school work; Developing a study and reading habit encouraging independent working; Motivation to learn and enjoyment of learning; The ability to use these skills confidently and independently and the ability to transfer these skills across the curriculum and beyond school; The development of interpersonal and social skills, including working collaboratively. The three groups of participants shared a common perception of how the SLRC could contribute to learning. The discussions also highlighted factors which were thought to influence the learning impact and these factors tended to reflect constraints imposed by finances, environmental conditions, personalities and social background, which were not easily controlled by individual teachers or the librarian. These potential areas of impact on learning formed the foundation of an impact framework of potential learning experiences, which related areas of impact to established educational and i

9 learning classifications. The result was a list of potential learning experiences that were categorised under the learning themes of motivation, progression, independence and interaction and a corresponding list of possible indicators which could be used to look for evidence that such learning experiences were taking place. This framework provided the basis for a qualitative, observational approach to case study investigations, in which evidence of the learning experiences was identified and the validity of the indicators was examined. Findings from the case studies (Sections 6 & 7) The case study units covered a range of learning across the curriculum and broader educational goals and they also represented a wide range of year groups. The activities selected for investigation covered a range of curriculum related project work, a skills course, the promotion of reading for pleasure, pupil librarians, special events and ad hoc use of the SLRC. The investigation examined the learning experiences and evidence of impact was identified in the learning themes of motivation, progression, independence and interaction. Examples of indicators that might be used to look for evidence of impact were identified. The various methods of monitoring impact and the value these methods have for particular learning experiences was examined. Finally the case studies highlighted a number of conditioning factors which influence the impact on learning. Evidence of motivation was seen across all the case units by pupil enjoyment and participation and absorption in the tasks set whether that was a project, looking for reading material or in the commitment of the pupil librarians. The indicators were identified as: Verbal and written expression of enthusiasm by pupils; Pupil willingness to participate in the activity set; Pupil application and absorption in the task; Willingness of pupils to continue their work either by returning to the SLRC or at home; A change in attitude towards work over a period of time. Evidence of progression was most easily identified as awareness of or ability to use a specific skill associated with the finding of information and sometimes the use and presentation of information. It was also possible to identify examples of the application of skills in new contexts. The indicators were identified as: Awareness of or the ability to use specific skills associated with finding, using and presenting information; The use of new knowledge in work or discussion of new knowledge; Personal achievement or quality of work; The ability to apply skills or knowledge in a new situation. Evidence of independence was identified in individual pupils who appeared to have mastered a skill and were seen to have the confidence and competence to proceed and progress unaided, either in the class session or in their own time. The pupil librarians were able to use their initiative by instigating their own activities and tasks. The indicators were identified as: The ability and confidence to continue and progress with a task unaided; Awareness of the need for help and the confidence to seek it; Awareness of the need for organisation and time management in work; Use of initiative; Increased self-esteem. Evidence of interaction was particularly relevant in the junior curriculum related activities and with the pupil librarians, where discussion and interaction were encouraged. Interaction ii

10 was seen as a learning experience in itself, as a means of enhancing the learning in other themes and as a means of establishing the learning that individuals were engaged in. The indicators were identified as: Discussion with others about the task; Peer co-operation; Ability to mix with other groups; Use of appropriate behaviour. The methods used for monitoring impact on learning in the SLRC were varied and some were easier to use within the class situation while others required discussion or questioning beyond the immediate environment. The methods identified as potentially useful tools for practitioners in schools were: Observation of pupils at work in the SLRC and connecting their learning both to the immediate context and to previous experiences; Discussion with and questioning of pupils about their work both during and at the end of the activities; Examination of work in progress and, if available and when appropriate, submitted written work; Discussion with other members of teaching staff about work, attitudes and related incidences; Examination of reader records. The factors that were seen to influence effective learning were varied and were often seen to be associated with the immediate context. An examination of the influencing factors may help teachers and librarians to make decisions about their aims, objectives and activities to optimise the learning potential. Interestingly, these factors reflect the need for joint teacher and librarian planning. The influencing factors identified during the case units included: Interest, enthusiasm and appreciation shown by others; Appropriate intervention to ensure progress could proceed; Familiarity with surroundings; Foundation of necessary skills to proceed; Understanding of task or tasks; Opportunity to try again and build on understanding; New stimuli, for instance the use of computers; Frustration caused by problems; Tensions within groups; Time constraints or inappropriate use; Lack of focus, skills, background theory. The way forward (Section 8) The project developed a framework of potential learning experiences and examples of indicators which can be used to evaluate learning. The framework provides a basic understanding of the SLRC learning context, which could provide a useful starting point for teachers, librarians and schools to begin looking at effective learning associated with the SLRC. The report highlights a number of key issues which need to be resolved if the SLRC is to make an effective impact on learning: the need for local and collaborative planning and decision-making. The report provides an understanding of how the learning experiences interact and interrelate with each other and within the curriculum and school context. There is evidence of learning beyond the immediate goals set by the individual teacher using the SLRC and, although these are iii

11 not necessarily readily visible, they are part of the overall learning experience within the school environment and play a vital role in the development of the cross-curricular, core skills needed to fulfil the educational aims of the current political and social climate. The framework could be used within the overall planning and evaluation processes at a school, subject department, SLRC and course level to help identify appropriate roles for the SLRC. An understanding of the influencing factors identified in the case units may help teachers, librarians and senior management to prioritise conflicting and competing agendas when using the SLRC. the need for more understanding or use of the skills involved in information handling and a mutual understanding of how the library and teaching professions can support each other and collaborate together in order to encourage effective use of the SLRC. the need to look beyond the immediate SLRC for impact. The evidence of the impact on learning is not always easily visible but, if it is thought to be valuable, measures need to be put in place to identify impact across learning contexts, beyond the immediate situation and over appropriate timescales. This requires some form of evaluation of courses or activities undertaken in the SLRC to ensure effective teaching and learning. There is also a need to assess individuals to ensure progression of skills and progress over time. It is not suggested that every use of the SLRC requires formal evaluation or assessment but that the stakeholders in the local context will be in a position to decide their own priorities. There is potential value in applying the experiences gained from the SLRC impact study to other information services (such as public libraries, museums and archives) or learning contexts (such as homework clubs). The framework could provide the starting point for practitioners to examine the provision of learning experiences against the impact on the learning of the user in a variety of educational environments. However, the findings of the research are expected to be particularly relevant to all members of the teaching and library professions who are interested in the use of the SLRC for effective learning and teaching, from policy-makers, senior management to individual teachers and librarians working directly with pupils in schools. iv

12 1. INTRODUCTION This report describes research carried out in secondary schools in Scotland to assess the impact of the school library resource centre (SLRC) on learning. The research was undertaken between August 1999 and February The project was commissioned by the Library and Information Commission, which was absorbed into Resource - The Council for Museums, Archives and Libraries in April The research has been undertaken at a time when value and impact studies are trying to establish the contribution made by library and information services in different community contexts including that of supporting traditional education, independent study and lifelong learning. At the same time there is increasing interest in ways in which links between the public libraries and the educational sector can promote lifelong learning 1. A recent consultation document has suggested the need for a cross-sectoral learning and access standard for museums, archives and libraries 2 and this requires an understanding of the relationship between the user, the context and the learning in educational environments outside the traditional school or college setting. While this study focuses on Scottish schools, the interest in establishing the impact of information and library services in schools is worldwide, not least because of the increasing pressures to justify the resource allocation from a school's budget or the existence of adequate professional staffing within the SLRC. While there may be an intuitive knowledge that the SLRC makes a valuable contribution, more concrete evidence is required to convince decision-makers of its importance. At the same time, the position of the SLRC within the school environment is changing. In the UK, recent government initiatives such as the National Grid for Learning (NGfL) 3 and the implementation of information and communications technology (ICT) networks within schools has increased awareness of information support for the curriculum. The volume of information and the speed with which it is available has also raised an awareness of the need to understand information in terms of its delivery, evaluation of purpose and its ultimate use. Information scientists have a key role to play in developing the skills for effective information use and librarians in schools have found their roles being redefined to enable schools to fulfil government objectives for the NGfL, and to develop a clearer strategy for integrating the work of the SLRC within the wider educational experience. While ICT initiatives have already resulted in new opportunities to enhance the work of the SLRC in support of the curriculum, such developments have also dramatically increased the expenditure in the area of ICT. Such expenditure requires careful consideration and with recent devolved management within schools the pressures to justify expenditure in terms of educational benefits are increasing. There is a growing awareness of the need to examine carefully how effective the various inputs and outputs are in terms of enhancing the quality of learning and raising achievements within schools as well as creating better links with the wider community. This focus on evaluation of the quality of learning within schools, shifts the emphasis from one of measuring outputs (as has traditionally been the case when measuring SLRC performance) to that of the outcomes, particularly the nature and quality of the learning experience, resulting from the provision of the SLRC and how this relates to the goals and aspirations of the school itself. It may never be possible or even desirable to attempt to "measure" learning impact in the same way as academic achievement may be measured, but it is the understanding of the impact on the learning experience which will ultimately be needed if the work of the SLRC is to be fully integrated within the curriculum and thus contribute effectively to achievement. 1

13 All these factors have led to the realisation that greater understanding is required in the relationship between information and learning and between the role that the SLRC and the librarian can play in enhancing the quality of the learning experience. This research focuses on this relationship with the aim of identifying the impact the SLRC can have on learning as well as the challenges involved in evaluating that impact. The aim of the study was: to assess the impact of a School Library Resource Centre (SLRC) on learning. The main objectives were: to develop frameworks describing pupils', teachers' and librarians' perceptions of a) the ways in which the SLRC contributes to the learning experience; b) direct and indirect benefits to learning from a SLRC; and using the frameworks as a starting point to identify learning outcomes to be used as indicators of SLRC impact; to assess the impact of a SLRC on specific aspects of learning. The research was conducted within secondary schools, examining the issues from the perspectives of librarians, teachers and pupils in semi-structured focus groups and a series of case studies. Sections 2 and 3 examine the background to the study and the rationale behind the chosen research approach, while Section 4 describes in more detail the methods used. The outcomes of the focus group phase (Section 5) and the case studies (Section 6) are analysed separately before the fuller implications are discussed and interpreted in Sections 7 and 8. 1 a. Library and Information Commission. Empowering the learning community. Report of the Education and Libraries Task Group to the Secretaries of State for Culture, Media & Sport and for Education & Employment. London: Library and Information Commission, March [online] [May 2000]. b. DfEE. Consultative Fora Good Practice for Learning Partnerships. Nottingham: DfEE Publications, c. Train, Briony, et al. The Value and Impact of Homework Clubs in Public Libraries. Boston Spa: The Library and Information Commission, Library and Information Commission Research Report re:source. A Learning and Access Standard for Museums, Archives and Libraries. London: re:source The Coucil for Museums Archives and Libraries, April [online] [May 2001]. 3 a. Becta. The National Grid for Learning. Dated January [online] [ ]. b. Connecting the Learning Society: Government's consultation paper on NGfL - a summary. DfEE [online] [2001]. 2

14 2. BACKGROUND 2.1 THE SLRC AND LEARNING The emphasis of the research is on learning and the relationship between a SLRC and the learning experience. The mission and goals of any SLRC indicate that support for the learning process is seen as central to the development of information and library services. SLRCs aim not only to provide a rich learning environment in terms of an appropriate range of resources but also in terms of developing the learning skills of pupils. Yet the research and professional literature of recent years indicates that the relationship between the provision of the SLRC and the quality of the learning experience remains unclear 1. The need for greater understanding of this relationship emerges from a consideration of recent curriculum related developments, which have placed greater emphasis on the development of transferable skills. In 1999, the Scottish Office identified key priorities for the curriculum in developing essential skills to foster the personal and social development of the individual in order to equip them for life 2. The implementation of the new Higher Still (post-16, S5-S6 3 ) curriculum is intended not only to build upon these foundations within the 5-14 (primary - S2) and Standard Grade (S3-S4) education but to increase employability by encouraging the development of core skills in communication, numeracy, problem solving, information technology and working with others. The English National Curriculum has similar developments in learning opportunities across the curriculum to encourage a lifelong learning 4. The SLRC is seen as having an active and supporting role in achieving these educational priorities but for librarians as well as teachers, the nature of the potential SLRC contribution remains vaguely defined and raises wider questions about the relationship between the SLRC and the learning experience. Alongside these educational initiatives are the management issues of quality assurance and benchmarking. The use of performance indicators to self-evaluate the quality of educational provision in school has been paralleled by education and library organisations' attempts to develop guidelines and indicators for evaluating the SLRC 5, within the context of the school as a whole. One key area identified for evaluation is that of Learning and Teaching, within which one Performance Indicator is that of "Quality of pupils' learning". However, the evaluation of the contribution made by the SLRC to quality of learning within the school requires an understanding of the relationship between SLRC activities and the learning goals of the curriculum and the school as a whole. This is all part of the wider pressure on managers of library services in the public sector, including schools, to move beyond measuring efficiency of performance into assessment of effectiveness. The Government s response to a report by The Library and Information Commission (LIC) recognises the important role played by high quality school library services in supporting teaching and learning within schools 6. However, how to recognise quality provision, particularly in respect of learning, presents challenges. Traditionally attempts to measure performance have focused on activities which are essentially to do with the running of the SLRC rather than the impact these activities and services have on the work of the school and its learners. Measures of outputs from a library perspective (covering expenditure, resource provision and levels of library use, e.g. numbers of library-related transactions, learning skills transactions and users) may tell us much about the efficiency with which the SLRC operates but tells us little about the outcomes in terms of the learning experience and achievement. It would appear that a new approach is needed if schools are to begin to assess the impact of the SLRC on learning. 3

15 In the UK, recent guidelines produced by Information Management Associates of assessing Best Value and better performance encourages managers of public and school library services to move beyond attempts to benchmark or measure performance in output terms and to focus more on assessment of outcomes, in other words impact. However, the guidelines also acknowledge that while there is growing interest in benchmarking on the basis of impact and achievement (rather than outputs) in a wide range of public sector services and contexts "...nobody is doing it well - yet!" 7 Clearly a better understanding of the relationship between SLRC activities and the learning experience could be useful as a way of demonstrating the value of, and attaining appropriate recognition for, the work of the SLRC, an issue of concern to the Information and Library Science (ILS) profession world-wide 8. However, a more fundamental need for such understanding relates to the future planning of an effective SLRC in support of the curriculum. Effective integration within the work of the school requires some sharing of a common language and vision between the library and teaching professions of what they want from their SLRC in relation to the learning process. Yet, curriculum guidelines offer little guidance to either teachers or librarians on how the services of the SLRC should be incorporated into curriculum and lesson planning in support of the achievement of targets. It has become increasingly clear that there is a need for research which can provide some sort of framework or starting point within which schools can begin this dialogue and more effective planning. 2.2 OTHER STUDIES Attempts to measure SLRC impact in terms, which will be understood by educators and decision-makers, have been emerging in studies in the USA in recent years. As in the UK, much of the impetus for such studies in the USA has come from the greater pressures on librarians to justify their existence and the pressure on decision-makers to justify and prioritise expenditure in relation to the achievement of learning goals 9. Key studies such as the original "Colorado Study" 10 and the follow up Second Colorado Study 11 have already reported a strong statistically relevant correlation between the provision of an SLRC and academic achievement, in research conducted over a large sample of schools in the state of Colorado. In particular it appears that the quantity of professional staffing is a key factor. Other studies have also pointed to a positive impact on achievement albeit on a more localised scale, for example in schools involved in the Library Power programme initiative 12. Such studies have provided the kind of evidence that will be reassuring to librarians, and convincing to decision-makers. However, by focusing on academic achievement they do not necessarily take account of the broader learning experience. Nor do they fully explain the way in which the SLRC activities and programmes interact with, and contribute to, the learning experience. While providing positive evidence of the overall impact on achievement, their value in terms of planning and integrating the work of a SLRC within the curriculum is more limited. This present study could be seen as complementary to, but different from, the Colorado and more recent follow up studies carried out in the USA 13 in that it attempts interaction between the SLRC and the learning experience. Research by Kuhlthau 14 and Pitts 15 has also focused much more on the learning experience, specifically in relation to the information search process. By revealing and describing the inquiry process as a series of stages that relate to the cognitive and affective domains of learning, this research has been influential in helping to establish a relationship between information and learning. Building on these earlier investigations, Kuhlthau's more recent studies in relation to the Library Power programme 16 have gone on to identify indicators of learning and frameworks for evaluating libraries' provision of opportunities for learning which take account of attitudes, development of information skills and the use of resources 4

16 for learning through enquiry. Though this research has concentrated on evaluating the provision of learning opportunities, the approach taken is closely related to the aims of the present study. By engaging librarians in the process of identifying indicators of learning it begins to move our understanding on from a focus on library outputs to a language and understanding which relate to outcomes and impact on learning. The work of Kuhlthau and Pitts, provides an invaluable reference point in this study, however, these studies tend to focus on particular aspects of the SLRC role, in particular those of developing information skills and the information retrieval process. In other words they focus on the information handling aspects of the learning process. This is clearly a focus for much of the planned activity in the SLRC but does not address the wider learning goals which schools as a whole seek to achieve. In Scottish schools, for example, recent evaluation guidance to schools defines quality of learning under the broad themes of motivation, progress in learning, independence and interaction 17. The curriculum for primary and early secondary defines learning in terms of a broad range of knowledge and understanding, dispositions to learning, social and personal responsibility, and core skills and capabilities 18 ; while the new post 16 Higher Still curriculum seeks to develop a broad range of core skills (communication, application of number, information technology, working with others, improving own learning and performance, and problem solving) 19. These broader definitions of the goals and aspirations of schools, teachers and learners provide a backdrop for the work of the SLRC, yet it is currently not clear how the kinds of services and activities of the SLRC relate to these various learning experiences and goals. It is with this in mind that the present study set out to examine the wider impact of the SLRC, not just in terms of the impact on pupils' ability to find and use information, important though that clearly is in this context, but also on their ability to learn effectively in the broadest sense. 1 Streatfield, David and Markless, Sharon. Invisible Learning? The contribution of school libraries to teaching and learning. The British Library, Library & Information Research Report The Scottish Office. Targeting Excellence - Modernising Scotland's Schools. Edinburgh: The Stationery Office, 1999, p S1, S2, S3 etc., are the accepted abbreviations for first year secondary, second year secondary, third year secondary and so on in Scottish education. Similarly P1, P2, etc. are standard abbreviations for the year groups in primary education. 4 National Curriculum for England online homepage. [online] [March 2001]. 5 Taking a Closer Look at the School Library Resource Centre Self-Evaluation using Performance Indicators. SOEID/SCCC/SLA/SLIC, Empowering the Learning Community - Home Page. The Government s response to the report of the Education and Libraries Task Group empowering the learning community, p.5. [online] [June 2001]. 7 Streatfield, D., et al. Best Value and Better Performance in Libraries Project. London: Information Management Associates, [online] [August 2000]. 8 Eisenberg, Michael B. and Brown, Michael K. "Current Themes Regarding Library and Information Skills Instruction: research supporting and research lacking." School Library Media Quarterly. Vol 20 (2), Winter 1992, p Kuhlthau, Carol Collier. "Implementing a Process Approach to Information Skills: A study identifying indicators of success in library media programs". School Library Media Quarterly. Vol. 22 (1), Fall Lance, K.C., Welborn, L., & Hamilton-Pennell, C. The impact of school library media centers on academic achievement. Castle Rock, CO: Hi Willow Research and Publishing, Lance, K.C., Rodney, M.J. & Hamilton-Pennell, C. How School Librarians Help Kids Achieve Standards The Second Colorado Study. San Jose: Hi Willow Research & Publishing,

17 12 Hopkins, Dianne McAfee and Zweizig, Douglas L. "Introduction to the Theme Issue: Library Power Programme Evaluation". School Libraries Worldwide. Vol. 5 (2), 1999, p. i-vi. Oberg, Dianne. "A Library Power Case Study of Lakeside Elementary School, Chattanooga, Tennessee". School Libraries Worldwide. Vol. 5 (2), 1999, p a. Lance, K.C., Welborn, L., & Hamilton-Pennell, C. The impact of school library media centers on academic achievement. Castle Rock, CO: Hi Willow Research and Publishing, b. Lance, K.C., Rodney, M.J. & Hamilton-Pennell, C. How School Librarians Help Kids Achieve Standards The Second Colorado Study. San Jose: Hi Willow Research & Publishing, c. Lance, K.C., Hamilton-Pennell, C. & Rodney, M.J. Information Empowered: The School Librarian as an Agent of Academic Achievement in Alaska Schools. Juneau: Alaska State Library, d. Lance, K.C., Rodney, M.J. & Hamilton-Pennell, C. Measuring Up to Standards: The Impact of School Library Programs & Information Literacy in Pennsylvania Schools. Camp Hill, PA: Pennsylvania Citizens for Better Libraries, forthcoming. 14 Kuhlthau, Carol Collier. Seeking Meaning: a process approach to library and information services. Norwood, New Jersey: Ablex Publishing Corp, Pitts, Judy M. "Mental Models of Information: the AASL/Highsmith research award study." School Library Media Quarterly. Vol. 23 (3), Spring 1995, p Kuhlthau, Carol Collier. "Student Learning in the Library: What Library Power Librarians Say." School. Libraries Worldwide; Vol. 5 (2), July 1999, p Taking a Closer Look at the School Library Resource Centre Self-Evaluation using Performance Indicators. SOEID/SCCC/SLA/SLIC, Scottish Executive. The Structure and Balance of the Curriculum 5-14 National Guidelines. Dundee: Learning and Teaching Scotland, [online] [2001]. 19 SCET. Higher Still Homepage. Scottish Council for Educational Technology, [online] [ ]. 6

18 3. ASSESSING IMPACT 3.1 INDICATORS OF IMPACT Some of the challenges faced in studying the impact of the SLRC within its community are on the face of it similar to the challenges encountered in attempts to assess impact of any kind of information and library service, namely identifying reliable, valid indicators of impact, and the methods to assess them within a complex environment. Indicators have to be appropriate to the type of information and library service or programme being studied and the environment within which users operate. Menou recognises this need in his framework for measuring impact of information on development 1. This framework advocates an in-depth qualitative study of the Information Use Environment (IUE) in order to identify a suite of indicators that can be used for evaluation at several levels. In dealing with impact of the SLRC on learning, the indicators chosen have to reflect the learning goals of the curriculum and the school as a whole since this is the context within which learning activities in the SLRC will be planned and undertaken. Moreover, the impact of any SLRC will be an individual impact depending on the learner and on local circumstances and the indicators of impact will tend to be localised and specific. To some extent learning in an information context can be seen as a decision-making process in that the learner has choices to make and will tend to make those choices in an individualistic way depending on previous experiences, motivation, confidence, information handling skills, and so on 2. Studies of real-world decision-making indicate that the point at which information may influence a decision will depend on the accumulated experience, knowledge, values and preferences of the decision-maker and not simply on their awareness of a particular piece of information 3. The way in which information is interpreted and used, and the timescale for information to have an impact, will vary from individual to individual. Thus the indicators selected should, ideally, allow for the assessment of SLRC impacts which may be direct or indirect, immediate or longer term. The complexity is compounded by the need to examine learning in its broadest sense (see Sections 2.2 and 3.2) identifying indicators which are appropriate to a wide range of learning experiences. While some may relate to the uptake and use of information others will relate more to motivation, interaction and communication with others. The choice of appropriate indicators will also be influenced by the decisions made by the teacher as an intermediary in the learning experience. Local interpretation of the curriculum, local setting of learning goals, and individual teaching styles will undoubtedly influence the kind of value and impact the SLRC will have. Conversely, it is only by developing an understanding of the potential for SLRC impact on the learning experience that the librarian and teacher can plan effectively to ensure that the learning goals of the SLRC are appropriate to the pedagogical approaches and to local teaching and learning styles. The work of Streatfield and Markless 4 clearly points to the wide variation in teaching approaches within schools, which will require appropriate but differing information and library provision. This once again stresses the need for research which will not simply assess impact in general terms but will provide frameworks for individual schools to evaluate the contributions made by their own SLRC within their own particular learning and educational environment. This has been a central consideration in the development of the approach taken within this study. It is clear, then, that the examination of the context of the "Information Use Environment" and identification of appropriate indicators of impact requires an understanding of the uses and expectations of the SLRC from the perspective of the learner as well as those involved in 7

19 mediating the learning experience, i.e. the librarians and teachers. However, it was recognised that this in itself was likely to raise some challenges within the study. Both teachers and librarians have some responsibility for the effective uptake of information and library provision in relation to the curriculum. Yet little clear guidance is provided to teachers (in, for example, curriculum guidelines) on what their expectations or plans should be when using the SLRC to support the curriculum or learning. Similarly while librarians have guidelines and standards to follow 5 these tend to relate more to outputs (services and targets) and do very little to help relate such services and outputs to the goals of learners and teachers. Therefore, while it was important to study impact on learning within a framework, which represented the uses, demands and expectations of the school as a whole, it was clear that defining what these expectations were in any degree of specificity might prove problematic. The approach taken was to conduct semi-structured focus groups with teachers and pupils and interviews with librarians to establish the framework of learning perceptions. 3.2 SCOPE OF IMPACT ASSESSMENT A further element of complexity is associated with the fact that the study sought to identify the scope and breadth of impact of the SLRC rather than impact of particular resources or information. The outputs from the SLRC which may impact on the learning experience are broad and varied. Alongside the provision of information, the SLRC provides planned instruction in information skills, individualised advice in the handling and presentation of information, provides advice and support within planned curriculum activities, promotion of reading for pleasure, afterschool activities and clubs, opportunities for library helpers and prefects, support for the professional development of the teacher, a social environment for meeting outwith timetabled classes, and so on. One way to make the process manageable is to restrict impact studies to specific aspects of the information and library service. It is interesting in this respect that recent guidance from Information Management Associates on "Best Value and Better Performance in Libraries" in the school and public sector, clearly signal the need for managers to restrict impact assessment to key aspects of the information and library service and not to attempt to assess impact of the whole service 6. This follows also the approach taken in many previous impact studies where the focus has often been restricted to the impact of specific information provision. For example, attempts to examine impact of information services in the health sector (e.g. Marshall 7 and Urquhart 8 ), financial sector (e.g. Marshall 9 ) or on development (e.g. Menou 10 ) have tended to examine the issues from the perspective of the impact of information on decision making. Moreover, as Streatfield and Markless point out, to be useful in any particular evaluation of impact indicators should: "be as few as possible allow meaningful comparisons to be made over time cover significant parts of the activities of the service[...] reflect the existence of competing priorities." 11 This being the case, the study sought to provide guidance for individual schools to develop a manageable and practical approach by focusing on particular aspects of the SLRC's work while still providing a broad framework for considering the impact of the SLRC as a whole. The approach adopted was a multi-site case study design in which specific examples of learning tasks, identified during the discussions with practitioners, were used as the case units. The analysis of these specific examples was complemented by relatively open reflection on the overall impact by practitioners and learners in focus groups and interviews. This allowed for an iterative building and refining of a framework which describes the ways in 8

20 which the SLRC can impact on the learning experience, based on the evidence of this study, and the ways in which schools can begin to monitor the impact of their own SLRC. 3.3 SOURCES OF EVIDENCE Any study which seeks to examine the outcomes of the use of an information and library service, faces further challenges in gaining insights into processes and experiences which may take place well outwith the initial library visit or transaction. Previous studies of the impact of information on decision-making have often relied on users being able to isolate and make judgements about the impact of information distinct from all the other influences on decision-making. Thus Marshall's work relied on self-reporting by clinicians on the way in which specific information gained from their information and library service had enhanced their ability to make sound judgements and decisions related to the diagnosis and treatment of patients 12. Similarly, Usherwood's study of the impact of the public library 13 relied on adult respondents, who identified the potential breadth of the impact of a library and information service in which the wider social as well as informational impacts can be seen. However, the question arises as to whether it is feasible and practical to build this kind of approach into a study of the impact of the information and library service within a school. Such studies provide useful pointers for examining the impact of information and library services in the educational sector but care has to be taken when attempting to adapt their methods for a school environment, when dealing with younger learners and a very broad range of potential areas for impact. Self-assessment of impact can be difficult enough for users when faced with relatively isolated instances of information and decisions, but is made much more problematic when dealing with the very wide ranging learning experiences of interest in the present study. In addition, previous impact studies in other sectors have tended to focus on mature adult users, whereas many of the users of the SLRC will be relatively immature and are likely to find it much more difficult to articulate their ideas, thoughts and feelings about the learning experience. Thus, while it was important to attempt to gain some feedback from learners, teachers and librarians themselves, it was also important to develop an approach which was not solely reliant on individuals' ability to isolate and make judgements about the SLRC distinct from all the other influences on learning. The approach taken was: one of triangulation, gathering data from different sources (learner, teacher and librarian) using a range of data collection techniques some of which relied on self-reporting, as well as direct observation and examination of the recorded (usually written) work of learners; not to attempt to separate impacts due to SLRC input from all the other influences, but instead to monitor the learning experiences taking place in SLRC related activities. (This became particularly important after it became clear that the teachers and librarians had quite general expectations and plans for SLRC activities and that it was going to be extremely difficult to develop a detailed picture prior to case studies of how the SLRC activity "fitted" into the planned curricular activities. (see Section 4.3.2)) Thus the study sought evidence of the kinds of learning which a range of SLRC-related activities could potentially stimulate, encourage and support, with the knowledge that the impact of the SLRC will itself be influenced by a wide range of other factors in the school. 9