Value for Money Review of the ICT Support Service for Schools

Transcription

1 Value for Money Review of the ICT Support Service for Schools Department of Education and Science, June 2008

2 Table of Contents List of Abbreviations List of Tables List of Figures List of Graphs Executive Summary Key Themes in Submissions Received Terms of Reference Page iv vi viii viii x xiii xv Chapter 1 - Introduction 1 Introduction 1 Background to the Value for Money Review Process 1 Methodology and Terms of Reference 2 History of the ICT in Schools Programme 5 Investment in the ICT in Schools Programme 7 Chapter 2 - General Overview of the ICT Support Service for Schools 8 Introduction 8 The NCTE and its Initiatives 8 ICT Advisory Service 10 Role of the NCCA 11 Chapter 3 - Objectives of the ICT Support Service for Schools 13 Introduction 13 International Context for the ICT in Schools Programme 14 Irish Government Policy on the Information Society 15 Education Policy 16 Objectives of the ICT Support Service for Schools 19 Validity of the ICT Support Service for Schools Objectives 21 Policy Framework for ICT in Schools Programme 22 Conclusion Principal Findings 22 Chapter 4 Efficiency of the ICT Support Service for Schools 24 Introduction 24 Efficiency of the NCTE 24 Inputs, Outputs and Activities of the Teaching Skills Initiative (TSI) 27 Inputs, Outputs and Activities of the Schools Support Initiative 38 Inputs, Outputs and Activities of the Innovative ICT Projects 47 Efficiency of the ICT Advisory Service 52 Efficiency of the NCCA ICT Education Officer posts 61 Conclusion re Efficiency of ICT Support Service 64 Chapter 5 Effectiveness of the ICT Support Service for Schools 66 Introduction 66 Effectiveness of the NCTE 66 Effectiveness of the Teaching Skills Initiative (TSI) 67 Effectiveness of the Schools Support Initiative 74 Effectiveness of the Innovative ICT Projects 84 Effectiveness of the ICT Advisory Service 91 Effectiveness of the NCCA ICT Education Officer posts 96 Conclusion Effectiveness of ICT Support Service for Schools 99 ii

3 Chapter 6 Alternative Policy or Organisational Approaches 102 Introduction 102 Achievement of Objectives and Future Funding 102 NCTE 104 Teaching Skills Initiative 104 Schools Support Initiative 105 Innovative Projects 106 The ICT Advisory Service 107 The ICT Education Officer Posts in the NCCA 108 Conclusion 108 Chapter 7 - Future Performance Indicators 111 Introduction 111 Potential Future Performance Indicators 111 Conclusion 114 Chapter 8 Conclusions and Recommendations 115 Introduction 115 Service Objectives 115 Continued validity of Objectives; compatibility with Education Policy 115 Efficiency of service 116 Effectiveness of service 117 Recommendations for the Service 120 Performance Indicators 122 Next Steps 122 Appendices 123 Appendix 1 - Programme Logic Models 125 Appendix 2 - Submissions from Stakeholders 131 Appendix 3A - Income and Expenditure for the NCTE ( ) 132 Appendix 3B - NCTE Income from Other Sources ( ) 133 Appendix 4 - NCTE Organisational Chart 134 Appendix 5 - A Comparison of the Cost of Providing TSI Courses by Course Provider 135 Appendix 6 - Number of TSI Courses run in Education Centres ( ) 136 Appendix 7 - Comparison of TSI costs between the full-time Education Centres ( ) 137 Appendix 8 - Comparison of full-time Education Centres and INTO cost of provision for TSI summer courses 138 Appendix 9 - Top 10 TSI courses by year delivered in the Education Centres 140 Appendix 10 - Analysis of 2005 Traffic to the Scoilnet Website 143 Appendix 11 - Analysis of the top 20 Scoilnet Search Terms in Appendix 12 - Analysis of Scoilnet Usage 147 Appendix 13 - ICT Implementing Bodies in other European countries 148 Bibliography 151 iii

4 List of Abbreviations ATECI AUP BECTA CAD CD CEO CIDREE CPD CSO CSPE CSPE erap DCU DES DISC DLLI DLN DVD ECDL ENIS EPV ERI ESRU EU EUN FÍS FTE HEA H.Dip HR IATSE IBEC IBM ICT ILSA INTO IPPN ISC IT LIA MS MUR NCCA NCTE NDP NPADC PC PCR PCSP PE PLM PRSI RTÉ SDPS SEN SIP SME SSI - Association of Teachers Education Centres of Ireland - Acceptable Use Policy - British Educational Communications and Technology Agency - Computer Aided Design - Compact Disc - Chief Executive Officer - Consortium of Institutions for Development and Research in Education in Europe - Continued Professional Development - Central Statistics Office - Civil, Social and Political Education - Civic Social and Political Education Report on the Action Project in Electronic Format - Dublin City University - Department of Education and Science - Dublin Inner City Schools Computerisation Project - Diageo Liberties Learning Initiative - Dundalk Learning Network - Digital Versatile Disc/Digital Video Disc - European Computer Driving Licence - European Network of Innovative Schools - Extra Personal Vacation - Expenditure Review Initiative - Evaluation Support Research Unit - European Union - European Schoolnet - Film in Schools Project - Full-time Equivalent - Higher Education Authority - Higher Diploma - Human Resources - Irish Association of Teachers in Special Education - Irish Business and Employers Confederation - International Business Machines - Information and Communications Technology - Irish Learning Support Association - Irish National Teachers Organisation - Irish Primary Principals Network - Interactive Software in the Curriculum - Information Technology - Litríocht Idirghníomhach na hardteistiméireachta - Microsoft - Minimum Unit Requirements - National Council for Curriculum and Assessment - National Centre for Technology in Education - National Development Plan - National Partnership Advice and Development Committee - Personal Computer - Pupil to Computer Ratio - Primary Curriculum Support Programme - Physical Education - Programme Logic Model - Pay Related Social Insurance - Radio Teilifís Éireann - School Development Planning Support - Special Education Needs - Schools Integration Project - Subject Matter Expert - School Support Initiative iv

5 TIF TSI UCD UL UNESCO VfM W3C WFL WWW - Telecommunications and Internet Federation - Teaching Skills Initiative - University College Dublin - University of Limerick - United Nations Educational, Scientific and Cultural Organisation - Value for Money - World Wide Web Consortium - Wired for Learning - World Wide Web v

6 List of Tables Page Table 1.1 Investment in the ICT in Schools Programme Table 4.1 NCTE annual expenditure for the period Table 4.2 Staff Pay and Non-Pay Costs for the NCTE ( ) 26 Table 4.3 Inputs and Outputs of the TSI Initiative ( ) 27 Table 4.4 Comparison of the number of TSI participants by course provider 29 Table 4.5 Comparison of full-time Education Centres and INTO cost of provision for TSI summer courses 38 Table 4.6 Service Costs of the Scoilnet Initiative ( ) 40 Table 4.7 Inputs and outputs of the Subject Matter Experts (SME) engaged for Scoilnet 41 Table 4.8 Number and type of supports provided for the assisted internet projects 42 Table 4.9 Service Costs of the Interactive Software in the Curriculum Initiative ( ) 43 Table 4.10 Costs of employing Subject Matter Experts on the ISC 45 Table 4.11 NCTE Developed Software, Distribution Medium and Costs 46 Table 4.12 Inputs and Outputs of the Innovative ICT Projects ( ) 49 Table 4.13 Service Costs of the Innovative Projects ( ) 49 Table 4.14 Breakdown of the schools involved in the SIP and Innovative Projects 50 Table 4.15 Outputs for the Innovative ICT Projects 51 Table 4.16 Breakdown of the pay and non-pay costs for the ICT Advisory Service 53 Table 4.17 Comparison of the 2005 activities and outputs for the ICT Advisors 56 Table 4.18 Number of Schools and Teachers by Education Centre Area 58 Table 4.19 Analysis of data for School Visits by the ICT Advisors in Table 4.20 Costs Associated with the ICT Education Officer Posts in the NCCA 62 Table 4.21 Documents produced by or in conjunction with the ICT Education Officer Posts in the NCCA ( ) 63 Table 5.1 Proficiency in ICT Skills for Primary School Teachers 68 Table 5.2 Proficiency in ICT Skills for Post-Primary School Teachers 69 Table 5.3 Graduate Numbers for Teachers in post-graduate ICT and Education programmes ( ) 70 Table 5.4 Percentages of Schools Providing Various Forms of ICT Professional Development in Past Two Years (2005) 71 vi

7 Table 5.5 Participant skills ratings before and after attending TSI courses 72 Table 5.6 The top 10 ICT courses delivered in the Education Centres in 2005 and Table 5.7 Schools benefiting from volume licensing agreements with Microsoft 82 Table 5.8 Usage statistics for the Software Libraries in Table 5.9 Star Ratings for Different Primary School Categories Table 5.10 Star Ratings for Different Post-Primary School Categories Table 5.11 Awareness and Usage of school ICT Advisory Service 93 vii

8 List of Figures Page Figure 1.1 Graphical presentation of the ICT Support Services for Schools 3 Figure 3.1 Influences on the Objectives of the ICT in Schools Programme 13 Figure 5.1 Strands of the NCCA programme of work for ICT in curriculum and assessment 97 List of Graphs Page Graph 4.1 Activities of the TSI staff in NCTE 28 Graph 4.2 TSI Unit Cost Trends by Course Provider 30 Graph 4.3 A comparison of the number of TSI courses run in the Education Centres with the number of courses with less than the minimum numbers ( ) 31 Graph 4.4 Average Cost per Participant Hour for the period 33 Graph 4.5 Trends in the yearly Average Cost per Participant Hour ( ) 33 Graph 4.6 A Comparison of Teacher Numbers with the Average Unit Cost per Participant 34 Graph 4.7 Unit Cost per participant for Graph 4.8 Trends in TSI Unit Costs for the full-time Education Centres ( ) 37 Graph 4.9 Breakdown of the activities of the Scoilnet Initiative 39 Graph 4.10 Activities of the Interactive Software in the Curriculum Initiative 43 Graph 4.11 Activities of the SIP and Innovative Projects 48 Graph 4.12 Breakdown of the Activities of the ICT Advisors for Graph 4.13 Breakdown of the Activities of the ICT Administrators for Graph 4.14 Activities of the ICT Education Officers ( ) 62 Graph 5.1 Trends in Traffic to Scoilnet in the month of May for the period Graph 5.2 Trends in Monthly Traffic to Scoilnet June 2003-April Graph 5.3 Traffic to Scoilnet in Graph 5.4 Traffic to Scoilnet in 2005 by Day of the Week 77 Graph 5.5 Scoilnet Search Terms in Graph 5.6 Frequency of Scoilnet usage 79 Graph 5.7 Parts of Scoilnet visited by teachers surveyed 79 Graph 5.8 Teacher ratings for certain elements of Scoilnet 80 viii

9 Graph 5.9 Trend in weekly traffic to the Software Central website ( ) 81 Graph 5.10 Trends in the weekly traffic to the SIP website 85 Graph 5.11 Number of school visits conducted or initiated by the ICT Advisors ( ) 91 Graph 5.12 Teachers Ratings of ICT Advisory Services 94 ix

10 Executive Summary This Value for Money Review of the ICT Support Service for Schools has been undertaken to analyse the expenditure of some 50m on the Service between 1998 and 2005, so as to inform future decisions in relation to the Service. The ICT Support Service for Schools was introduced as part of the overarching ICT in Schools Programme to advance and embed the use of ICTs in schools. The expenditure of 50m represents 29% of the total investment in the period on the ICT in Schools Programme. The ICT Support Service encompasses the work of the ICT Advisory Service, the two ICT Education Officer posts in the NCCA and the work of the NCTE on its Teaching Skills Initiative, its School Support Initiative and its Innovative Projects Initiative. The Review identifies the objectives of the Service and examines their validity and compatibility with education policy objectives. It defines the outputs associated with the activities of the Service and identifies the level and trend of those outputs. Having identified the costs associated with the Service it considers the efficiency with which the objectives of the Service have been met. The effectiveness with which the objectives of the Service have been achieved is addressed and the rationale for continued funding is considered. The scope for alternative approaches to achieving the Service objectives is examined and potential future performance indicators to monitor the performance of the Service are considered. Service Objectives Identified The objective of the ICT Support Service for Schools is to support the realisation of the overarching objectives of the ICT in Schools Programme, which are to promote the use of ICT in schools and to encourage pupils to achieve computer literacy and acquire the necessary skills for participation in the Information Society. The specific objectives of the component parts of the ICT Support Service for Schools are: NCTE To promote, facilitate and advise on the use of ICT in schools and to support its integration in teaching and learning in schools. In support of this objective the NCTE aims to provide an ICT teacher professional development programme; to maintain and develop Scoilnet; to promote and support the application and development of digital content; and to foster more innovative use and integration of ICT in schools. ICT Advisory Service To support the implementation of the ICT in Schools Programme at a local level and help build a knowledge base on good practice in relation to ICT in schools. In support of this objective the ICT Advisors aim to provide pedagogical advice and support to teachers and schools in utilising and integrating ICT in teaching and learning; and to support the Teaching Skills Initiative by identifying local training needs and preparing appropriate programmes of training courses. NCCA ICT Education Officer Posts To further the integration of ICT into curriculum and assessment. In support of this objective, the NCCA aims to develop a theoretical base for ICT in curriculum and assessment; to develop guidelines and resource materials for teachers; to specify the ICT skills that pupils should acquire at each stage of education; and to contribute to the training of trainers who deliver in-career development to teachers. x

11 Are Objectives Valid and Compatible with Education Policy? The EU and OECD contexts, the domestic Information Society context and the domestic education policy contexts within which the Service operates have been considered. The Steering Committee concluded that the objectives of the Service continue to be valid having regard to the Programme s overarching objectives and the wider educational, economic and Information Society contexts, both nationally and internationally. There is a need to articulate a policy framework for the overall ICT in Schools Programme and the Steering Committee noted the NDP provision of 252m to provide for a comprehensive Schools ICT Strategy in this context. Efficiency Findings The Steering Committee concluded that the overall expenditure of some 50m on the ICT Support Service was generally efficient in delivering on the objective of promoting and supporting the integration of ICT in teaching and learning in schools. The expenditure had facilitated the provision of a range of ICT-related continuing professional development courses for teachers; general support and advice to schools; a range of digital content, including on-line resources and software resources; pedagogical support for teachers; a range of innovative projects to encourage the use of ICT in a variety of settings; and had also facilitated the integration of ICT into curriculum and assessment. Effectiveness Findings The extent to which the ICT Support Service for Schools has met the objectives set for it was examined. In doing so, the Steering Committee noted that there are inter-relationships between the various elements of the Service and it is difficult to measure the particular influence of each on the extent to which ICT is used in teaching and learning in Schools. The Steering Committee found that, having regard to the resources available to it, the ICT Support Service had generally been effective in meeting the objectives set for it. The Steering Committee found the Teaching Skills Initiative has been an effective strategy for enhancing teachers skills and confidence in using ICT in their teaching and learning the Schools Support Initiative has been effective in increasing awareness and usage of the internet in schools the Innovative ICT Projects have been effective in meeting the outcomes identified for them including the collaboration between schools, universities and industry, the development of best practice models and methods for introducing innovative ICT to Irish schools, the creation of a research base for ICT in education and stronger ICT programmes in pre-service teacher training programmes the NCCA ICT Education Officer posts have been effective in pursuing the integration of ICT into teaching and learning, having regard to the relevant curriculum developments The Steering Committee considered the NCTE s ability to engage in meaningful long-range planning to be hampered by the uncertainty surrounding its future and staff, a situation which is compounded by the absence of governance structures. The Steering Committee concluded that the effectiveness of the ICT Advisory Service had not been clearly demonstrated having regard to awareness and usage levels. The Committee believed there was a compelling requirement to ensure that the integration of ICT is specifically incorporated into the range of support services operated by the DES. This strategic approach would be more effective in ensuring that ICT integration is not simply an add-on to existing practice but is fully integrated in all teacher continuing professional development and support service activities. xi

12 Recommendations While much progress has been achieved, there is substantial work to be undertaken to fully integrate ICT into teaching and learning in Schools. This position is reflected across Europe. In its Review of Studies of ICT Impact on Schools in Europe, the European Schoolnet noted: The use of ICT in education and training has been a priority in most European countries during the last decade, but progress has been uneven. A small percentage of schools in some countries have embedded ICT into the curriculum, and demonstrate high levels of effective and appropriate use to support and transform teaching and learning across a wide range of subject areas. Most schools in most countries, however, are in the early phase of ICT adoption, characterised by patchy uncoordinated provision and use, some enhancement of the learning process, some development of elearning, but no profound improvement in learning and teaching. 1 Specific recommendations included: Structures: The NCTE should be established on a statutory/corporate basis as a dedicated agency for implementing national policy on ICT integration in schools and there should be a review of its staffing in the context of the new ICT Strategy. Teaching Skills Initiative: There should be greater integration of ICT into continuing professional development activities, more in-school training, courses to complement the ICT framework in curriculum and assessment, more subject-based ICT courses and continued targeting of ICT training to teachers, including towards older teachers. Schools Support Initiative: The extent and range of resources on Scoilnet need to be expanded to ensure the ready availability of curriculum relevant digital content. The number of subject-specific software titles on the Software Central site for the postprimary sector should be increased, and critical gaps in the resources for particular subject areas should be identified with a view to meeting them with Innovative ICT Projects. Innovative Projects: There should be a National Co-ordinator for the evaluation and dissemination of material from Innovative ICT Projects, and for identifying and managing potential areas for new projects. ICT Advisory Service: The existing arrangements for the ICT Advisory Service should be reviewed. There is a compelling requirement to ensure that the integration of ICT is specifically incorporated into the range of support services operated by the DES. This strategic approach would be more effective in ensuring that ICT integration is not simply an add-on to existing practice but is fully integrated in all teacher continuing professional development and support service activities. The resources used by the ICT Advisory Service should be diverted into an alternative arrangement that focuses supports for ICT leadership and change within each school. In this context, the potential for extra professional development and ICT supports for school principals is highlighted, as is the potential for supporting and enabling the role of the school ICT Co-ordinators as leaders/champions of ICT use and integration. NCCA Posts: The provision of temporary additional resources to the NCCA to implement the ICT Framework, and to further the integration of ICT in curriculum and assessment, should be continued. The effectiveness of the ICT Support Service for Schools is dependent on the articulation of a strategy for the ICT in Schools Programme as a whole and the Steering Committee noted the investment signalled in the NDP for a comprehensive Schools ICT Strategy. 1 The ICT Impact Report: A Review of Studies of ICT Impact on Schools in Europe European Schoolnet, pg 2 xii

13 Key Themes in Submissions Received The absence of a national policy for ICT in Schools together with the lack of any general ICT financial provision to schools since 2003 was raised in the majority of submissions. The need to articulate a multi-annual policy and implement a multi-annual investment programme was repeatedly highlighted. The submissions were received prior to the publication of the NDP, which includes provision for 252m for a comprehensive Schools ICT Strategy. The overall effectiveness of the ICT Support Service for Schools was seen to be adversely affected by the absence of a general ICT in Schools policy and specific funding therefor. It was also noted that the Service was attempting to change pedagogy and practice in an education system that was reluctant to change where new and unproven technologies are concerned. Given the evolving and ever changing nature of technology it is important to build a vision around the future deployment of ICT provision in schools based at the very least on a three to five year plan driven by an implementation agency which can deliver on that plan over the longer term. The need to address the technical support requirements of schools was raised repeatedly. The broadband capacity being provided to schools was generally considered to be inadequate. The provision of adequate broadband capacity together with a concerted drive towards linking to and developing appropriate content was advocated in order to maximise the learning potential from having a broadband connection. Consultative Forum With the lapse of the National Policy and Development Committee, there is no mechanism for education partner input to ICT strategies and future policy. Some submissions advocated the establishment of such a forum to facilitate the articulation of an appropriate vision to pursue the integration of ICT in education in a meaningful and educationally defensible manner. NCTE Several submissions felt that the NCTE had achieved far more than could have been expected of it, having regard to its limited resources, its ad-hoc status, operating on a year-to year basis and the general policy void within which it operates. There was a consensus that it had provided a very efficient service to schools and had considerable impact on ICT usage in schools. Its achievements in delivering a range of courses in ICT; in sourcing a range of highquality digital resources, in developing the Scoilnet web-portal and associated websites; in supporting the range of SIP projects; in publishing a range of advice packs/ infrastructural surveys; in facilitating national ICT events and in representing Ireland in international fora were generally acknowledged. The need to establish the NCTE on a statutory basis with a Board of Management and allow it plan to implement a multi-annual strategy was highlighted. ICT Advisory Service While ICT advisors were seen to provide a very valuable service to schools, their effectiveness was argued to be curtailed by the ratio of ICT advisors to schools and the unrealistic expectations often made of them. The structural arrangements whereby the ICT advisors are employed by Education Centres were seen to be problematic. The service was seen to be particularly effective in supporting SIPs and specific initiatives, such as Fís. The challenge envisaged was to establish a truly nationwide service focussed on the integration of ICT into pedagogical practice. It was noted that there is a need to develop a standard mechanism for capturing data/feedback at local level and for bi-directional data exchange between local and national levels. Teacher Professional Development The Teaching Skills Initiative was seen to have provided a comprehensive and responsive service to schools in the area of teacher ICT training and development. It was suggested that there could be greater co-operation between the teacher training colleges and the ICT Support Service, in terms of the accreditation of courses leading to recognised postgraduate qualifications which would meet the needs of practising teachers, providing a range and depth of professional development courses while embedding their development within a sound learning framework. xiii

14 Digital Content While Scoilnet was seen to have been a positive development, along with the Software Central website, there were calls for the availability of curriculum relevant digital resources to be pursued as a priority. SIPs (Schools Innovative Projects) The need to develop projects which meet the needs, interests and experiences of teachers and students while having a well-defined third level link and research agenda, was articulated. Work of the NCCA The work of the NCCA in relation to the ICT Framework was generally welcomed and the critical bridging role played by the ICT Education Officer Posts between the NCTE and NCCA was acknowledged. xiv

15 Terms of Reference Review of ICT Support Service for Schools Schools ICT Support Service National Council for Curriculum and Assessment National Centre for Technology in Education Education Centres ICT Education Officers Teaching Skills Initiative Schools Support Initiative Innovative Projects ICT Advisory Service Identify the objectives of the Service and examine the current validity of those objectives and their compatibility with education policy objectives and the Department s policy for ICT in education. Define the outputs associated with the activities of the Service and identify the level and trend of those outputs. Identify the level and trend of costs and staffing resources associated with the Service and thus comment on the efficiency with which objectives have been achieved. Examine the extent to which the objectives of the Service have been achieved, and comment on the effectiveness with which they have been achieved. Evaluate the degree to which the objectives of the Service warrant the allocation of public funding on a current and ongoing basis and examine the scope for alternative policy or organisational approaches to achieving these objectives on a more efficient and effective basis. Specify potential future performance indicators that might be used to better monitor the performance of the Service. xv

16 Chapter 1 Introduction Digital technologies are best understood as tools for thought. There is no important domain of human or economic activity unaffected by them. Their key significance lies in amplifying brainpower in much the same way that the energy technologies of the industrial revolution amplified muscle power. (Information Society Commission, 2005) 1.1 Introduction This chapter outlines the background to the Value for Money and Policy Review process (also known as the Expenditure Review Initiative), the methodology and terms of reference for this review and a general overview of the ICT in Schools Programme, which encompasses the ICT Support Service for Schools. 1.2 Background to the Value for Money Review Process The Expenditure Review Initiative (ERI), which commenced in 1997, is a systematic process of evaluation conducted by Government Departments and Offices under the guidance of the Expenditure Review Central Steering Committee and the Department of Finance. Its objectives are to analyse Exchequer spending in a systematic manner and to provide a basis on which more informed decisions can be made on priorities within and between programmes. The Comptroller and Auditor General (Amendment) Act, 1993 and the Public Service Management Act, 1997 set the context for expenditure reviews in terms of the achievement of economy, efficiency and effectiveness, and the maintenance of appropriate systems, practices and procedures for the purpose of evaluating effectiveness 2. Expenditure reviews are also consistent with the Strategic Management Initiative (1996) as they encourage Government Departments and Offices to move away from the traditional focus on inputs to concentrate more on the achievement of results. In June 2006 the ERI was enhanced and renewed as the Value for Money and Policy Review process. The Department of Education and Science (DES) has conducted this Value for Money (VfM) Review of the ICT Support Service for Schools under the Value for Money and Policy Review Process. 2 Expenditure Review Initiative First Formal Report to the Minister of Finance by the Expenditure Review Central Steering Committee, For the period June 2002 June 2004, page 41 1

17 1.3 Methodology and Terms of Reference Establishment of the Steering Committee Membership of the Steering Committee was drawn from the DES and the National Centre for Technology in Education (NCTE). The Sectoral Policy Division of the Department of Finance was represented at the initial meeting. Details are as follows: Mary McGarry, Principal Officer, ICT Policy Unit, DES Enda Hughes, Assistant Principal Officer, DES Antoinette Ryan, Administrative Officer, ICT Policy Unit, DES 3 Fintan O Brien, Assistant Principal Officer, Central Policy Unit, DES George Porter, Post-primary Inspector, DES Jerome Morrissey, Director, NCTE Niall O Sullivan, Administrative Officer, Sectoral Policy Division, Department of Finance (meeting of 1 st December, 2005) The Steering Committee agreed the terms of reference for the VfM review at its first meeting on 1 st December These terms of reference were then approved by the Secretary General of the DES. Following the planning stage of the review, the field work on the Review commenced in May The Steering Committee guided the direction of the field work, which was undertaken by Mr. Enda Hughes, and subsequently considered the material prepared and agreed the analyses and recommendations Terms of Reference In its deliberations, the Steering Committee considered the merits of conducting a review of the entire ICT in Schools Programme, which aims to promote the use of ICT in schools and encourage pupils to achieve computer literacy and acquire the necessary skills for participation in the Information Society. Such an approach would have included the range of capital supports provided to schools, the Schools Broadband Access Programme and the work of the ICT Policy Unit in the DES. The Committee concluded that the inclusion of the full Programme would have become too unwieldy. The inclusion of the capital expenditure would have necessitated an examination of various models for the provision of ICT equipment such as central procurement agreements as against grant aid to schools. It was felt that such an examination would warrant an expenditure review in its own right and accordingly, it was agreed to confine the current review to the ICT Support Service for Schools. The Steering Committee agreed the scope of the review to include the work of the NCTE on the Teaching Skills Initiative, the Schools Support Initiative and on Innovative Projects, 4 the ICT Advisory Service and the two ICT Education Officer posts in the National Council for Curriculum and Assessment (NCCA). The review examines whether or not the ICT Support Service for Schools should be continued in its current format and what changes, if any, should be made to enhance its efficiency and effectiveness having regard to the needs of schools, teachers, learners and evolving education policy. The agreed terms of reference for the review are to: 1) Identify the objectives of the Service and examine the current validity of those objectives and their compatibility with education policy objectives and the Department s policy for ICT in education. [Chapters 2 and 3] 2) Define the outputs associated with the activities of the Service and identify the level and trend of those outputs. [Chapter 4] 3 Ms. Ryan resigned from the Steering Committee in November The other work areas of the NCTE include the Technology Integration Initiative, the Schools Broadband Programme and the EU-funded work undertaken in the area of Internet Safety. 2

18 3) Identify the level and trend of costs and staffing resources associated with the Service and thus comment on the efficiency with which objectives have been achieved. [Chapter 4] 4) Examine the extent to which the objectives of the Service have been achieved, and comment on the effectiveness with which they have been achieved. [Chapter 5] 5) Evaluate the degree to which the objectives of the Service warrant the allocation of public funding on a current and ongoing basis and examine the scope for alternative policy or organisational approaches to achieving these objectives on a more efficient and effective basis. [Chapter 6] 6) Specify potential future performance indicators that might be used to better monitor the performance of the Service. [Chapter 7] In confining the scope of the VfM review to the ICT Support Service for Schools, the Steering Group acknowledged at the outset that the effectiveness of the service would be significantly influenced by the extent of the capital supports for ICT equipment provided to schools and the availability of broadband services under the ICT in Schools Programme. Clearly, teachers with the necessary skills and motivation to integrate ICT into their teaching and learning can only do so when they have access to the necessary ICT facilities. Easy access to reliable, appropriate ICT equipment by teachers and students is a sine-qua-non for meaningful engagement with the technology in their teaching and learning. Similarly, embedded, reliable and high-capacity broadband in the classroom increases the quality and quantity of educational activities that can be undertaken. 5 Figure 1.1 provides a graphical presentation of the ICT Support Service for Schools and the linkages between the different support services considered in this review. Figure 1.1 Graphical Presentation of the ICT Support Services for Schools Schools ICT Support Service National Council for Curriculum and Assessment National Centre for Technology in Education Education Centres ICT Education Officers Teaching Skills Initiative Schools Support Initiative Innovative Projects ICT Advisory Service 5 The ICT Impact Report; A Review of Studies of ICT impact on Schools in Europe, European Schoolnet, December 2006; pg 5 (quoting the Impact of Broadband in Schools Underwood et al, 2005) 3

19 1.3.3 Methodology and Consultation Process A number of complementary approaches were used during the review. An active theory of change was developed using Programme Logic Models (PLM) via semi-structured interviews and informal contact with the various stakeholders involved in the delivery of the programme. The Programme Logic Model involved the identification of the strategic objectives for the service as a whole and the objectives of each individual component. Then the inputs, activities, outputs and outcomes (in terms of results and impacts) were identified for each component. 6 These in turn provided a framework for the review, which facilitated the efficiency and effectiveness analyses and helped to identify performance indicators. A series of semi-structured interviews with the CEO and staff in the NCCA, the Director and four National Co-ordinators in the NCTE and the Directors, ICT Advisors and other staff in two of the full-time Education Centres were held between June 2006 and January These semi-structured interviews helped with the development of the PLMs for each element of the support service under review. They provided valuable insights into how the various services were delivered, helped to clarify issues and provided an opportunity for stakeholders, involved in delivering services, to highlight issues. Management statistics in the NCTE were used extensively throughout the review and other data sources gathered by the NCTE were also referenced, such as the 2005 NCTE Census of ICT Infrastructure in Schools and previous NCTE censuses. Data from other sources including Teacher Education Centres, Teacher Education Colleges, Universities, Institutes of Technology and the Central Statistics Office was sought as required. The Evaluation Support Research Unit (ESRU) of the Department s Inspectorate undertook an evaluation of the impact of ICT on teaching and learning in 2005/06, which involved a national survey of school principals and teachers, the use of ICT Review Schedules during classroom inspections as part of whole school evaluations and subject inspections, complementary visits by school inspectors to a number of case study schools and a student questionnaire administered in the case study schools. The Steering Committee relied on the data from the report of this evaluation to assess outcomes for a number of the support services under review. During the review it became apparent that there were gaps in the information relating to the ICT Advisory Service and questionnaires were issued to both the ICT Advisors and the Directors in the full-time Education Centres. The ICT Advisors questionnaire helped to gather information on activities and outputs for the service. The Directors questionnaire provided valuable information on the management of the ICT Advisors and the management and activities of the ICT Administrative support provided to the ICT Advisors. Both questionnaires included a number of open ended questions which gave respondents opportunities to raise issues about elements of the ICT support service. Submissions were invited from the key stakeholders, including the main educational partners, staff and representative bodies involved in delivering the services involved, groups representing industry and a number of academics specialising in ICT in education. Appendix 2 lists the submissions received and the key themes are summarised on page xiii. Finally, a review of relevant literature and published research documents was undertaken as part of the process Quality Assessment When the Report was completed, the Steering Committee engaged Mr. Kealan Flynn, Principal, iwrite, who is a member of the Independent Panel of Evaluation Experts, to consider the evaluative process and methodologies. The finalised Report takes account of Mr Flynn s suggestions. 6 The PLMs are included in Appendix 1. 4

20 1.4 History of the ICT in Schools Programme Schools IT2000, A Policy Framework for the New Millennium The ICT in Schools Programme commenced with the publication of Schools IT2000, A Policy Framework for the New Millennium in At that time, Ireland lagged significantly behind its European partners in the integration of ICT into first and second level education and the need for ICT integration into teaching and learning right across the curriculum was seen as a major national challenge which needed to be addressed in order to safeguard Ireland s future economic wellbeing. 7 The core objective of Schools IT 2000 was to establish a permanent infrastructure, which would ensure that all pupils have the opportunity to achieve computer literacy and to equip themselves for participation in the information society and teachers are supported to develop and renew their professional skills, so as to enable them to utilise ICTs as part of the learning environment. Three strategies were pursued to achieve this objective. The first strategy was the development of a Technology Integration Initiative to ensure that there would be at least 60,000 multimedia computers in Irish schools by the end of 2001 and the connection of every school to the internet within two years. The second strategy involved the development of a Teaching Skills Initiative, which would provide professional skills development in ICTs to at least 20,000 teachers, the establishment of distance learning programmes and the introduction of pre-service training in the use of ICTs in education for all student teachers. Finally the third strategy involved the development of a Schools Support Initiative to include the introduction, in co-operation with the National Council for Curriculum and Assessment (NCCA), of curriculum innovations to enhance learning through the use of ICTs in the classroom, the establishment of a national network to advise and support schools, incorporating the development of Scoilnet and the Schools Integration Project. Of the 50.8m funding committed under Schools IT2000, at least 60% was to be invested in infrastructure with the remainder being devoted to training, curriculum resources and support. The actual expenditure under the initiative was some 52m, with 31m (60%) being capital investment in infrastructure. This funding was complemented by private sector support by companies such as Telecom Éireann which undertook to invest 12.7m ( 10m) to support the increased integration of ICT into first and second level education (see para 1.4.5) Implementation Structures Established under Schools IT2000 The Schools IT2000 programme was managed within the Department by the IT Unit up until August 2000, when the ICT Policy Unit was established to manage the ICT in Schools Programme. The section works closely with the NCTE which was established in 1998 to implement Schools IT2000 and which, in turn, directs the network of ICT advisors operating out of the full-time Education Centres. The NCCA is also actively involved in the Programme and is specifically resourced in this regard National Policy Advisory and Development Committee The National Policy Advisory and Development Committee (NPADC) was established pursuant to the Schools IT2000 initiative and comprised representation from the partners in education and the social partners. Its remit was to advise the Minister on ICT development needs in Irish education, to investigate future policy directions and to liaise in the formulation of ICT policy advice. The Committee was not reconstituted following the expiry of its first term, in Schools IT2000: A Policy Framework for the New Millenium, Department of Education and Science 1997, pg 2 5

21 1.4.4 Blueprint for the Future of ICT in Irish Education 3 Year Strategic Action Plan The Blueprint was the second strategic action plan to enhance ICT in Irish schools. It focussed on augmenting the ICT capital provision to schools; expanding access to, and use of, Internet technologies; further integrating ICTs into school curricula and the enhancement of teacher professional development. The Action Plan identified the critical role of the school principal in facilitating ICT development at school level and envisaged that every principal would be provided with the opportunity to attend workshops in the local Education Centres, which would assist them consider the ICT needs of their schools, raise awareness of the optimum level of ICT infrastructure and advise on ICT planning at school level. Investment of m was committed under the Action Plan, comprising 78.72m capital and 29.2m current. The actual expenditure under the Blueprint amounted to some 77.7m, comprising 58m capital and 19.7m current Schools Internet Access Scheme In parallel with the launch of Schools IT2000, Telecom Éireann (Eircom) agreed, under the Information Age Schools Scheme, to fund the provision of a computer, a phone line rental and dial-up internet access (up to a limit of 1 hour per day) for a period of three years for all schools. This arrangement continued up to June 2002 when Eircom decided to conclude its sponsorship. The Department came to an arrangement with Eircom in relation to the continuation and enhancement of the service to schools in July The Department agreed to subvent the cost of schools internet phone line rentals and to meet the cost of up to 2 hours dial-up internet access per day, based on vouched usage. This arrangement applied for the 2002/03 academic year, at a cost of 2.48m. In autumn 2003, while the Department was considering the feasibility of providing broadband connectivity to schools, it agreed to support school internet access by way of a direct grant to schools of 200 towards the cost of a PSTN (basic analogue) phone line. This support cost 787,200 in Schools whose internet connectivity was provided via an ISDN (digital) line and whose costs were therefore higher, were given a further grant of 100 in early 2004, at a cost of 151,000. The overall cost of this provision for the 2003/04 academic year was 938, Computer Networking Grants The existence of a school network enables multi-user access to the Internet and facilitates the sharing of software resources and the shared use of peripherals. The NCTE 2002 ICT Census found that only 30% of computers in primary and 70% of computers in post-primary schools were networked. Having regard to the Department s intention to provide broadband connectivity to schools, the development of internal networks in schools was identified as a priority need for support in the period, within available resources. Accordingly, schools were requested in June 2004 to submit proposals for the development (or expansion) of their networking capability. Grants amounting to some 23m have issued to schools Schools Broadband Access Programme This programme is being undertaken in partnership with industry. The Agreement with IBEC/TIF (Telecommunications and Internet Federation) provides for a three year 18m joint Fund, with industry contributing 5 million per annum and the Government contributing 1 million per annum, and meets the costs of schools local connectivity. Under the Programme, schools connectivity is being routed to the Internet through a National Broadband Network, which has been developed by HEAnet. Each school is connected to the network via a highspeed broadband router, provided to the school and housed in the school, and a broadband access link. The Network provides centrally managed services for schools such as security, anti-spam / anti-virus and content filtering. Following the procurement process, the roll-out of broadband connectivity commenced in June 2005 and continued during The vast majority of schools have been connected to the Network using a mixture of DSL, wireless, and satellite technologies and the Department is monitoring the position of the outstanding schools, where the installation is awaited due to building programmes or installation 6

22 difficulties. The overall cost of the Schools Broadband Access Programme, including the initial set-up and ongoing costs over the three years, was estimated at some 30m. 1.5 Investment in the ICT in Schools Programme Between 1998 and 2005, some 173m has been invested under the Programme ( 114m capital and 59m current investment) as follows: Table 1.1 Investment in the ICT in Schools Programme Capital Current Total ,132,098 3,809,214 20,941, ,998,049 9,012,019 19,010, ,792,796 8,257,257 12,050, ,570,875 4,759,356 32,330, ,748,260 7,446,283 31,194, ,510,155 7,698,059 14,208, ,719,533 7,964,219 27,683, ,585,506 9,582,265 15,167, ,057,272 58,528, ,585,944 The expenditure of 15m on the Programme in 2005 represented 0.2% of the Department s overall expenditure of 6,934m that year. Expenditure on the ICT Support Service for Schools Within the overall expenditure of 173m on the Programme to end 2005, expenditure on the ICT Support Service for Schools accounted for some 50m, or 29% of the total investment. The 50m can be roughly apportioned as 39.4m on the NCTE and its initiatives, 9.6m on the ICT Advisory Service and 0.6m on the NCCA ICT Education Officer posts. This review considers the investment made in the ICT Support Service for Schools between 1998 and

23 Chapter 2 General Overview of the ICT Support Service for Schools 2.1 Introduction This chapter provides a general overview of the various elements of the ICT Support Service for Schools, commencing with the NCTE and the various initiatives under its remit. The work of the ICT Advisory Service is then outlined followed by an overview of the relevant work of the NCCA. 2.2 The NCTE and its Initiatives The NCTE The NCTE was established in January 1998 following the signing of a Memorandum of Agreement between the Department of Education and Science and Dublin City University (DCU) dated 30 th January This Memorandum established the NCTE within Dublin City University. The Memorandum allowed for the establishment of a Board of Management for the NCTE and set out the funding arrangements for the NCTE and how the annual accounts and reports would be presented. A Board of Management was established in April 1998 and operated until its term expired in December A new Board was appointed in October 2003 for a one year term, during which period the remit and structure of the NCTE was to have been reviewed. The Board met on a number of occasions and effectively lapsed in April Having been established under a Memorandum of Agreement, the NCTE is not a corporate or statutory entity and neither is it a constituent part of DCU. In practice the NCTE reports directly to the Department with DCU s role being confined to providing office accommodation, personnel and accounting/financial services in respect of which it receives an annual fee. Many of the provisions of the Memorandum are out of date and not operated. The NCTE agrees an annual budget (current and capital) and work programme with the DES and its finances are channelled via DCU. The NCTE has a staffing complement of 22 and an annual budget of approximately 8m in Initiatives operated by the NCTE The NCTE operates a number of initiatives under the ICT Support Service for Schools. The initiatives that are being considered as part of this review include the Teaching Skills Initiative; the Schools Support Initiative, encompassing the Scoilnet Initiative and the Interactive Software in the Curriculum Initiative; and the Innovative ICT Projects Initiative. The NCTE also operates the Technology Integration Initiative, which comprises a number of programmes to support improved levels of ICT infrastructure in schools. It supports the Schools Broadband Access Programme and manages the National Service Desk, which interfaces between the Network, Broadband Service Access Providers and Schools and provides schools with on-going advice and assistance. Both of these latter areas are outside the scope of this VfM review, as is the considerable work undertaken by the NCTE in the area of Internet Safety and work undertaken under EU Funded Projects. 8