Video Conferencing Project CYDAG

Transcription

1 Evaluation of Learning Challenge Fund Final Mini Evaluation Report Video Conferencing Project CYDAG MAY 2006 SHARED INTELLIGENCE 1 Fitzroy Square, London, W1T 5HE

2 CONTENTS 1. INTRODUCTION... 1 Video Conferencing Project... 1 LCF evaluation methodology PROJECT ACTIVITIES AND OUTPUTS... 4 Project activities... 4 Project progress... 4 Project outputs PROJECT ACTIVITIES AND IMPACT... 6 Outcomes and benefits CONTRIBUTION TO LCF OBJECTIVES... 7 Aims of the Learning Challenge Fund... 7 Sustainability and mainstreaming CONCLUSIONS AND LEARNING POINTS... 9 Conclusions... 9 Learning points... 9

3 1. INTRODUCTION 1.1. Video Conferencing Project Background and aims The school Videoconferencing Project was funded by LCF between December 2003 and July 2005 as a pilot project to support the establishment of videoconferencing in a number of schools in Wales The Executive Committee of CYDAG (Association of Schools for Welsh Language) submitted an application for LCF funding in April 2003 with a target start date that would have allowed A/AS level courses to be provided for 6th form students from September Invitations were extended to all members of CYDAG and 12 schools applied. CYDAG used the following criteria to select the 6 participating schools geography, type of school, financial contribution for training, appropriate room to use as a studio, mix of videoconferencing experience. The following schools were selected: Caereinion High School in Llanfair Caereinion Ysgol Tryfan in Bangor Ysgol Maes Garmon in Mold Ysgol Morgan Llwyd in Wrexham Ysgol Aberteifi in Cardigan Ysgol Glantaf in Cardiff The approval process was longer than CYDAG had originally anticipated and the application was finally approved in December 2003, missing the start of the school year The aims of the CYDAG project were to install video-conferencing equipment in bilingual and Welsh-medium schools, enabling them to offer greater choice to 6th form pupils for A and AS level students; and to instigate a partnership that would allow bilingual and Welsh-medium schools to share expertise and resources and develop methodology skills in flexible teaching The objectives were to: get agreement between 6 schools to establish the project; adapt rooms in 6 schools and install equipment; train staff in use of video-conferencing equipment; provide teacher training and sharing good practice; prepare teaching materials in 9 subject areas; provide 200 teaching hours in each school in 9 subject areas It was anticipated that the equipment would allow the schools to offer a greater choice of subjects to 6th form students who would be able to receive lessons delivered by teachers located at any of the other schools in the group. The schools SHARED INTELLIGENCE Page 1

4 were added to the Welsh Video Network (WVN) which had been established to allow universities and colleges across Wales to share tutorials and lectures In consultation, the project manager outlined some of the challenges and barriers that he believed the pilot project faced from the outset; challenges that would have to be addressed: Coordinating the project across 6 locations throughout Wales. Delivering the building refits and equipment installation on 6 sites to a very tight timetable. Staff and management changes in schools and varying levels of confidence amongst teachers. Coordination between schools differing timings of school day and different timetable structures. Forecasting demand locally, when students inevitably change their minds about A/AS level courses. Project structure and funding The project was overseen by CYDAG and coordinated on behalf of CYDAG by the head teacher of Caereinion High School in Powys, one of the participating schools. He was a member of CYDAG s Executive. Senior managers were chosen from the other 5 schools to act as local project managers. The Project Committee also involved CYDAG s Professional Officer, a representative of ELWa s Bilingual Unit and a representative of Canolfan Sgiliaith. The first meeting took place in January 2004 and the committee met at least every two months thereafter CYDAG received 448,000 over approximately 18 months from the LCF. The bulk of the funding (some 62%) was targeted on the capital programme involving studio refit, equipment and ISDN line installation. The remaining funding was allocated to studio maintenance, ISDN line rental, call charges and support services from WVN LCF evaluation methodology Shared Intelligence has been commissioned by ELWa to evaluate the Learning Challenge Fund on an ongoing basis. This comprises a Mini-Evaluation of each of the LCF projects, as well as an assessment of the LCF s progress as a scheme. The Mini-Evaluations are not intended to duplicate the projects own evaluations which are a requirement of their LCF contracts with ELWa The LCF evaluation programme began in The first stage of the evaluation reviewed the process of setting up and administering the Learning Challenge Fund. The evaluation programme was extended out to individual projects in summer The Video Conferencing Project was funded for approximately 18 months from December 2003 until July This Mini-Evaluation is based on: a meeting with the project manager and ELWa contract manager in Llanfair Caereinion in October 2005 a meeting with the Head of the ELWa Bilingual Unit and the project manager SHARED INTELLIGENCE Page 2

5 of Canolfan Sgiliaith at the Fforwm Conference in Cardiff in May 2005 an introductory meeting with the contract manager in Newtown in May 2005 consultation interview with CYDAG representative a review of project documentation, regular monitoring reports and the selfevaluation report produced in September It is believed that such a spread of consultation and research has provided the basis for an effective assessment of activities, outputs and strategic impacts from the perspective of ELWa, stakeholders and the project manager. SHARED INTELLIGENCE Page 3

6 2. PROJECT ACTIVITIES AND OUTPUTS 2.1. Project activities The components of the project included: Selecting the 6 participating schools Refitting a room in each to create a studio Installing and testing high quality videoconferencing equipment Developing partnerships between and amongst participating schools Training staff in use of equipment and in teaching methods Delivering videoconferencing teaching to students in different schools 2.2. Project progress Selection of Schools CYDAG selected 6 schools from a group of 12 applicants. The Head of the Bilingual Unit said that with hindsight she would have preferred this process to have been more robust and participative and for ELWa to have had more input Refitting and Installation - The equipment was installed in all 6 schools but the planned timetable proved too ambitious. Only 2 studios were ready to start at the beginning of the school year in September The process was particularly complex in one school where the LEA took the view that they would wish to take responsibility for the installation and in others there were unforeseen problems such as asbestos and the need to move pipes. However all studios were completed and basic staff training undertaken by the end of September The project coordinator believes the capital component of the project required a longer timescale. Procurement of equipment under EU rules takes 3 months and he suggested a 6 month period to include procurement, refit, installation and testing Partnerships The project reinforced partnerships between schools that already existed and created some new ones. However the project coordinator felt that effective partnerships had only been established in 4 of the schools. The reasons for this related to the problems of timetable coordination, the levels of demand from students and staff confidence levels. He realised that the more schools that have the equipment and training, the greater are the opportunities for effective multi-school partnership Training The take-up of staff training across the 6 schools was patchy, with some schools only training a very small number of staff. The project coordinator believes the training component of the project should also be extended to 6 months, beginning only after the installation and testing of equipment. Training would begin with technology and familiarisation and then move on to teaching methodology and preparation of new materials to exploit the videoconferencing opportunities. This would mean a project development phase of 12 months before staff were expected to begin teaching. SHARED INTELLIGENCE Page 4

7 Delivering Teaching The effective offer of courses for delivery by videoconferencing has varied in each school. The project coordinator believes that there was real enthusiasm to participate but that the short project timescale and the problems with getting some of the studios up and running and the staff trained in time for the start of the school year had an unfortunate impact on delivery in some schools Project outputs The internal evaluation report in September 2005 outlined the achievement of the operational objectives. Objective Target Agreement from schools Completed June 2003 Rooms adapted and equipment installed Rooms adapted and equipment installed Staff trained Teaching methods Materials in 9 subjects 200 hours of teaching in each school Two schools completed in time (September 2004) but others delayed due to physical and organisational reasons. All completed by end of trained in use. From 4-54 per school. 34 trained in teaching methods. From 1-15 per school. 6 subjects achieved due to changed demand. Only achieved in 3 schools. Some con below Additional training for 106 staff in 4 schools was funded by Global-Leap, a body set up in England by the DfES with significant experience of using videoconferencing in the classroom. Training mainly involved curriculum enrichment and proved successful in stimulating teachers initial interest A survey of students undertaking videoconferencing teaching was very positive. It found that 88% considered the course to have been very good or good; and the same percentage found the tutors and facilities very good or good. It also found that 76% felt the teaching materials very good or good. No student found any of these categories poor. The survey also asked about students views on whether the videoconferencing courses had better equipped them in terms of confidence and employability, but the results were inconclusive. This might have been expected as many students had only had limited exposure to such teaching. SHARED INTELLIGENCE Page 5

8 3. PROJECT OUTCOMES AND IMPACT 3.1. Outcomes and benefits This was a pilot project in just 6 schools across Wales and the key outcomes and impacts are therefore most likely to be in the lessons learned and in the establishment of effective processes that might be rolled out in future. This was particularly the case in this project because of its relatively short funding period where the measurable outcomes and impacts relating to learners cannot easily be identified. The project coordinator believes that it will really only be possible to assess how the teacher training and new teaching methods have impacted on demand and student performance after the end of the 2006 summer term. However the outcomes have been shown to include: Building of partnerships between some schools across Wales with the same issues and opportunities; Raising awareness of Welsh language and bilingualism in schools throughout Wales; Opening up opportunities for Welsh medium learning where they did not previously exist; Raising new technology and teaching skills amongst 100 teachers and support staff; Providing a foundation for extending videoconferencing facilities throughout Wales Providing experience of managing the process, helping identify good practice for future delivery. SHARED INTELLIGENCE Page 6

9 4. CONTRIBUTION TO LCF OBJECTIVES 4.1. Aims of the Learning Challenge Fund This section looks at the extent to which the Video Conferencing Project contributed to the aims and objectives of the Learning Challenge Fund. In summary, the key objectives and success criteria for the LCF, are as follows: innovation and new approaches in learning provision; a strategic approach at national, regional and local levels; collaboration between learning providers leading to improved learning networks; improving access and widening participation; delivering more and high-quality learning outcomes from existing resources (value for money); and effective mainstreaming of successful projects The contribution of each project towards the aims of the LCF is being assessed, partly to see whether the projects that were funded fitted with the LCF s aims, and partly to see whether the LCF as a whole achieved its aims. Therefore, this section summarises both the intentions of the project ( fit ), and the achievements of the project in terms of the LCF aims. Innovation This was a project with innovation at its heart, both in terms of technology and its application to learning. The creation of 6 state-of-the-art videoconferencing studios in schools incorporating translation software, interactive whiteboards, video recording facilities and document readers was a completely new development in Wales The whole approach can also be seen to be innovative in that it was aimed at creating partnerships between schools throughout the country and also at developing a relationship with WVN, bringing the experience gained in universities and colleges to the school sector. Strategic approach The pilot project has been strategic as it aimed to establish a platform for extending the videoconferencing approach across the whole school sector. It has a strong strategic fit with ELWa and WAG bilingual strategy (Meeting the Needs of Bilingualism in Post-16 Education) and linked well with other LCF projects (Canolfan Sgiliaith and Cyfrwng) However the Head of the Bilingual Unit believed that stronger drive and leadership perhaps through a dedicated project coordinator role with more capacity to influence and make strategic connections would have increased the strategic impacts of the project. SHARED INTELLIGENCE Page 7

10 Collaboration The project was partly successful in building collaboration between the 6 schools effective partnerships were formed between 4 of the schools. Collaboration was achieved successfully with WVN. It is anticipated that as more schools are introduced to the videoconferencing concept and a s teachers build their confidence with the equipment and the medium, the opportunities for collaboration will increase significantly. Improving access and widening participation The pilot project was focused on the development of infrastructure and capacity building in just 6 schools but was aimed at improving access to A/AS level courses for students who had previously been unable to access Welsh language and bilingual courses in many subjects. The impact on participation is only beginning to be felt but the fact that more schools will be able to offer videoconferencing provision will inevitably have a positive impact on access and participation. Delivering more and high-quality learning outcomes The timescale of the pilot project and the delays in getting everything in place limited the achievement of the short-term learning outcomes with students. However it is anticipated that the outcomes will emerge from summer 2006 and in future years, both in the original 6 schools and in subsequent host schools Sustainability and mainstreaming The pilot project has been successful in attracting funding for the roll-out of the videoconferencing concept across Wales. The United Kingdom Education and Research Networking Association (UKERNA) and WVN are currently working on the second phase of the project. UKERNA are undertaking a procurement process on behalf of ELWa for video conferencing equipment for another 8 Welsh medium or bilingual schools. ELWa have also extended the WVN support and maintenance contract for the original 6 schools and for another 12 schools as part of the new project. SHARED INTELLIGENCE Page 8

11 5. CONCLUSIONS AND LEARNING POINTS 5.1. Conclusions The school Videoconferencing Project has been an effective pilot project. It took a very innovative, ground-breaking and challenging remit and delivered it with some commitment over an 18 month period. The project was working in relatively uncharted territory for the schools sector, in relation to the technology, school collaboration, staff skills and student demand. The project met its capital objectives by getting the studios up and running, even if the programme was delayed. It was partly successful in building inter-school partnerships but there were some schools that did not collaborate fully. It was also partly successful in training up teachers and support staff in all schools and developing videoconferencing teaching opportunities. Three schools met their targets The project contributed effectively to the LCF objectives in terms of innovation, strategic impact and sustainability and to some extent in terms of collaboration. No doubt it will achieve wider access and more learning outcomes once it has had time to bed-in The project has provided a number of key learning points as a successful pilot is expected to do and these are being taken account of in developing the second phase of the videoconferencing roll-out across Wales Learning points From the experience of the project itself, key learning points would be that: The co-ordinator role should have been a dedicated position with an adequate time commitment available for the project. This is perhaps the biggest learning point, supported by all consultees. Initially allocated one day a week, the job was impossible. The job description was very wide with leadership, support and administrative tasks sitting side-by-side. The project coordinator admitted that he had to focus on the day-to-day administration tasks and was unable to travel, support and lead the partnership building process with the other schools. The co-ordinator role should not have been allocated to one of the local project managers, for reasons of independence and efficiency. The project coordinator had one day a week to deliver his own studio and training as well as to lead the national programme. That would result in difficult decisions about how to allocate time, that would inevitably harm the national project. More hands-on support with the building refit and equipment installation programme would have been beneficial. Each school had its own problems and constraints and would have benefited from visits and leadership. Early discussions must be held with the relevant LEA regarding refitting the premises. One LEA - understandably took the view that the responsibility for physical alterations to school premises should have sat with them. A longer-term commitment to the project would have allowed time for building teacher confidence, developing the curriculum and growing demand. It is considered that it takes 12 months to reach a stage where teachers are ready to begin delivering. SHARED INTELLIGENCE Page 9

12 The importance of training, practical experience and curriculum enrichment might have been underestimated in the project design. The Head of the Bilingual Unit believed the programme was too front-ended and had inadequate resources and time allocated to the people aspects It is considered that ELWa managed the project effectively both in terms of monitoring and managing the contract on the one hand and in supporting the wider strategic components of the project. They also managed the mainstreaming of the project as it neared the end of the LCF funding period and considered this within an overall national strategy. SHARED INTELLIGENCE Page 10