Report of External Evaluation and Review

Transcription

1 Report of External Evaluation and Review New Zealand Institute of International Understanding (NZIIU) Confident in educational performance Confident in capability in self-assessment Date of report: 3 July 2013

2 Contents Purpose of this Report... 3 Introduction TEO in context Scope of external evaluation and review Conduct of external evaluation and review... 5 Summary of Results... 6 Findings... 8 Recommendations Appendix MoE Number: 7447 NZQA Reference: C06133 Date of EER visit: 12 and 13 March

3 Purpose of this Report The purpose of this external evaluation and review report is to provide a public statement about the Tertiary Education Organisation s (TEO) educational performance and capability in self-assessment. It forms part of the accountability process required by Government to inform investors, the public, students, prospective students, communities, employers, and other interested parties. It is also intended to be used by the TEO itself for quality improvement purposes. Introduction 1. TEO in context Name of TEO: Location: Type: New Zealand Institute of International Understanding (NZIIU) Head Office, 8 Manukau Road, Epsom, Auckland Incorporated non-profit charitable trust First registered: 1986 Number of students: No domestic students Approximately 850 international students (Japanese and Thai) per year Number of staff: Seven full-time and one part-time management and administration staff Contracted teaching staff when required Contracted regional homestay managers Scope of active accreditation: Sites: Distinctive characteristics: Introduction to English Communication Certificate in Educational/Cultural/Homestay Programme Organisation (not currently delivering in 2012) Two sites Auckland and Christchurch Established in 1987, NZIIU is an incorporated nonprofit charitable trust approved by the Ministry of Education to promote international understanding through overseas student exchange. The programmes have been operating for over 20 years. 3

4 NZIIU facilitates exchange students between New Zealand and other countries, in addition to arranging and hosting tour groups, predominately from Japanese schools. Specific programmes are tailored for groups of young international students who come to New Zealand both for a cultural exchange and to improve their confidence in their English language skills. Recent significant changes: A restructure at management level has occurred in the past year. A new general manager was appointed in 2011 from the position of acting general manager and a national programme manager has recently been appointed in place of the North Island programme manager. As a result of the Christchurch earthquakes, the premises of the Christchurch branch were relocated to another site in Christchurch. Previous quality assurance history: At the NZQA quality audit in 2008, NZIIU met all but two requirements of the standard applicable at the time, relating to governance and management and learner information and support. Photocopies of the title pages of learners passports and copies showing their permit details were not held on file, although a complete list of passport numbers was available and the detailed information could be obtained from the overseas agents who arranged the travel. These issues have since been addressed. External assessment and moderation by NZQA or other standard-setting bodies is not required as NZIIU does not currently carry out assessments against any New Zealand Qualification Framework unit standards. Other: NZIIU is a signatory to the Code of Practice for the Pastoral Care of International Students (Code of Practice), approved for students aged who are not living with a parent, students aged 14-17, students aged 18 upwards, and young international group students. Annual Code of Practice review completed. 4

5 2. Scope of external evaluation and review The mandatory focus areas selected were governance, management, and strategy, and international student support. The other focus area selected was Introduction to English Communication. This programme was chosen because all students were enrolled in the programme at the time of the evaluation. 3. Conduct of external evaluation and review All external evaluation and reviews are conducted in accordance with NZQA s published policies and procedures. The methodology used is described fully in the web document Policy and Guidelines for the Conduct of External Evaluation and Review available at: The TEO has an opportunity to comment on the accuracy of this report, and any submissions received are fully considered by NZQA before finalising the report. The external evaluation and review team was made up of one lead evaluator and one external evaluator. The evaluation was conducted over two days and the evaluators met with the general manager, the national programme manager, a teacher coordinator, and the Japanese English teacher. Phone calls were made to regional homestay managers and contracted teaching staff during and following the evaluation. The evaluation team sighted extensive documentation, including the strategic plan, student handbooks and tutor manuals, student and homestay orientation packs, and programme booklets and evaluation forms from students, homestay families, and tutor coordinators. The evaluators visited a class of Japanese students on site at a secondary school where they were learning English and integrating with the school students. 5

6 Summary of Results Statement of confidence on educational performance NZQA is Confident in the educational performance of New Zealand Institute of International Understanding (NZIIU). NZIIU occupies a distinctive niche in the international education market. Whereas the primary purpose of many PTEs is to enhance the English language abilities of their students, NZIIU acts mainly as a liaison between overseas student groups and New Zealand schools, matching the needs of the two parties. In this role of education broker, NZIIU arranges short-term programmes for students, cultural exchanges with local students, educational field trips, and homestays. Its success in this role is evidenced by its annually renewed contracts, the achievement of the students brought into New Zealand, and the expressions of support by its stakeholders, domestic and international. Students sourced by NZIIU for its programmes tend to be those who wish to gain confidence as English language users in a supportive environment; some are trialling New Zealand as a travel destination for long-term study. Feedback indicates that as a result of participating in the NZIIU programmes, these students become more knowledgeable of, and responsive to, different cultural views, and gain an enhanced sense of the options available for future study and work. The organisation has a comprehensive support structure in place for its students, teaching staff, and homestay families. Policies and processes are well documented, vetting of homestays rigorous, and communication across the multiple stakeholders well managed. All student groups include a tour escort and English teacher who travel with the groups from Japan. On arrival in New Zealand, students are given comprehensive information packs and orientation sessions which seem to be effective in preparing the students for study in an unfamiliar environment. More importantly, by the end of their stay, the majority of students attest to increased confidence in English language usage and a new understanding of the kiwi lifestyle, including an introduction to Māori culture. Statement of confidence on capability in self-assessment NZQA is Confident in the capability in self-assessment of New Zealand Institute of International Understanding (NZIIU). NZIIU has some good self-assessment processes in place to gather and collate feedback from students, homestay families, and tutor coordinators which is used effectively to enhance the programme and the student experience. NZIIU also has effective processes to critically analyse feedback around student pastoral support and student learning resources. Management and teaching staff demonstrated a good understanding of the factors and resources that lead to student learning and stated that they regularly discuss 6

7 ideas and analyse ways for making the classes and resources more useful and enjoyable for the students. However, evidence to show that the NZIIU programmes result in longer-term improvements to students learning and knowledge are at this point limited. Although the evaluation team acknowledges that formative and summative student assessments are not required by the NZIIU programmes, it believes that gathering more evidence of this kind, and using it to drive ongoing organisational improvement, would enable NZIIU to gain a more comprehensive knowledge of how well its students achieve over time. 7

8 Findings How well do learners achieve? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Adequate. NZIIU students develop an appreciation of their own culture, an initial understanding of Māori culture, and direct exposure to the kiwi lifestyle. They leave the programme with a more internationalised view of life. Students interviewed during this evaluation said they have gained confidence in their English language as a result of their time in New Zealand. They also stated that they were looking forward to going back to 'show off' their new English skills to their classmates in Japan. The gain in confidence in the English language was also confirmed to the evaluation team during interviews with the students teacher, the Japanese school tour companion, and the visiting English teacher from Japan, who remain with the students during their stay in New Zealand. However, while NZIIU presented strong evidence for the immediate gains made by its students, the longer-term effectiveness of this achievement is currently unclear. The evaluation team believes that NZIIU should consider developing additional resources to measure the achievement of each of its students over time. This information would contribute directly to a more comprehensive review of NZIIU programmes. 1.2 What is the value of the outcomes for key stakeholders, including learners? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Adequate. Students gain confidence to help them practise basic English skills and learn aspects of New Zealand culture and diversity. The organisation was able to demonstrate, and show evidence for, the extent to which it is achieving these aims by presenting the completed and marked workbooks from students. The students were able to demonstrate their achievement to the evaluators by talking to them about their learning of New Zealand and Māori culture. 1 The findings in this report are derived using a standard process and are based on a targeted sample of the organisation s activities. 8

9 The primary value of the programmes lies in the direct immersion in English language study, exposure to the kiwi lifestyle, and introductory knowledge of Māori culture. Students attend classes to improve speaking, reading, and using English during their stay. They also learn aspects of Māori culture, visit different parts of New Zealand, and live in homestay accommodation. While all students receive a certificate from NZIIU, the organisation does not determine how well students improve their English language skills or understanding of New Zealand culture. The extent to which this is achieved is not currently evaluated by NZIIU as there are no summative assessments for the programme. Additional benefits are gained through the interaction between domestic and international students, which helps promote mutual understanding across the different cultures. Each student develops an appreciation of their own culture, and they often leave with a broader perspective of life. Observations and discussions with students by the evaluation team revealed that students enjoyed their programme and the activities completed to date. Students commented that they felt more confident speaking English and were keen to show the evaluators their English skills, and also performed a Māori action song they had learned. The evaluation forms that students use to evaluate the tutor coordinators and homestay placements are not well set out for the students to provide explicit feedback. A review of these forms to assess their suitability for use with secondlanguage students will assist NZIIU in gathering quality information for analysis to ascertain trends and improvements. This is required to ensure effective teaching and student support. 1.3 How well do programmes and activities match the needs of learners and other stakeholders? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Good. Each programme offered is tailor-made and reflects the customers (overseas schools ) requests, needs, and interests as much as practically possible. NZIIU continues to improve and change the programmes based on feedback to and from the customers. Summaries of feedback and recommendations from students, teachers, and host families to help improve the study programme are sent to each customer. Repeat business by some schools gives NZIIU confidence that the programmes are working well and fulfilling customer expectations. The selection of the particular New Zealand schools used by NZIIU is based on the general needs and special interests of the arriving group, as noted above. For example, if an arriving group has a particular interest in a specific activity or endeavour, a school is selected that can meet this need. The students from Japan complete their English studies in a classroom in the school of choice and are also 9

10 integrated into the school alongside domestic students at various times. Students are taken on trips to scenic sites around New Zealand and can experience horse riding and various sports activities to give them some experience of life in New Zealand. NZIIU communicates with overseas agents to ensure they hold meetings or orientations with parents and students several times before they depart their countries. This enables the students to begin to understand New Zealand culture and differences between their countries. In order to stress the importance of the Code of Practice to overseas agents who send short-term groups of students, NZIIU has sent the Code of Practice summary in the agents languages and has identified points to be emphasised. The programme review process involves evaluations of the programme, the teacher, activities, and homestays and is completed by students, host families, and group escorts; teaching staff also evaluate the programme. The information obtained from the evaluations is used to create programmes for future groups, although most programmes are tailor-made to suit the arriving group s needs and special interests. NZIIU holds a training session once a year for all teaching staff, which provides an opportunity for staff to share new information and give feedback on improvements and changes necessary to update lessons, including new textbooks and modifications to teaching and learning materials and activities. NZIIU notes that student feedback often expresses surprise at the diversity of cultures in New Zealand. As a result, the tutor coordinators have incorporated information about many cultures into their lessons. Host families complete an orientation/induction process prior to the arrival of the students. The orientation includes strategies and techniques for hosting students to ensure the experience is successful for both parties. Each host family signs a Code of Practice form and is issued a host family booklet with important information and contact details for the host families. If issues are identified with the host family, they are investigated and can lead to families no longer being involved in hosting students. The organisation s self-assessment process in this area is effective and strategic, because it allows NZIIU to identify how well its programmes are matching the needs of learners and other stakeholders. 1.4 How effective is the teaching? The rating for performance in relation to this key evaluation question is Adequate. The rating for capability in self-assessment for this key evaluation question is Adequate. Tutor coordinators at NZIIU are contracted on a casual basis when groups arrive in New Zealand. When they are initially contracted, they are fully inducted into the 10

11 organisation and are issued with a comprehensive tutor coordinator manual that details all aspects of their role, including the correct policies and procedures to use for various duties. Some tutor coordinators are qualified teachers and others are qualified in various other fields. All are invited to participate in teacher training run internally by NZIUU staff qualified in ESOL (English for Speakers of Other Languages), and all receive regular updates on the Code of Practice. Many of the tutor coordinators interviewed were eager to participate in further formal teacher training in order to achieve formal qualifications as no individual professional development plans are offered by management because of the tutor coordinators specific contractual arrangements. Tutor coordinators use a range of teaching activities in the classroom, including word games, practical hands-on learning, questioning, pair-work, class presentations, observations, and guest speakers. All classes are taught in English and students are expected to communicate in English at all times. Teaching staff are supplied with the necessary resources required to undertake their roles; however, tutors also develop and draw on their own personal resources to supplement what has been issued. The additional personal teacher resources viewed by the evaluators at the school site were not at the level of quality and consistency of NZIIU s own resources, and this was acknowledged by management. Annual appraisals of full-time staff are conducted regularly and verbal feedback occurs at the end of each programme. Evaluations of the teacher s role and a formal self-check are completed following the teaching of each group of students. Peer evaluations are undertaken regularly and feedback is provided. Each year during teacher coordinator training, staff have an opportunity to report back on the year and offer recommendations. Students enjoy the classes and find their teacher to be helpful in their learning of the English language, New Zealand culture, and Māori traditions, including Māori action songs, poi making, and flax weaving. Students interviewed at this evaluation said they felt confident in their achievements and ability to understand and speak English more easily than when they arrived in New Zealand, and were keen to point out that they will be better at English than their fellow classmates in Japan. Special requests from Japanese agencies for specific teaching staff are seen as a positive endorsement of the quality of the teaching. 1.5 How well are learners guided and supported? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Good. NZIIU provides an extensive and well-organised entry and orientation process for arriving students. The regional homestay manager oversees the entire process of entry and settling the students into New Zealand. This involves an orientation 11

12 session prior to the students departing Japan and immediately upon arrival in New Zealand. Students are met at the airport and go through a quick overview of the day and then travel to their selected school for an induction into their programme and the homestay process. Homestay families meet with NZIIU management before the students arrive in New Zealand, in order to work through all aspects of homestay rules and regulations. They are given information on the Code of Practice with specific reference to homestay providers, as well as information about the students. A Code of Practice for homestay accommodation providers is also signed at this time. There is a comprehensive support system in place for students attending NZIIU which involves teaching staff, regional homestay managers, the national programme manager, general manager, and other administrative staff, each having specific support roles. Students also have access to a group escort and English teacher (both from Japan), who are with them on a daily basis during their study and throughout their stay. If possible, NZIIU ensures that the learning and home environment are comparable to learners lifestyle back in Japan, to ensure students are comfortable during their stay. NZIIU also uses a buddy system where students are individually integrated into the school system during their visit. To ensure NZIIU s support processes work effectively, evaluations are completed by students, host families, teachers, and escorts and are viewed by the national programme manager and general manager, and any actions required are organised and any trends noted are analysed. Students are informed of grievance procedures on their arrival through their orientation materials, and first-language-speaking staff are available for students at all times. When issues are identified, they are dealt with immediately. In instances involving student safety, students are immediately removed from their host families. Long-term students have monthly meetings with their regional homestay managers and attend a camp in April. The regional homestay managers meet with long-term students three to four times per year. Regional homestay managers are the first point of call for long-term students and submit reports to head office each term. NZIIU checks that all passports, visas, medical insurance, and student history are arranged before students enter the country. Passport copies, numbers, and insurance details are collected and kept in student files. All samples sighted by the evaluators seemed to comply with regulatory requirements in full. A review of the Code of Practice is undertaken annually, with outcomes noted and actions undertaken; code updates and review training are undertaken regularly by all staff. 12

13 1.6 How effective are governance and management in supporting educational achievement? The rating for performance in relation to this key evaluation question is Adequate. The rating for capability in self-assessment for this key evaluation question is Adequate. The NZIIU governing board is made up of three members, including a chairperson (with a background in human resources) who lives locally and provides support and guidance to management as and when needed. Two other members of the board include the ex-general manager, now residing in Tauranga, and the organisation s founding member, now residing in Japan. Managers submit a report to the board quarterly and are invited to the twice-yearly board meetings to speak to their reports and discuss any issues. NZIIU currently markets its services offshore in Japan and Thailand. Enrolment has been significantly affected by a range of domestic and international factors such as immigration numbers, the Christchurch earthquakes, swine flu epidemic, and severe acute respiratory syndrome (SARS). NZIIU has managed to accommodate some of these issues by placing more student groups into the Auckland and North Island regions. NZIIU is a member of the New Zealand Council for International Exchange (NZCIE) and is currently ranked at 3/12 in terms of Quantity of enrolments per year. NZCIE meets twice annually and discusses issues and trends relating specifically to exchange students within New Zealand. NZIIU s relationship with NZCIE allows NZIIU to work collaboratively within the sector, building relationships, accessing resources, and identifying strategies for improvement to the programmes. NZIIU said membership of this organisation allows it to benchmark its services and provide a reasonable standard compared with those of a similar nature within the sector. However, no evidence of benchmarking was noted at this evaluation. With the departure of the North Island programme manager, a national programme manager has recently been appointed. Previous to this appointment, the national programme manager was based in Christchurch as the South Island programme manager, and had been with NZIIU for years. Staff interviewed noted that the national programme manager offered stability to the organisation while it was going through its internal restructuring. NZIIU has a small management team. The general manager and the national programme manager have moved into their new roles easily and are evaluating their roles and changing processes to suit their management style, with both regularly evaluating how decisions have affected NZIIU. Lines of communication and support between management and staff are clear and positive, with an evident open-door policy in the office and via and phone. Teaching staff are fully inducted into the organisation and are issued with a comprehensive tutor coordinator manual that details all aspects of their role, including the correct policies and procedures to use for various duties. Tutors 13

14 interviewed outlined the support and guidance of the regional homestay manager, who is always available to assist with their needs. Evaluations are undertaken following each student tour and annually as part of the strategic plan, and changes are made to resources, goals, objectives, and performance measures when this is found necessary. Teaching staff are able to recommend changes to the organisation and often see the results. Examples of this are the introduction of a student workbook and the use of photographs in the student application forms. NZIIU supports students well by effectively managing the planning and provision of resources for each new cohort of students, depending on their requirements and the length of their programme. Governance and management at NZIIU work together effectively. NZIIU managers collaborate to create a management report that outlines the goals and direction for the organisation. This is discussed and approved at a board meeting. The organisation has a strategic business plan with a clear purpose, vision statement, targets, and resource requirements. Performance of the activities of NZIIU is measured and reported to the board, although there is no formal assessment of the performance of student learning. Evaluations of the language skills of students before and after the programme would enable NZIIU to gain more comprehensive knowledge of the longer-term achievement of the students and the effectiveness of the programmes of learning. 14

15 Focus Areas This section reports significant findings in each focus area, not already covered in Part Focus area: Governance, management and strategy The rating in this focus area for educational performance is Adequate. The rating for capability in self-assessment for this focus area is Adequate. 2.2 Focus area: International student support The rating in this focus area for educational performance is Good. The rating for capability in self-assessment for this focus area is Good. 2.3 Focus area: Introduction to English Communication The rating in this focus area for educational performance is Good. The rating for capability in self-assessment for this focus area is Adequate. 15

16 Recommendations There are no recommendations arising from the external evaluation and review. 16

17 Appendix Regulatory basis for external evaluation and review Self-assessment and external evaluation and review are requirements of programme approval and accreditation (under sections 249 and 250 of the Education Act 1989) for all TEOs that are entitled to apply. The requirements are set through the Criteria for Approval and Accreditation of Programmes established by NZQA under section 253(1)(d) and (e) of the Act and published in the Gazette of 28 July 2011 at page These policies and criteria are deemed, by section 44 of the Education Amendment Act 2011, to be rules made under the new section 253. In addition, for registered private training establishments, the criteria and policies for their registration require self-assessment and external evaluation and review at an organisational level in addition to the individual programmes they own or provide. These criteria and policies are also deemed, by section 44 of the Education Amendment Act 2011, to be rules made under section 253. Section 233B(1) of the Act requires registered PTEs to comply with these rules. NZQA is responsible for ensuring non-university TEOs continue to comply with the rules after the initial granting of approval and accreditation of programmes and/or registration. The New Zealand Vice-Chancellors Committee (NZVCC) has statutory responsibility for compliance by universities. This report reflects the findings and conclusions of the external evaluation and review process, conducted according to the EER process approved by the NZQA Board. The report identifies strengths and areas for improvement in terms of the organisation s educational performance and capability in self-assessment. External evaluation and review reports are one contributing piece of information in determining future funding decisions where the organisation is a funded TEO subject to an investment plan agreed with the Tertiary Education Commission. External evaluation and review reports are public information and are available from the NZQA website ( Information relevant to the external evaluation and review process, including the publication Policy and Guidelines for the Conduct of External Evaluation and Review, is available at: 17

18 NZQA Ph E qaadmin@nzqa.govt.nz 18