Report of External Evaluation and Review

Transcription

1 Report of External Evaluation and Review National Technology Institute Limited Highly Confident in educational performance Highly Confident in capability in self-assessment Date of report: 4 April 2013

2 Contents Purpose of this Report... 3 Introduction TEO in context Scope of external evaluation and review Conduct of external evaluation and review... 5 Summary of Results... 6 Findings... 8 Recommendations Appendix MoE Number: 7832 NZQA Reference: C09183 Date of EER visit: 3-5 December

3 Purpose of this Report The purpose of this external evaluation and review report is to provide a public statement about the Tertiary Education Organisation s (TEO) educational performance and capability in self-assessment. It forms part of the accountability process required by Government to inform investors, the public, students, prospective students, communities, employers, and other interested parties. It is also intended to be used by the TEO itself for quality improvement purposes. Introduction 1. TEO in context Name of TEO: Type: Location: Delivery sites: National Technology Institute Limited (NTIL) Private training establishment (PTE) 20 Hobson Street, Auckland Central, Auckland 146 Durham Street, Tauranga First registered: 30 March 2007 Courses currently delivered NTIL currently delivers the following courses: National Diploma in Hospitality (Management) (Level 5) National Diploma in Hospitality (Operational Management) (Level 5) National Technology Institute Certificate in Hospitality (Operations Supervision) (Level 4) National Diploma in Computing (Level 5) Diploma in Computing (Level 6) Diploma in Computing (with strands in Software Development and Computer Networking) (Level 7) Code of Practice signatory? Number of students: Yes International: 227 equivalent full-time students Number of staff: NTIL has 29 full-time staff, 23 part-time staff (11 full-time equivalent staff) and 33 support staff from 3

4 the Ntec group. Scope of active accreditation: Computing and Information Technology/Computing Subfield Service Sector/Hospitality Domain Distinctive characteristics: NTIL is part of National Tertiary Education Consortium Limited (Ntec). Ntec provides each of the four schools in the consortium with shared services. 40 of the 227 students are based in Tauranga, mostly on the information technology (IT) courses 74 per cent of the students are male 68 per cent are under 25 Class sizes are limited to a maximum of 25 Each student receives a laptop/notebook to use during their study Recent significant changes: Previous quality assurance history: A chief executive officer was appointed in August 2011 who reports directly to the board of directors. At the previous quality assurance visit by NZQA, an audit (2008), NTIL met all but two requirements of the then standard. The two requirements not met related to governance and management, and learner information, entry and support. The one requirement not met at the earlier audit had been resolved. Of the five unit standards submitted for moderation in 2011, two required modification and one was not approved. Assessor decisions were verified for four of the five unit standards, but not for unit standard 7910 Demonstrate knowledge of computer data types and data structures (level 5) (7 credits). 4

5 2. Scope of external evaluation and review In accordance with NZQA policy, the mandatory focus areas of governance, management, and strategy, and international student support were included in the scope of this external evaluation and review (EER). The National Diploma in Hospitality (Operational Management) (Level 5) programme was selected as a focus area as it is one of the main courses in the hospitality area. The Diploma in Computing (Level 6) programme was selected as a focus area as it typifies the courses delivered in the provider s other main subject area of computing. 3. Conduct of external evaluation and review All external evaluation and reviews are conducted in accordance with NZQA s published policies and procedures. The methodology used is described fully in the web document Policy and Guidelines for the Conduct of External Evaluation and Review available at: The TEO has an opportunity to comment on the accuracy of this report, and any submissions received are fully considered by NZQA before finalising the report. The EER was conducted over three days by two NZQA evaluators at the institute s premises in Auckland. The evaluation involved interviews with: The chief executive officer The management team The managers of the two programmes selected as focus areas All the tutors in the focus areas Students from each of the two programmes selected as focus areas Tutors and students from the Tauranga campus via Skype The student support team Nine ex-students Two ex-students via phone Seven stakeholders The evaluation also involved a review of relevant documentation, such as the student handbook, achievement data, student evaluation forms, the staff professional development plan, moderation reports, graduate destination data, collaboration agreements, meeting minutes, self-assessment reports, and associated correspondence. 5

6 Summary of Results Statement of confidence on educational performance NZQA is Highly Confident in the educational performance of National Technology Institute Limited. NTIL has a proud record of educational achievement which has been sustained over a number of years. Students on the National Diploma in Hospitality (Operational Management) programme have achieved a rate of qualification completion of 93 per cent so far in 2012, and the rate is expected to rise as more students graduate. Students in the Diploma in Computing programme have achieved a rate of 100 per cent qualification completion so far this year. By any measure, these rates are high, and this impression is endorsed by the Tertiary Education Commission (TEC) figure of an 80 per cent qualification completion rate for all domestic students at levels 5 and 6 in The rates of graduate employment and involvement in higher education are similarly high, indicating other valuable outcomes. NTIL has a holistic approach to education. It realises that its role is far more than providing a way of gaining qualifications. It sets itself lofty targets and achieves them. A strong set of core values ensures that the students are the focus of attention, and NTIL takes extensive measures to ensure their needs are met. These measures include providing additional courses in professional and personal skill development, and conversational classes, as well as a laptop for each student. Such measures are supplemented by the use of outside speakers from industry and from welfare organisations. The links with the community are also strong as NTIL recognises that one of its primary goals is to ensure the students successfully integrate into the community. The tutors are experienced and skilled. They have a very good relationship with the students. The students appreciate the tutors passion and the additional help they provide. Further help is provided by the pastoral care team whose assistance was recognised by the positive results of an i-graduate survey undertaken by the Ministry of Education and Education New Zealand. NTIL receives very positive feedback from current students, ex-students, and external stakeholders. The feedback affirms the high standard of educational performance. NTIL is always seeking new ways of improving its operation. It is currently developing additional forms of collaboration with other educational institutions, such as a university in England. It is also encouraging the growth of student-run businesses through an innovations centre it is establishing. Such developments exemplify the proactive and student-centred approach followed by NTIL. 6

7 Statement of confidence on capability in self-assessment NZQA is Highly Confident in the capability in self-assessment of National Technology Institute Limited. Self-assessment is embedded at NTIL. It is not satisfied with students achieving rates of qualification completion that are clearly high, but benchmarks them by comparing them with rates achieved by other PTEs and performance figures from other organisation such as the Hospitality Standards Institute. It also compares the current achievement rates with the achievement rates of previous years. In this way, NTIL obtains a well-balanced view of its achievement. NTIL keeps detailed records of graduate destinations which it analyses to check the links between the nature of the jobs and the courses studied. Self-assessment, for example, revealed that 81 per cent of the employed graduates of the level 6 Diploma in Computing from were employed in IT-related positions. Additional data used by NTIL is generated by a range of evaluation forms. The student feedback form is effectively used at the end of each term; the scores are collated for each tutor and used to assess each tutor s performance. The end-ofcourse evaluation forms and the facilities and service evaluation forms also provide valuable data which is analysed and used as a basis for reflection on different aspects of NTIL. The data is supplemented by the feedback gained from various groups such as the advisory groups. NTIL quickly takes action when there is negative feedback, such as that received about the accounting unit standard. The response was to devise an introductory accounting course before the unit standard was undertaken. Another example was the quick response to a concern expressed by a company employing hospitality staff on a casual basis about the use of languages other than English. NTIL effectively utilises the annual review of the Code of Practice for the Pastoral Care of International Students (Code of Practice) to assess its performance. When the review undertaken in August 2012 found a need for the appointment of a pastoral care staff member to deal with students aged under 18, an appointment was made within a month. Similarly effective and prompt action was taken to run a Code of Practice workshop for staff and to formulate a template to use in communicating with the parents of students aged under 18. NTIL takes its role as an educational institution very seriously and uses every practical way of ensuring it is providing a valuable service. 7

8 Findings How well do learners achieve? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. NTIL students have achieved excellent academic results over a number of years. Students on the National Diploma in Hospitality (Operational Management) programme, for example, achieved a 100 per cent completion rate in 2010, followed by 95 per cent in 2011 and 93 per cent so far in 2012, but the rate is expected to rise as more students graduate. Over the same period, students in the Diploma in Computing programme achieved rates of 89.5 per cent, 95 per cent, and so far in 2012, 100 per cent. By any measure, these rates are high, and this impression is endorsed by the TEC figure of 80 per cent qualification completion rate for domestic students studying at levels 5 and 6 in Further confirmation of the high levels of achievement of the hospitality students was gained when NTIL s students won medals in a regional culinary competition organised by the Restaurant Association of New Zealand. The evaluation team met a stakeholder working for a hospitality recruitment company who described how well prepared NTIL s students were to start work. The directors and managers emphasise the importance of ensuring the students have the skills required to secure employment. When a gap is identified, NTIL quickly devises a way of remedying it. This often takes the form of additional courses for the students, provided at no charge. Good examples of these are the pronunciation classes and the Toastmasters classes, both of which develop soft skills such as communication and presentation. NTIL has an excellent overview of the sectors in which it is offering courses and uses this to assess the educational performance of its students. Comparisons are made with other PTEs offering hospitality and computing courses and with the overall performance of the PTE sector according to TEC statistics. Further comparisons are made with previous years, and trends are identified. NTIL has a good understanding of the reasons why a few students have to withdraw from their programme. When NTIL has some concerns about its students, it takes corrective action. An example of this was the prompt response to feedback given by a stakeholder about some hospitality students using their own language while being employed in a casual capacity. 1 The findings in this report are derived using a standard process and are based on a targeted sample of the organisation s activities. 8

9 1.2 What is the value of the outcomes for key stakeholders, including learners? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. The directors and managers emphasised from the outset of the EER that NTIL aims to provide a holistic experience. The college succeeds in this. The students gain far more than a qualification they are successfully integrated into the community in all respects. This goes much further than the basics of employment, even though high rates of success are achieved there also. For example, 100 per cent of the hospitality students in 2011, and so far in 2012, have gained full-time employment in hotels, restaurants, and cafes. Their jobs include restaurant supervisor, duty manager, and the manager of a club. Even though it is recognised that it takes longer for IT students to secure full-time positions, positive outcomes have been achieved. A study of the computing graduates from 2009 and 2010 revealed that 75 per cent had moved into employment and 25 per cent had moved to further study. Of the employed graduates, 81 per cent were in IT-related employment such as IT support staff, system administrators, programmers, and network administrators. NTIL recognises the importance of connecting the academic sector with industry and provides financial support to students gaining Microsoft certifications. NTIL is also developing a relationship with an IT company to provide internships and assistance with projects. This will be extended next year by the formation of an innovations centre to undertake projects highlighting the talents of NTIL s IT and hospitality students, with a view to developing them into businesses. This type of approach is assisted by the frequent use of outside speakers in both the programmes. NTIL demonstrates its commitment to developing the whole person through its personal and professional skills courses. They cover topics such as workplace culture, team building, decision-making, communication, and research skills. Students are encouraged to practise their skills by working with charitable organisations such as the Kidney Foundation. NTIL s students also benefit from the work experience gained through the Auckland Regional Migrant Services Charitable Trust and the wide range of industry links enjoyed by Ntec. There is also an extensive range of academic links such as the collaboration agreement with a local polytechnic. NTIL has ongoing student destination data going back to 2009 for both hospitality and computing. It continuously updates this data and analyses it to check that students are achieving the outcomes expected of them. The rates of employment are benchmarked against the rates of employment achieved at other PTEs. This self-assessment is effective and endorses the value of NTIL s courses. 9

10 1.3 How well do programmes and activities match the needs of learners and other stakeholders? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. The different types of courses offered by NTIL, such as the professional skills and the Toastmasters courses, along with the academic programmes, allow NTIL to ensure that its learners needs are fully covered. These courses are supplemented by pronunciation classes and a variety of supplementary short courses such as the barista course and the Licence Controller Qualification course. NTIL appreciates that valuable learning takes place outside the classroom and facilitates this by organising social outings and talks by groups such as the Family Planning Association. The police also regularly make presentations to the students on topics associated with student safety. A new move is the provision of workshops, particularly for IT students, about how to build relationships with people that develop into business networks. The breadth and variety of these initiatives owe much to the experience of the directors and the way in which they use the advisory groups. These groups discuss NTIL s courses and are always looking for ways in which NTIL can make its students more employable. In this way, NTIL ensures that its programmes and activities are as relevant as they can be and that they are matching stakeholder needs. NTIL closely monitors industry needs. The introduction of the level 7 Diploma in Computing is an example of the result of such monitoring, as it has been developed to prepare students for two key parts of the IT industry, networking and software development. Similar work has been undertaken with the Hospitality Standards Institute. NTIL takes this a step further by ensuring the needs of students for courses of further education are met through a variety of pathways such as the pathway to the Bachelor of Tourism and Hospitality at Le Cordon Bleu in Australia and the Bachelor of Information Technology at the Southern Institute of Technology. It is typical of NTIL s approach that it is not resting on its laurels and has developed another educational pathway with Northampton University in England, which will start in NTIL has a number of groups that help it to assess the effectiveness of its coverage of stakeholder needs through the provision of feedback. NTIL uses the feedback to make improvements. The student council, for example, requested that more outside speakers be provided and, in response, sessions now take place on an almost weekly basis. Feedback also prompted NTIL to use e-learning to help students track their progress and to provide Licence Controller Qualification courses for all hospitality programmes. 10

11 1.4 How effective is the teaching? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. The teaching at NTIL is highly effective. The tutors are experienced and have relevant qualifications in their specialist industries. All of them either have, or are completing, qualifications in adult education. The good mix of full and part-time tutors ensures that many of them are still involved in their areas of specialisation and are bringing up-to-date practice to their delivery. The very positive written feedback from the students confirms the high standard of the teaching. The high course completion rates are supported by effective systems of moderation. Twenty per cent of the hospitality assessments, for example, are internally moderated, and a range of external contractors are used for external moderation. The most recent moderation by the Hospitality Standards Institute took place in November 2012, and the report confirmed that moderation requirements were being met. The external moderation undertaken by NZQA in November 2011 also confirmed that national external requirements were being met for the majority of the four unit standards moderated. The one exception was where NZQA stated that NTIL was expecting too much from its learners. Students and ex-students spoken to by the evaluation team described the strong relationships they have with the tutors. They praised their passion for their subjects and the way in which they respond quickly to requests for assistance. The tutors make effective use of the students prior experience by encouraging them to share in class. Throughout the classes, the emphasis is on hands-on teaching. This approach is aided by NTIL s decision to supply laptops to all its students for them to use in their lessons. NTIL appreciates the importance of effective teaching and has systems that check that it is taking place. The student feedback form is effectively used at the end of each term and the scores are collated for each tutor. NTIL quickly takes action when there is negative feedback, such as that received about the accounting unit standard. The response was to devise an introductory accounting course before the unit standard was undertaken. When it was found that another tutor was writing too much on the board, steps were taken to ensure that PowerPoint presentations were used as a supplement. The evaluation team saw an example of a comprehensive class observation report and details of subsequent s and meetings held to develop support mechanisms for a tutor. Even though the visit to the class only took place two weeks before the EER, improvements to the tutor s teaching had already taken place. 11

12 1.5 How well are learners guided and supported? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. NTIL puts the students at the centre of its operation. The interests of the students are paramount, and the additional courses, such as Toastmasters and professional skills, and the facilities provided for the students, are testament to this commitment. However, the support mechanisms go much further than courses and facilities. The pastoral care team oversees a network of support that is both extensive and individualised. The network is evident as soon as the students enter the country. They are provided with free airport pick-up, a SIM card for a phone call home, and free accommodation in a hostel for two weeks while they are finding their feet. The orientation takes two days and is very thorough, including such aspects as CV writing and Māori culture as well as the items covered in the comprehensive student handbook. The i-graduate survey undertaken by the Ministry of Education and Education New Zealand reported student satisfaction scores for the four aspects of arrival, learning, living, and support. NTIL recorded scores in all four sections above those of an accumulated group of global private providers, and in three of the four sections above those of the PTEs who participated in the survey. The students benefit from the shared services provided by the Ntec consortium. This arrangement allows NTIL to concentrate on academic services. Buddy systems are established when necessary for the students, and the students appreciate the one-to-one help they receive from the tutors when they have a problem understanding any material. The student management system warns the students in advance of the expiry dates of their visas and insurance policies and they are helped to renew them. The evaluation team also saw instances where an individualised approach was taken, such as taking a student to a Citizens Advice Bureau and helping a student with mental health issues to fly home. NTIL undertakes a very thorough review of its international student support each year when it completes its self-review of the implementation of the Code of Practice. An action plan is drawn up and the evaluation team saw evidence of improvements being made after the review in August 2012, such as the appointment of a pastoral care staff member to look after students aged under 18, and the formulation of a template to be used in communicating with the parents of those students. Further effective self-assessment of student support forms part of the end-of-course evaluation forms and the facilities and service evaluation form. 12

13 1.6 How effective are governance and management in supporting educational achievement? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. A strong set of values, including respect, integrity, ethics, and inclusiveness are at the core of NTIL. The directors stressed these values at the outset of the EER, and it soon became evident to the evaluation team that everything NTIL does is imbued with them. Profits are seen as necessary for the sustainability of the organisation, but are not a driving force. The board of directors supports educational achievement by allocating funds to provide a laptop for each student and by providing generous professional development for the staff, such as the availability of up to $1,000 for IT staff to gain additional IT certifications. Ntec also has a staff development manager who closely monitors staff effectiveness, training, and development. A large amount of money is allocated to academic scholarships for distribution to the top students in each programme. Further scholarships are awarded in the form of reduced fees to students who spend a second year at NTIL. Other scholarships are awarded to students who have the ability to be tertiary students but not the means. Three full scholarships, for example, have been awarded to three Vietnamese boys by a charity which helps street kids, child victims of trafficking and the rural poor in Vietnam. The governance and management teams further support the students by organising meetings, which are used by managers and heads of faculty to improve current ways of supporting educational achievement and to identify new ways of learning. Two examples of new directions being taken are the innovations centre mentioned in section 1.2, and Enactus (a group of students, academics, and business leaders), both of which will also stimulate the growth of student-run businesses. The meetings where these ideas are discussed include the weekly management meetings, programme committee meetings, and the continuous improvement committee meetings. The strategic plan is reviewed thoroughly each year. The students are kept aware of developments by efficient methods of communication. Feedback is gained through a variety of means, as mentioned, and the governance and management teams are responsive to the feedback. The provision of laptops arose from feedback by the students concerned about the lack of access in the evenings to NTIL s computers. The new student lounge and café is another example of an effective response to feedback. In this case, the students requested a larger space where they could meet. The continuous improvement committee is an effective way of overseeing the annual review of qualifications and programmes of study to ensure that they remain relevant and in line with contemporary practice. 13

14 Focus Areas This section reports significant findings in each focus area, not already covered in Part Focus area: Governance, management, and strategy The rating in this focus area for educational performance is Excellent. The rating for capability in self-assessment for this focus area is Excellent. 2.2 Focus area: International student support The rating in this focus area for educational performance is Excellent. The rating for capability in self-assessment for this focus area is Excellent. 2.3 Focus area: National Diploma in Hospitality (Operational Management) (Level 5) The rating in this focus area for educational performance is Excellent. The rating for capability in self-assessment for this focus area is Excellent. 2.4 Focus area: Diploma in Computing (Level 6) The rating in this focus area for educational performance is Excellent. The rating for capability in self-assessment for this focus area is Excellent. 14

15 Recommendations There are no recommendations arising from this external evaluation and review. 15

16 Appendix Regulatory basis for external evaluation and review Self-assessment and external evaluation and review are requirements of programme approval and accreditation (under sections 249 and 250 of the Education Act 1989) for all TEOs that are entitled to apply. The requirements are set through the Criteria for Approval and Accreditation of Programmes established by NZQA under section 253(1)(d) and (e) of the Act and published in the Gazette of 28 July 2011 at page These policies and criteria are deemed, by section 44 of the Education Amendment Act 2011, to be rules made under the new section 253. In addition, for registered private training establishments, the criteria and policies for their registration require self-assessment and external evaluation and review at an organisational level in addition to the individual programmes they own or provide. These criteria and policies are also deemed, by section 44 of the Education Amendment Act 2011, to be rules made under section 253. Section 233B(1) of the Act requires registered PTEs to comply with these rules. NZQA is responsible for ensuring non-university TEOs continue to comply with the rules after the initial granting of approval and accreditation of programmes and/or registration. The New Zealand Vice-Chancellors Committee (NZVCC) has statutory responsibility for compliance by universities. This report reflects the findings and conclusions of the external evaluation and review process, conducted according to the EER process approved by the NZQA Board. The report identifies strengths and areas for improvement in terms of the organisation s educational performance and capability in self-assessment. External evaluation and review reports are one contributing piece of information in determining future funding decisions where the organisation is a funded TEO subject to an investment plan agreed with the Tertiary Education Commission. External evaluation and review reports are public information and are available from the NZQA website ( Information relevant to the external evaluation and review process, including the publication Policy and Guidelines for the Conduct of External Evaluation and Review, is available at: NZQA Ph E qaadmin@nzqa.govt.nz 16