Report of External Evaluation and Review

Transcription

1 Report of External Evaluation and Review Auckland Hotel and Chefs Training School Confident in educational performance Confident in capability in self-assessment Date of report: 11 February 2014

2 Contents Purpose of this Report... 3 Introduction TEO in context Scope of external evaluation and review Conduct of external evaluation and review... 5 Summary of Results... 6 Findings... 8 Recommendations Appendix MoE Number: 8858 NZQA Reference: C12496 Date of EER visit: 16 and 17 October

3 Purpose of this Report The purpose of this external evaluation and review report is to provide a public statement about the Tertiary Education Organisation s (TEO) educational performance and capability in self-assessment. It forms part of the accountability process required by Government to inform investors, the public, students, prospective students, communities, employers, and other interested parties. It is also intended to be used by the TEO itself for quality improvement purposes. Introduction 1. TEO in context Name of TEO: Type: Location: Delivery sites: Auckland Hotel and Chefs Training School (AHCTS) Private training establishment (PTE) 8 Mountain Rd, Newmarket, Auckland One site, at the above address First registered: 1 December 1992 Courses currently delivered: National Certificate in Hospitality (Introductory Cookery) (Level 2) National Certificate in Hospitality (Basic Cookery) (Level 3) National Certificate in Hospitality (Cookery) (Level 4) National Diploma in Hospitality (Business Management) (Level 5) Code of Practice signatory? Yes; approved for students aged 18 years and upwards Number of students: Domestic: 134 International: 12 (Māori 27 per cent; Pasifika 19 per cent; under 25 years 75 per cent; male 60 per cent) Number of staff: 12 full-time One part-time 3

4 Scope of active accreditation: Distinctive characteristics: Recent significant changes: An assortment of domains and unit standards up to and including level 5 in the fields of Business, Computing and Information Technology, Core Generic, Humanities, Manufacturing and Service Sector. Accreditation also includes 11 local and national certificate and diploma programmes up to and including level 5. These are all hospitalityrelated programmes. AHCTS is a specialist school that focuses on the training of professional chefs. It also uses cookery training as the basis of Tertiary Education Commission (TEC)-funded programmes (Student Achievement Component and Youth Guarantee) in which at-risk youth and second-chance students are given the opportunity to gain skills and the confidence to gain employment or engage in further study. AHCTS has recently gained approval to offer the National Certificate in Hospitality (Introductory Cookery) (Level 2) and the National Diploma in Hospitality (Business Management) (Level 5). There are plans to share the premises in Newmarket with the National College of Security Personnel and Technology, which has the same owner-director as AHCTS. Previous quality assurance history: There was an external evaluation and review (EER) of AHCTS in December 2009 with the following outcomes: NZQA is Confident in the educational performance of Auckland Hotel and Chefs Training School NZQA is Confident in the capability in selfassessment of Auckland Hotel and Chefs Training School NZQA s July 2013 external moderation report indicates that some of AHCTS s assessment procedures were non-compliant. On the other hand, Service IQ, the industry training organisation which administers the majority of AHCTS s standards, confirmed that the provider met all of its moderation requirements. 4

5 2. Scope of external evaluation and review The agreed scope of the EER of AHCTS included the following mandatory focus areas: Governance, management and strategy International student support Other focus areas were: National Certificate in Hospitality (Introduction Cookery) (Level 2) National Certificate in Hospitality (Cookery) (Level 4) These programmes have been selected as examples of an entry-level introductory programme for the cookery trade, and an upper-level programme with aligned professional skills for the cookery trade. 3. Conduct of external evaluation and review All external evaluation and reviews are conducted in accordance with NZQA s published policies and procedures. The methodology used is described fully in the web document Policy and Guidelines for the Conduct of External Evaluation and Review available at: The TEO has an opportunity to comment on the accuracy of this report, and any submissions received are fully considered by NZQA before finalising the report. Two NZQA evaluators conducted the evaluation visit over two days at the PTE s only site in Newmarket, Auckland. The evaluators met with the owner-director, the general manager, programme management, teaching staff and students. A range of relevant documents was sighted by the evaluators. A representative of the industry training organisation, ServiceIQ, as well as graduates and employers, were interviewed by telephone. 5

6 Summary of Results Statement of confidence on educational performance NZQA is Confident in the educational performance of Auckland Hotel and Chefs Training School. AHCTS steadily performs at a good level of educational performance. It achieves this by: Maintaining a specialised cookery school which allows it to concentrate on the skills and resources required for training and meeting industry expectations in one focus area. Focusing on culinary skills to allow the successful student to staircase to higher-level programmes within the same institution; significant numbers of students are taking advantage of this opportunity. Consistently meeting most parts of the TEC funding targets in recent years. Significant numbers of students gaining employment within the hospitality industry; the high regard that is placed on the quality of the training from this PTE. Providing a high standard of tuition, a very good learning environment and high-quality facilities. Employing teachers who are experienced in their trade and passionate about their industry; those interviewed were qualified teachers. In this way, AHCTS shows that it is focused on the quality of delivery. Providing a high level of support to students, with appropriate additional attention being given to international students. Maintaining tuition based on the skills being current, relevant to industry and of a high standard. Having good management control over the direction of the school and making some determined efforts to improve performance. Working on lifting its general performance in course and qualification completions, and the progression and retention of students. 6

7 Statement of confidence on capability in self-assessment NZQA is Confident in the capability in self-assessment of Auckland Hotel and Chefs Training School. AHCTS has developed a good basis for effective self-assessment. It achieves this by: Ensuring that management is in the process of developing and fine-tuning to further improve the value of the developments based on self-assessment. Confirming that the self-assessment strategy is still a work in progress, with not all areas being evaluated as well as they could be. Areas to be improved include: promoting learning that is more studentcentred, developing more self-reflective practice in teaching staff, and improving systems of formal data collection from external stakeholders. Engaging in some useful self-assessment initiatives which are helping the organisation to understand how best to meet the requirements of its stakeholders. Establishing an advisory committee to make recommendations to the senior management group. Jointly initiating a local consortium of education providers to influence and lobby for the hospitality training sector, and to share resources. Recently enhancing self-assessment activities by engaging in risk analysis. Regularly reviewing its quality management system and moving towards formalising self-assessment strategies through the documentation and implementation of policies and procedures. 7

8 Findings How well do learners achieve? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Good. AHCTS achieves generally good outcomes from its cookery programmes. The organisation provides programmes from level 2 to level 5, from foundation to professional level in the field of cookery. The TEC performance data for AHCTS achievement against its funding requirements shows mostly good results over the last four years, as shown in Table 1 below, with performance mostly meeting objectives. Progression to further study is shown as being low in the funded programmes, and this is an area that deserves more analysis and attention as it indicates that the students are leaving with elementary industry skill-sets. Table 1: Comparison of TEC completion data, * AHCTS outcomes 2011 AHCTS outcomes 2012 Course completions Qualification completions Progression 79% 77% 10% 75% 76% 74% 11% 76% * Figures from TEC performance reports Retention The PTE s educational performance indicators (EPIs) for TEC Student Achievement Component-funded programmes in 2012 met most of the PTE s funding objectives even though they were below the national subsector medians. Table 2 shows the PTE s figures for all programme outcomes for 2012, which are favourable. 1 The findings in this report are derived using a standard process and are based on a targeted sample of the organisation s activities. 8

9 Table 2: 2012 overall AHCTS programme outcomes* Over-all course completions Progression to further education Progression to employment Progression from level 3 to level 4 Completion rates at level 4 83% 60% 23% 50% approx. 90% approx. * Figures provided by AHCTS AHCTS is working on lifting its general performance in course and qualification completions, and progression and retention, and its overall figures for all programmes in 2012 are encouraging. The numbers of graduates progressing to further education and to employment is also encouraging. Management is able to show that projected outcomes for 2013 are above the sector median. One of the strengths of having progressive programmes in the same discipline is that students are able to advance to the next level within the same school, if they qualify. The evaluators learned that a significant proportion of successful students move upward from the level 2 and level 3 programmes within AHCTS. There was evidence that a significant number of students (as shown in Table 2 above) from the level 3 programme elect to enter the level 4 programme. Management also stated that course and qualification completion rates are higher in the level 4 programme (as also shown in Table 2 above), because the students are more motivated and focused at that level. At level 4, the students have the opportunity to sit the London City & Guilds examinations and gain this internationally recognised qualification, and this adds value to the programme. The learning environment at AHCTS is spacious and well equipped and engages highly experienced teachers. There was evidence that this factor has attracted a few students from other institutions and abroad to complete their cookery studies at AHCTS. In response to assessment of student needs, the level 2 programmes at AHCTS are strengthened with the inclusion of generic unit standards that allow the students to develop general life skills and employment skills as well as cookery knowledge. The evaluators saw valid evidence that the additional employment skills are valued by the students because disciplines such as time management, punctuality, hygiene, writing skills and communication skills are necessary when the students look for employment. Teachers spoke of noticing the change in confidence and self-assurance levels in the students as a result of the students participation in the programme. Self-assessment in the area of student achievement is comprehensive and is based on AHCTS programme outcomes provided to the funding agencies. In response to the shortcomings in its performance against TEC requirements (as shown in Table 1 above), AHCTS is looking at the reasons for student withdrawals. It is engaging in more comprehensive monitoring of work experience, and this data has given better information about student performance. The data is collated and discussed at the weekly staff meetings. At the conclusion of each programme, the completion data is analysed by the tutors and progress is monitored. 9

10 1.2 What is the value of the outcomes for key stakeholders, including learners? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Good. The value of the learning provided by AHCTS is good. This value is shown in different ways at the different levels of the programmes. At level 2, students of the National Certificate in Hospitality (Introductory Cookery) are engaged in a foundation course that prepares them as much for employment or advancement to further study as it furnishes them with the basic skills of cookery. The significance of this course lies in its social importance as a fully funded Youth Guarantee programme. Many of the students are at-risk youth and students who have not achieved at school, and the programme also attracts second-chance learners such as parents re-entering the workforce. AHCTS recognises that not all the students want to be professional chefs, and so the cookery component becomes a vehicle for some students to gain confidence, self-assurance and work skills. In the level 4 programme, the predominance of higher-level practical skills is appreciated by the students, and suits the students who are not academically inclined. It is important for an organisation s ability to be able to conduct workplace assessments that the training facilities are of an appropriate standard, so that ServiceIQ rates them as commercial kitchens, and this is the case at AHCTS. Work experience placement is built into the lower-level programmes so that the students gain an appreciation of the speed, punctuality, attitude and discipline required in the industry. Visits to SkyCity and Auckland Fish Markets are also organised to further the students exposure to the industry. The level 2 programme has some generic material attached to it which helps to foster responsible behaviour, hygiene and professional standards, and helps the students with planning career pathways. A few students have realised through attending the programmes that they want to go back to secondary school, and this is regarded as a positive outcome because the student has taken a constructive option. The National Certificate in Hospitality (Cookery) (Level 4) is an advanced skills course for people who are already in the workforce. All of the students at this level work in the industry. These students further enhance their skills by engaging in culinary competition work, supervised by their tutors. The importance of student achievement is recognised internally by celebrating the students success at graduation, and in the gallery of photographs of successful graduates displayed in the corridors, which increases the motivation of the current students. The value of the AHCTS programmes is understood through the collection of feedback from the students. A short questionnaire is handed to the students monthly, and a more substantial evaluation exercise is carried out at the conclusion of every programme. This feedback is collated and presented at the management 10

11 meetings. The monthly feedback helps the teachers identify any delivery difficulties. The AHCTS Facebook page has about 300 friends and helps to stimulate discussion and feedback about the school in an informal way. The PTE has close links with the local industry through its tutors and graduates. These links are largely informal, and it would be beneficial to formalise more of these relationships so that a structured system of industry feedback and input can be established. The evaluators saw evidence from graduates, tutors and employers that endorses the quality of the AHCTS programmes and shows that graduate skills are of a high standard. Some employers indicated that they sought out graduates from AHCTS to engage as employees because of the quality of training they had received. It is noted that AHCTS has recently hired a quality assurance manager to ensure that data will be appropriately collated and presented for discussion, and that the systems will be refined to make better use of the data that is collected. While the effects of this have yet to become apparent, it is a positive step. Another positive development is the implementation of a new process for tracking graduates in employment, which is expected to provide further information on the value of the learning after the students leave AHCTS. 1.3 How well do programmes and activities match the needs of learners and other stakeholders? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Good. The evaluators saw many very positive features in the way AHCTS organises its programmes and teaching environment to provide a high-standard learning situation for its students. AHCTS operates a rolling intake system for the level 2 programme, which enables students to enter the course at the earliest opportunity rather than having to wait until the next programme starts in order to be admitted. The institute is distinctly cookery focused, and any other disciplines that are covered such as food service provide a relevant adjunct to the cookery skills. This specialisation in one vocational stream allows AHCTS tutors to focus on their area of expertise. The relatively small class sizes and spacious, well-equipped workrooms allow for a good learning environment. At the lower levels, the additional generic content helps students understand the important qualities required to prepare for employment. International students are currently in the level 4 programme, which has been successful to date, with all these students engaged in employment. Hospitality students are not traditionally academically inclined, and the programmes have been tailored so that the mix of theory and practical content is appropriate and acceptable. In the lower-level programmes practicums, workplace visits and work experience help the students to understand and adapt to the rigours of the industry. 11

12 Level 4 students are encouraged to directly apply and reflect on the skills they learn at AHCTS in their place of work wherever possible. These students are taught more complex skills, with some self-directed periods. The classroom demographic comprises significant proportions of Māori, Pasifika and international students, which makes for a varied cultural mixture. The predominant culture promoted in the classroom is kitchen culture of attention to food hygiene, cross-contamination awareness, work speed and team communication, so the tutors treat all students the same irrespective of background. Students understand this and graduates are keen and enthusiastic about the learning environment. AHCTS collects and analyses feedback from external stakeholders, and there is evidence that the information is used to inform improvements and development. The PTE is taking steps to initiate a more formalised system of feedback-gathering from employers and graduates, and the managers expect to gain a more organised and structured insight into the needs of students and industry as a result. These developments are in their formative stages and the effects are expected to be evident in the near future. The PTE is not currently involved in the NZQA-led Targeted Review of Qualifications process, which is engaged in the development of a set of New Zealand qualifications for all sector groups. It does have an advisory committee which contains representatives from other schools, a polytechnic and industry and employers, and gives advice on programme changes and resources. The committee meets annually, but selected members are contacted for advice on specific matters as the need arises. Currently, a gap has been identified in the committee and a secondary school representative is being sought. It is felt that a secondary school representative will help AHCTS gain an understanding of how it can best link with secondary school leavers to develop another form of student catchment. 1.4 How effective is the teaching? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Good. The quality of teaching at AHCTS is of a good standard. The tutors are all highly skilled practitioners with much industry experience. AHCTS realises the importance of teaching skills, as many of the tutors have teaching qualifications and the knowledge required to teach unit standards, or are in the process of acquiring these with the assistance of the school. The tutors are passionate and supportive of the students and have sound knowledge of the rigours of the hospitality industry. The evaluators observed that the teaching relationships are inclusive and collegial. Tutors are all accountable for developing their teaching and trade skills under individual performance agreements and undergo formal observations of their teaching practice. The reassessment policy is adding to the higher completion 12

13 rates. The validity of assessment for commercial competence is enhanced by verification of the processes by workplace supervisors. As stated earlier in this report, AHCTS has very good facilities and equipment. Attention to positive educational principles is shown by maintaining small class sizes and three spacious kitchens. Important soft skills are being developed in the students as a consequence of the training. These include motivation, selfassurance and independent learning. The quality of the teaching and the resources encourages students to advance to programmes at higher levels. The evaluators noted that the students would benefit from having a programme outline and tracking schedule to plot and plan their own progress through the programme. The PTE engages in internal post-assessment moderation as required, but further improvement to moderation practices needs to be developed. Those assessment and teaching materials that are not generated by ServiceIQ require careful design and moderation to ensure they are fit for purpose and of the highest standard. Internal moderation will ensure the tools are adequate for use, and annual external moderation in these areas is an overall quality check that the processes are working. Teaching staff would improve their educational practice if they engaged in more self-reflective practices and had a better understanding of the significance of outcomes data. Teaching performance will be improved with a focused emphasis on understanding student individual achievement and adopting a more student-centred approach in the future. 1.5 How well are learners guided and supported? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Good. AHCTS supports its students very well, both educationally and in pastoral care. A good level of formal and informal communication is maintained with the students. As mentioned, the students enjoy the informal discussion that Facebook provides. The evaluators also saw evidence that tutors took a constructive interest in students extramural problems that may have an effect on the learning, for example in finding accommodation for a student. The AHCTS website provides a good source of information for current and prospective students, and a marketing tool for the school. The AHCTS student handbook is an important document which provides programme information, support services contact information and a reference point for AHCTS regulations and academic rules. Workbooks are provided to regulate and streamline the learning. Work experience for level 2 students is carefully monitored by the tutors to ensure the students are gaining a worthwhile experience. AHCTS provides further support through the enrolment process where learning requirements are appraised through the written application and the interview 13

14 procedure. Aspects such as cultural needs, literacy and numeracy requirements, English language proficiency, disabilities and special learning requirements are checked at enrolment. Student orientation gives students an introduction to the school and their programme. These actions are effective in determining student needs and assisting the students to become comfortable in their learning environment. Level 2 students are given a uniform by AHCTS, and do not have to purchase their own knives. The evaluators were told that at level 2, the rolling enrolment makes literacy and numeracy testing difficult, and so progress in this important area is not able to be quantified and analysed. This an area where improvements are needed to allow literacy and numeracy development to be measured and monitored. Generally, students feel supported and know who to approach for help. International students receive additional support, especially when they first arrive at the school. The school arranges airport pick-ups and initial transport to the student s accommodation. It further supports the international students by arranging trips and activities and assistance with extramural activities such as setting up bank accounts, negotiating public transport and locating medical services. Management has a good understanding of the Code of Practice for the Pastoral Care of International Students. Student feedback of their experiences at AHCTS is very positive. There are few complaints from students. The school deals promptly with disruptive students so that the general student body is not adversely affected. A few students have been expelled from the school in the past for such indiscretions as drug abuse and/or inappropriate behaviour. AHCTS also has a designated recruitment manager who provides support to new students. 1.6 How effective are governance and management in supporting educational achievement? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Good. Governance and management of the school is effective, with a strong emphasis on meeting the requirements of the funding agencies. The school has one ownerdirector who maintains a financial overview of the school but has little direct involvement in the operational side. This provides a clear focus on overarching governance matters. The general manager has overall management control, and there is a senior management team that meets monthly to make operational decisions. Good use of self-assessment is made in that senior management decision-making is informed by recommendations from the advisory committee, which includes members of other learning institutions and industry. As stated earlier, the advisory committee also provides advice on programme development 14

15 and curriculum design, and on the purchase of resources. This allows industry and educational perspectives to be consulted in the planning of future developments. There is good cross-institutional engagement at senior management level. The general manager is also the chairman of the advisory board at Tai Poutini Polytechnic. Industry focus groups are put together for special projects. An important initiative that has been fostered by management is the formation, with other institutions, of a consortium of local hospitality providers to pool resources and act as a lobby group for hospitality education issues. While still in its early stages, this group has approached ServiceIQ to seek clarification of various issues arising from the Targeted Review of Qualifications, and the group realises the potential for future collaboration and for working together to provide mutual services such as external moderation. The immediate focus for AHCTS is on the development of the international student market, but management is also aware of other commercial opportunities. Although there is plenty of competition in the international student market, management at AHCTS believes their point of difference is quality. This quality is achieved by having a focused curriculum and keeping the operation and the student body small to give particular attention to the students. The engagement of a designated quality assurance manager will enable management to maintain a focus on implementing informed improvement. The evaluators noted that since the last EER (2009) the PTE has become more focused on self-assessment. The PTE engages in extensive gathering of student achievement data, mostly collected to meet the requirements of funding agencies. Other recent self-assessment activities include needs analysis through the risk development plan, a review of policies and procedures, and a move to formalise self-assessment strategies. AHCTS is taking positive steps to improve its business as a PTE through the extended establishment and embedding of a structured selfassessment strategy. 15

16 Focus Areas This section reports significant findings in each focus area, not already covered in Part Focus area: Governance, management and strategy The rating in this focus area for educational performance is Good. The rating for capability in self-assessment for this focus area is Good. 2.2 Focus area: National Certificate in Hospitality (Introduction Cookery) (Level 2) The rating in this focus area for educational performance is Good. The rating for capability in self-assessment for this focus area is Good. 2.3 Focus area: National Diploma in Hospitality (Cookery) (Level 4) The rating in this focus area for educational performance is Good. The rating for capability in self-assessment for this focus area is Good. 2.4 Focus area: International student support The rating in this focus area for educational performance is Good. The rating for capability in self-assessment for this focus area is Good. 16

17 Recommendations NZQA recommends that Auckland Hotel and Chefs Training School: 1. Assist teaching staff to improve educational practices by engaging in more self-reflective procedures and adopting a more student-centred approach to learning. 2. Improve the understanding of student individual achievement among the teaching staff. 3. Improve formal data-gathering from external stakeholders, particularly employers. 4. Continue improving the analysis and utilisation of self-assessment data to inform improvement and development. 17

18 Appendix Regulatory basis for external evaluation and review External evaluation and review is conducted according to the External Evaluation and Review (EER) Rules 2013, which are made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. Self-assessment and participation in external evaluation and review are requirements for maintaining accreditation to provide an approved programme for all TEOs other than universities. The requirements are set through the NZQF Programme Approval and Accreditation Rules 2013, which are also made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. In addition, the Private Training Establishment Registration Rules 2013 require registered private training establishments to undertake self-assessment and participate in external evaluation and review, in accordance with the External Evaluation and Review Rules (EER) 2013, as a condition of maintaining registration. The Private Training Establishment Registration Rules 2013 are also made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. NZQA is responsible for ensuring non-university TEOs continue to comply with the rules after the initial granting of approval and accreditation of programmes and/or registration. The New Zealand Vice-Chancellors Committee (NZVCC) has statutory responsibility for compliance by universities. This report reflects the findings and conclusions of the external evaluation and review process, conducted according to the External Evaluation and Review (EER) Rules The report identifies strengths and areas for improvement in terms of the organisation s educational performance and capability in self-assessment. External evaluation and review reports are one contributing piece of information in determining future funding decisions where the organisation is a funded TEO subject to an investment plan agreed with the Tertiary Education Commission. External evaluation and review reports are public information and are available from the NZQA website ( The External Evaluation and Review (EER) Rules 2013 are available at while information about the conduct and methodology for external evaluation and review can be found at NZQA Ph E qaadmin@nzqa.govt.nz 18