Report of External Evaluation and Review

Transcription

1 Report of External Evaluation and Review Helicopter Flight Training Ltd trading as HFT Limited Highly Confident in educational performance Confident in capability in self-assessment Date of report: 21 January 2014

2 Contents Purpose of this Report... 3 Introduction TEO in context Scope of external evaluation and review Conduct of external evaluation and review... 4 Summary of Results... 6 Findings... 8 Recommendations Appendix MoE Number: 7994 NZQA Reference: C11610 Date of EER visit: 27 and 28 June

3 Purpose of this Report The purpose of this external evaluation and review report is to provide a public statement about the Tertiary Education Organisation s (TEO) educational performance and capability in self-assessment. It forms part of the accountability process required by Government to inform investors, the public, students, prospective students, communities, employers, and other interested parties. It is also intended to be used by the TEO itself for quality improvement purposes. Introduction 1. TEO in context Name of TEO: Type: Location: Delivery sites: Helicopter Flight Training Ltd (HFT), trading as HFT Limited Private training establishment (PTE) De Havilland Lane, Ardmore Airport, Auckland Head office, hangar and training centre as above First registered: 21 July 2011 Courses currently delivered: New Zealand Diploma in Aviation (Helicopter General) New Zealand Diploma in Aviation (Helicopter Flight Instruction) New Zealand Diploma in Aviation (Helicopter Instrument Rating) Code of Practice signatory? Number of students: Yes (but no international students currently) Domestic: 25 EFTS (equivalent full-time students) International: nil at present Number of staff: Scope of active accreditation: Distinctive characteristics: Seven full-time and two part-time Accreditation for New Zealand Diploma in Aviation (as above) HFT is one of only three helicopter schools in New Zealand approved by the Tertiary Education Commission (TEC) for funding, student loans and allowances for the New Zealand Diploma in 3

4 Aviation. Recent significant changes: Previous quality assurance history: Between 2010 and 2012, HFT operated under subcontract to Whitireia Community Polytechnic to offer the Certificate in Aviation and Advanced Certificate in Aviation. HFT secured approval from the TEC in its own right to deliver the newly registered New Zealand Diploma in Aviation from January HFT was registered in July This is the first quality assurance visit by NZQA. 2. Scope of external evaluation and review The scope for the external evaluation and review (EER) consisted of the mandatory focus area: Governance, management, and strategy 1 In addition, the following focus area was selected: Diploma in Aviation (all strands). 3. Conduct of external evaluation and review All external evaluation and reviews are conducted in accordance with NZQA s published policies and procedures. The methodology used is described fully in the web document Policy and Guidelines for the Conduct of External Evaluation and Review available at: The TEO has an opportunity to comment on the accuracy of this report, and any submissions received are fully considered by NZQA before finalising the report. The EER was conducted in June Prior to the EER visit, the lead evaluator visited the school and met with the deputy chief flying officer to discuss and agree on the scope and process for the EER. The evaluation team consisted of two evaluators. A one and a half-day EER visit was made to the HFT site at Ardmore Airport, where the evaluation team reviewed a range of documentation and met with the owner, teaching staff, two graduate students and three groups of current students. The evaluation team also had telephone discussions with four external 1 Although HFT is a signatory to the Code of Practice for the Pastoral Care of International Students, because the organisation does not currently have international students enrolled, support for international students was not treated as a separate focus area but rather included as part of governance management and strategy. 4

5 parties representing aspects of the aviation industry and including Aviation Services Limited (the body delegated to test and examine pilots). 5

6 Summary of Results Statement of confidence on educational performance NZQA is Highly Confident in the educational performance of Helicopter Flight Training Ltd trading as HFT Limited. The students who have studied at HFT have achieved excellent results. In the period 2010 to 2012, 94 per cent of students successfully completed their qualifications. Retention for 2013 is 100 per cent. Although the organisation is relatively new, the processes and systems that have been put in place are systematic and robust and are expected to continue to produce good outcomes. HFT is strongly employment focused and driven by the belief that students must receive good value from their experience and their qualification. This is borne out by the fact that all but one of the students who graduated between 2010 and 2012 are now working in the aviation or related industries. Feedback from graduates and industry indicates that courses have a well-constructed mix of theory and practice and that the courses are delivered in a manner that enables students to understand and achieve. The quality of the training provided at HFT is affirmed by a range of stakeholders who describe the graduates as work-ready, well presented and with well-developed skills and knowledge. HFT is in regular contact with its graduates and the results indicate that the organisation is preparing students well for employment in the aviation industry. Feedback from students indicates that HFT programmes are delivered and assessed in a manner that enables them to understand and apply the material being presented. The organisation has a student support structure that is appropriate to the needs of its students and removes the barriers to students achieving their goals. Students are very well supported pastorally, academically and vocationally. The organisation is well governed and managed, engages well with its profession and industries, is providing quality training and is valued for the positive contribution it makes. Statement of confidence on capability in self-assessment NZQA is Confident in the capability in self-assessment of Helicopter Flight Training Ltd trading as HFT Limited. HFT has established a highly reflective organisation and a culture that encourages students, staff and stakeholders to discuss their performance and put forward ideas for improvement. The school is long term in its thinking and closely attuned to the needs of its students and the aviation industry. Student achievement is examined individually and HFT has a strong focus on individual student achievement. 6

7 While processes to critically analyse contributing factors to student achievement and other quality improvements are largely informal and intuitive, there is evidence that self-assessment is taking place across the organisation. Staff at HFT demonstrated a good intuitive understanding of the factors that lead to student achievement. They regularly discuss and assess ideas for improving achievement and making the courses more useful and enjoyable for students. The evaluation team believes that as the self-assessment processes continue to be implemented at HFT, and the organisation further engages with the key evaluation questions, significant and meaningful ongoing improvements will result. HFT is aware that the self-assessment processes need to be organised into a planned and coordinated approach across the organisation. 7

8 Findings How well do learners achieve? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. The data collected by HFT from 2010 to 2012 shows that, of the 71 students who enrolled during this period 3, 67 (94 per cent) successfully completed their qualifications. The national average across the PTE sector for completions of qualifications at this level was 80 per cent. Student retention so far for 2013 is 100 per cent. Total Enrolments Successful Completion Percentage Completion National Average Period % 80& The excellent results being achieved by students at HFT have come about through good teaching, outstanding support and the will of the organisation to understand the factors that contribute to achievement and to ensure that every student achieves. HFT staff members are able to account for the four students who had not completed, and offer explanations as to why they thought the students did not complete and the lessons learnt by the organisation as a result. For instance, two of the four students were over 50 years of age and realised they could not sustain the nature of the demands to become a helicopter pilot. As a result, HFT now ensures that mature applicants are made fully aware of the demanding nature of the course and the industry so that they can make an informed choice before committing to enrolment. Both the organisation and its staff have a strong focus on learner achievement and set targets and benchmarks for student progression. The staff members expect that every student who attends an HFT course can succeed, and staff members work hard to keep all students focused on completion. Individual student progress is closely monitored and then regularly shared between instructional staff. Staff at HFT demonstrated a sound understanding of the factors that lead to student achievement and regularly analyse and discuss and implement ideas for improving achievement. For instance, HFT has made the application process more rigorous, including at least two written tests, interviews and a trial flight, to ensure that 2 The findings in this report are derived using a standard process and are based on a targeted sample of the organisation s activities. 3 During this period students were studying for the Certificate or Advanced Certificate in Aviation, which HFT offered under sub-contract to Whitireia Community Polytechnic. 8

9 applicants are well informed about the requirements of the course. This is not done with a view to excluding students but rather to enable students to self-select from an informed position, and this has resulted in more focused and committed students. Aviation Services Limited, the body that flight and theory tests pilots under delegated authority from the Civil Aviation Authority, reports that HFT students perform well under test conditions. Aviation Services Limited also reported that HFT is respected within the industry for the quality of pilots it produces. The deputy chief flying officer closely monitors exam achievement and uses the information as a measure of overall student achievement. Students reported that they received timely, accurate and supportive feedback on their progress from HFT staff. All students meet with a staff member at least once each week for one-to-one discussion about progression, achievement and goalsetting. Students are focussed on their qualifications and are highly motivated to achieve them. Staff and employers reported that students are learning useful skills, and staff members see positive changes, such as confidence and self-esteem, in students as they progress. 1.2 What is the value of the outcomes for key stakeholders, including learners? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. The courses and staff at HFT are highly employment focused and staff members have a good understanding of what it takes for graduating students to be successful in the helicopter industry. HFT staff members ensure that prospective and incoming students have realistic expectations about their employment prospects in a very competitive industry. The immediate outputs from this training are licences and ratings as awarded by the Civil Aviation Authority, and these also meet the requirements of the International Civil Aviation Organization. Students achievement of these licences and ratings equip them to apply for a range of roles within the aviation industry in New Zealand or internationally. HFT knows the destination of every graduate. All but one of the 67 graduates from 2010 to 2012 are working in the aviation or related industries. Those working in related industries, for example agricultural groundwork and tourism, are doing so with the intention of positioning themselves for work as a pilot when the opportunity arises. The evaluation team interviewed one recent graduate who had been involved in an emergency landing in early 2013, about six months after graduation. She 9

10 described hearing a loud noise and experiencing a complete loss of power just after take-off with three passengers on board. She successfully ditched the helicopter in Lake Rotorua without loss of life or injury. A senior emergency services officer said that, the pilot did a fantastic job landing the helicopter safely. 4 The pilot attributed her response directly to her training at HFT. She said she had trained for emergency landings so many times that she was able respond automatically: In the few seconds I had to get the helicopter down, I even managed to get off a mayday call! I don t know how I did it, but I felt I could see and hear my instructor and simply acted instinctively. This pilot s employer also confirmed the value of her training and said that he would happily employ more HFT graduates and that HFT will be his first point of contact when he next requires staff. HFT also has good relationships within the rescue helicopter services network, which in turn benefits students. For instance, the Northland Emergency Services Trust (NEST) offers internships for HFT graduates. According to the NEST chief pilot, all three internships have so far resulted in the interns being offered full employment piloting similar aircraft with other companies or services. 1.3 How well do programmes and activities match the needs of learners and other stakeholders? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Good. Evidence from stakeholders, including graduates and students, indicates that HFT engages well with its sector and is valued for the contribution that it makes. HFT management and staff have ongoing interactions with stakeholders, including ongoing contact with graduates, Aviation Services Limited, the Civil Aviation Authority and the rescue helicopter network. HFT management staff are variously involved both professionally and informally in the aviation sector and are well respected for the part they play, often in leadership roles. These ongoing connections benefit students in that HFT staff members are able to keep abreast of developments in the industry and tailor courses to address emerging changes in industry. For instance, HFT has identified that one of the growth areas in helicopter use is for frost protection in orchards and vineyards work that due to its repetitive nature is often available to new pilots but which nevertheless requires some specialist skills such as night flying and understanding thermal layering. HFT has built these aspects into the course. The goals and aspirations of every student are individually identified early in the programme and an individual programme tailored to suit. For instance, if a student 4 Rotorua Daily Post, 25 February

11 particularly wishes to enter the tourism industry, instructors will, whenever possible, contextualise their training accordingly. HFT does not structure its ground school in the same way as many other schools, whereby students complete all of the theory study and examinations before they start flying. Rather, HFT delivers its ground school concurrently with practical flight training. Management and staff believe that this structure not only motivates students, but most importantly enables students to apply the theory they learn immediately to a practical context and vice versa. The organisation has worked hard over time to ensure that the teaching is closely related to the needs of the students. HFT seeks feedback from learners through a variety of informal mechanisms. Staff members have an open-door policy and it is clear that students approach them with problems and issues. Every Tuesday morning there is a full staff and student meeting which acts as a forum for discussion and which is actively used and appreciated. Staff members were able to cite examples of how course content and delivery methodology had been changed as a result of feedback from learners or external stakeholders, for instance the contextualising of training to meet individual needs, previously referred to in this section. The formal student feedback process (which used to be administered by Whitireia Community Polytechnic under the partnership agreement) will not be implemented until towards the end of this year. Use of formal student feedback processes will enable HFT staff members to better analyse and understand the learning and teaching processes. 1.4 How effective is the teaching? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Good. There was evidence of good teaching practice taking place at HFT. Instructors are enthusiastic and passionate about their students and teaching and they are well supported by management. Students enjoy the open, friendly style and have good access to the teaching staff outside of class hours. Students noted that staff members are responsive to any concerns or issues they raised. The strong rapport between students and staff is confirmed from discussions. There was evidence of the whole organisation purposefully engaging in meaningful discussion about teaching practice and student progress. Instructors spoke about how they share ideas, experiences and knowledge through their daily interactions, staff meetings and peer observations. The current ratio of students to instructors is low and this allows for a high level of one-to-one attention and ready access to aircraft to build up flying hours. Instructors give students comprehensive feedback after every training flight. Students report that assessments are returned to them promptly and that they 11

12 contain comprehensive feedback, indicating that staff members acknowledge that timely feedback to students is important so the students have the opportunity to reflect and build on their learning. It was encouraging to hear from several of the instructors at HFT that they see instruction as a long-term career, rather than simply as a means to build up flying hours to enable them to move on to a commercial aviation job. Although instructors study basic adult teaching as part of their instructor ratings, further and ongoing study in, for example, the National Certificate of Adult Education and Training, would further improve teaching effectiveness. Likewise, regular input by an external adult teaching expert would also assist staff to identify and implement improvements to their teaching. Teachers know their subjects well but have not been taught how to adapt their teaching style to meet the needs of different learners or learning styles. Instead, when a learner has trouble grasping a skill or process with one tutor, another tutor may be used to help the learner achieve that component. Tutors appear to be unaware of differing learning styles and needs and how to capably address these in the material they deliver. However, learners do not see this as a concern as they feel exposure to other tutors ways and methods enriches their learning. 1.5 How well are learners guided and supported? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. HFT has robust and client-friendly systems for the pastoral care of its students. The organisation is providing a safe and supportive learning environment for its students and is in close contact outside of the normal hours of tuition. The relatively small number of student means that they are given individual support. Students report that they feel comfortable contacting HFT staff for assistance and always get a helpful response. The students, who come from a wide variety of backgrounds and previous experiences, are strongly engaged with their learning, and have input into the direction their study should take in response to clear, intentional guidance from HFT staff. The organisation has developed a rigorous yet informative process of preenrolment guidance for intending students. This involves at least two written tests, an interview and a trial flight. The purpose of this testing, rather than to exclude students, is to ensure that before committing to the course, they are fully aware of the demands of the course and the industry and that HFT has identified any individual support the student may need. HFT reports that this regime is working well. In the past year, the TEC has put a cap on the amount of student loan funding available to aviation students. This change has meant that HFT students now have 12

13 to fund approximately 30 per cent of the cost, or about $30,000 for the Diploma of Aviation. HFT staff report that the change has led to a decrease in the number of applicants but a noticeable increase in the motivation and determination of those who do apply. In the interests of equity, HFT has put aside sufficient funds to provide scholarships for up to four students per year, targeted at applicants who they feel will be an asset to the aviation industry but who do not have the means to privately fund their fees. To date, the scholarship funds have been particularly targeted towards Māori and Pasifika applicants. HFT has a strong belief in the positive correlation between student attendance and achievement and therefore monitors attendance closely. Students are expected to be on site for 40 hours per week for the duration of their study. When not flying or in formal classes, they are expected to be studying individually, planning, or participating in rostered duties such as flight following, refuelling, checking equipment etc. HFT has found that this total immersion environment helps students to stay focused and enables HFT staff to remain connected with the students and constantly monitor their progress. As well, the total immersion approach enables learners to experience the daily realities they will face in their roles as pilots, having to adapt to changes in schedules and/or aircraft due to weather or mechanical issues. The HFT student management system requires students to register in and out daily using their swipe card. In doing so, they are also automatically provided with daily electronic NOTAMS (Notices to Airmen) and must check the box to confirm that they have read them before entering the hangar. Students fees are held in trust and students are provided with a weekly reconciliation of their flying hours and student loan and/or trust fund drawdown which they are required to sign off each week. Students report that the reconciliations are always on time, accurate and easy to understand. HFT is a signatory to the Code of Practice for the Pastoral Care of International Students for students over the age of 18 years. At present, HFT does not have any international students but is in the process of embarking on a recruitment strategy (see next section). 1.6 How effective are governance and management in supporting educational achievement? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Good. HFT is a privately owned and operated aviation training provider with a clear set of goals and objectives. The organisation has developed and embedded very effective systems for monitoring student achievement and supporting staff to improve educational achievement. HFT s management recognise that the key to 13

14 success of the business lies in the success of the students and the value they gain from their training. HFT actively supports strategies to maintain and increase student achievement and to support students to meet their employment goals. HFT has employed well-qualified and experienced staff and is well supplied with physical and learning resources for the number of students that it currently has. There are systems in place to constantly monitor resourcing to ensure that there are always sufficient resources, mainly aircraft and instructors, to meet the needs of the students. HFT also have a Civil Aviation Authority-approved flight simulator which each student uses for approximately 20 hours during the course of their training, to simulate cross-country VFR 5, night flying, instrument flying and GPS for VFR pilots. This is the only Civil Aviation Authority-approved helicopter simulator used in a training school in New Zealand. The organisation has moved away from the industry norm of employing instructors on an hourly basis (for flying hours only) and has employed the majority of instructors on salary. Although it is more costly, HFT s management considers that this practice ensures that staff are available to support students with groundwork study, and that it also creates a more collegial environment as instructors are not competing with each other for flying hours. Management monitor organisational and individual performance at all levels of the organisation in a regular, transparent and open manner. HFT has developed a very good proprietary student management system which enables the staff to easily and effectively monitor student progression and achievement, aircraft use, flying hours, etc. The evaluation team observed coherence across all staff in their focus on giving students the best experience possible, to equip them to perform and achieve. The organisation encourages opportunities for reflection on its role and how to continue to make ongoing and continuous improvements to how it meets the needs of students and other stakeholders. Evidence indicates that the owners actively seek feedback from many sources to use as a learning and improvement tool, and students and staff spoke of the no blame, no shame culture that is promoted throughout the organisation. While the focus of HFT s self-assessment has centred on identifying and responding to student and industry needs which has resulted in a sound reputation within the industry the next challenge for HFT is to focus its efforts towards increasing its understanding of educational performance to sustain current levels of achievement by embedding systematic, whole-of-organisation analysis and ensuing action from self-assessment activities. The organisation intends to expand by recruiting international students, mainly from China and India. Although targeted numbers are relatively conservative, it may still result in a 200 per cent increase in student numbers, two-thirds of whom will be international. This 5 Visual Flight Rules 14

15 situation will surely test HFT s systems and processes, so it is important that they are thoroughly embedded prior to the expansion so that the organisation can maintain the excellent performance and value it is currently providing. 15

16 Focus Areas This section reports significant findings in each focus area, not already covered in Part Focus area: Governance, management and strategy The rating in this focus area for educational performance is Excellent. The rating for capability in self-assessment for this focus area is Good. 2.2 Focus area: Diploma in Aviation (all strands) The rating in this focus area for educational performance is Excellent. The rating for capability in self-assessment for this focus area is Excellent. 16

17 Recommendations NZQA recommends that: 1. HFT consider the possibility of either contracting regular in-house customised sessions about adult learning and teaching, or encouraging teaching staff members to pursue formal qualifications in adult education. Either way, in addition to having well-honed knowledge and skills in aviation, instructors should also have equally well-developed knowledge and skills in the underpinning principles of how adults learn. 2. HFT invite NZQA Support Services to visit in 12 months time, to provide feedback on progress with self-assessment processes, improvements and results. 17

18 Appendix Regulatory basis for external evaluation and review External evaluation and review is conducted according to the External Evaluation and Review (EER) Rules 2013, which are made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. Self-assessment and participation in external evaluation and review are requirements for maintaining accreditation to provide an approved programme for all TEOs other than universities. The requirements are set through the NZQF Programme Approval and Accreditation Rules 2013, which are also made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. In addition, the Private Training Establishment Registration Rules 2013 require registered private training establishments to undertake self-assessment and participate in external evaluation and review, in accordance with the External Evaluation and Review Rules (EER) 2013, as a condition of maintaining registration. The Private Training Establishment Registration Rules 2013 are also made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. NZQA is responsible for ensuring non-university TEOs continue to comply with the rules after the initial granting of approval and accreditation of programmes and/or registration. The New Zealand Vice-Chancellors Committee (NZVCC) has statutory responsibility for compliance by universities. This report reflects the findings and conclusions of the external evaluation and review process, conducted according to the External Evaluation and Review (EER) Rules The report identifies strengths and areas for improvement in terms of the organisation s educational performance and capability in self-assessment. External evaluation and review reports are one contributing piece of information in determining future funding decisions where the organisation is a funded TEO subject to an investment plan agreed with the Tertiary Education Commission. External evaluation and review reports are public information and are available from the NZQA website ( The External Evaluation and Review (EER) Rules 2013 are available at while information about the conduct and methodology for external evaluation and review can be found at NZQA Ph E qaadmin@nzqa.govt.nz 18