Report of External Evaluation and Review

Transcription

1 Report of External Evaluation and Review South Seas Film and Television School Limited Highly Confident in educational performance Highly Confident in capability in self-assessment Date of report: 1 September 2015

2 Contents Purpose of this Report... 3 Introduction TEO in context Scope of external evaluation and review Conduct of external evaluation and review... 5 Summary of Results... 6 Findings... 7 Recommendations Appendix MoE Number: 8655 NZQA Reference: C18632 Dates of EER visit: 3 and 4 June

3 Purpose of this Report The purpose of this external evaluation and review report is to provide a public statement about the Tertiary Education Organisation s (TEO) educational performance and capability in self-assessment. It forms part of the accountability process required by Government to inform investors, the public, students, prospective students, communities, employers, and other interested parties. It is also intended to be used by the TEO itself for quality improvement purposes. Introduction 1. TEO in context Name of TEO: Type: South Seas Film and Television School Limited Private training establishment (PTE) First registered: April 1992 Location and delivery site: Courses currently delivered: 75 Ellice Road, Glenfield, North Shore, Auckland Diploma in Film and Television Production (Level 5) Diploma in On Screen Acting for Film and Television (Level 5) South Seas Diploma in Animation Production (Level 6) Diploma in Digital Photography (Level 6) Code of Practice signatory: Number of students: Number of staff: Scope of active accreditation: Distinctive characteristics: Yes 167 full-time domestic students, including 34 Māori and nine Pasifika students; 20 international students (2014 figures) 12 full-time and 15 part-time academic staff; six full-time and one part-time support staff A variety of domains in relation to media training up to level 7: South Seas is a school specialising in intensive, practical training for the film, television and media 3

4 industries. The school is set up as a production house and students engage in production or shooting projects throughout the programme. Recent significant changes: Previous quality assurance history: Other: N/A The previous external evaluation and review (EER) of South Seas was conducted in June NZQA was Highly Confident in both the educational performance and capability in selfassessment of South Seas. South Seas receives SAC (Student Achievement Component) funding from the Tertiary Education Commission (TEC). 2. Scope of external evaluation and review The scope of this EER included the mandatory focus area of governance, management and strategy. Other focus areas selected were: Diploma in Film and Television Production (Level 5) This programme was one of the focus areas in the previous EER and has consistently been the programme with the largest number of students enrolled in the school (usually about per cent of students). Diploma in Digital Photography (Level 6) This programme was nominated by South Seas as another focus area, as a subject outside film and television production. It was not selected as a focus area at the previous EER and is at a higher level to the flagship programme. Together the two programmes provide good representation of the overall operations of South Seas. 4

5 3. Conduct of external evaluation and review All external evaluation and reviews are conducted in accordance with NZQA s published policies and procedures. The methodology used is described fully in the web document Policy and Guidelines for the Conduct of External Evaluation and Review available at: The TEO has an opportunity to comment on the accuracy of this report, and any submissions received are fully considered by NZQA before finalising the report. Self-assessment materials were delivered to NZQA in a timely fashion to inform the scoping of this EER. Prior to the on-site visit, the lead evaluator met with representatives of the school to agree on the focus areas and logistical matters. The evaluation team comprised three evaluators. The on-site visit lasted two days. The evaluation team interviewed the three company directors (one of whom is also the school director and another the school s general manager); all course directors for the four programmes currently being delivered; the six heads of department within the faculty of film and television production; the photography tutor and the tutor assistant; 11 and seven students from the film and television production and photography programmes respectively; the company accountant and the pastoral care officer. A range of organisational documents were reviewed during the on-site visit. 5

6 Summary of Results Statements of confidence on educational performance and on capability in self-assessment NZQA is Highly Confident in the educational performance of South Seas Film and Television School Limited. NZQA is Highly Confident in the capability in self-assessment of South Seas Film and Television School Limited. Completion rates at South Seas are consistently high, as are positive employment outcomes. Sound statistical and qualitative analysis is used to inform faculty discussions and regular reviews of students progress and achievement, to ensure students are appropriately supported. These processes have contributed to the highly positive outcomes. Students acquire relevant technical skills through hands-on practice in a production house environment, and develop soft skills that are crucial within the media industries such as problem-solving on the fly, time management and social skills through working as a production team crew. Graduates are issued with a detailed skills report that covers both technical and soft skills achievement and which clearly identifies strengths and capabilities to prospective employers, supporting employment within the desired sector. The excellent achievement is supported by a comprehensive selection and admission process. Each applicant is screened through an in-depth interview, with emphasis on motive, drive and attitude. A portfolio of work is examined for Diploma in Digital Photography applicants, while a short taster course is available to those interested in the Diploma in Film and Television Production. Teaching is effective and is demonstrated via the engagement level of students. Students drive the learning process and academic staff act in a mentoring role which is appropriate for the artistic nature of the training concerned. Academic staff retain close connections to the industry and use such networks effectively in the teaching, the support to students and ongoing programme reviews. South Seas demonstrates an embedded student-centric organisational culture. Issues such as emotional support and counselling to artists have been identified in a timely manner and an appropriate response put in place promptly. Governance and management is highly effective. There is a cohesive alignment of operations to the organisational purpose and learning ethos. South Seas demonstrates a thorough understanding of its educational and business performance through very effective, comprehensive, coherent and timely self-assessment. A reflective culture is well-embedded across the school. Capability for continuous improvement is strongly evident. 6

7 Findings How well do learners achieve? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. Students acquire technical skills that are relevant to media industries (film and television or digital photography, depending on the programme enrolled in) through hands-on practice in a production house environment, giving graduates a solid foundation for employment within the desired sector. Students also refine personal and professional attributes that are crucial within the media industries, such as problem-solving on the fly, punctuality, time management, working as part of a team, and social, communication and presentation skills. Graduates are issued with a detailed skills report that covers both technical and soft skills achievement. The school keeps a complete record of all its learner achievements since establishment. Achievement rates, in terms of completions, are consistently very high, as demonstrated in Tables 1 and 2 below for the two focus area programmes. Table 1. Statistics on enrolment and completion for Diploma in Film and Television Production at South Seas Film and Television School, Calendar year Enrolments Completions Data source: South Seas Film and Television School 96.7% 97.7% 93.3% Table 2. Statistics on enrolment and completion for Diploma in Digital Photography at South Seas Film and Television School, Intake 2012A 2012B 2013A 2013B 2014A Enrolments Completions Data source: South Seas Film and Television School 1 The findings in this report are derived using a standard process and are based on a targeted sample of the organisation s activities. 2 The completion percentage is not presented for this programme because of the small base numbers. 7

8 Completion rates of priority groups are generally comparable to other ethnic groups. South Seas advised that what appears to be lower completion percentages by priority groups are due to the small base numbers for those groups. Table 3. Course and qualification completion rates by ethnic group, Calendar year Māori 91% 90% 83% Course completion Pasifika 100% 82% 89% Other 92% 94% 94% Māori 90% 97% 84% Qualification completion Pasifika 100% 93% 88% Data source: South Seas Film and Television School Other 97% 98% 95% South Seas holds regular faculty meetings to discuss achievement progress and operational matters. In particular, the evaluation team found the use of data and the level of analysis applied to inform faculty discussions and decision-making particularly impressive regular progress reviews are conducted based on sound statistical and qualitative analysis. Furthermore, the evaluation team observed informal, student-centric practices for monitoring and evaluating students academic progress on a continual basis. The school demonstrates that it is able to maintain a very high level of achievement results and reacts to any issue or concern hindering educational performance in a timely manner. 1.2 What is the value of the outcomes for key stakeholders, including learners? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. Employment outcomes from the Diploma in Film and Television Production are very high. Outcomes for Māori graduates are comparable to non-priority groups. Table 4. Percentage of graduates from the Diploma in Film and Television Production that successfully secured a contract within the film and television industry within six months of graduation Calendar year * Māori 86% 96% TBA Other ethnicities 88% 95% TBA Data source: South Seas Film and Television School *Final figures for 2014 were not available at the time of the evaluation team s on-site visit in June 2015, as graduates are not tracked until six months after graduation (i.e. in July). 8

9 Film and television production students acquire a broad-based skillset during the first quarter of the programme and specialise in one of six areas in the remaining months. 3 Feedback from the industry confirms that such an all-encompassing, generalist knowledge base further enhances the employability and mobility of graduates. Employment outcomes are also strong for the Diploma in Digital Photography taking into consideration the nature of work for a commercial photographer. South Seas only counts contractual photography work or self-employment as a freelance commercial photographer as a successful employment outcome, which can be conservative as there may be graduates who manage to apply the skills learnt in the programme to their work in other fields. Thus, the evaluation team considers the successful employment outcome of over 70 per cent since 2013 as strong performance overall. Table 5. Number of graduates from the Diploma in Digital Photography that successfully became a commercial photographer within six months of graduation Intake 2012A 2012B 2013A 2013B 2014A Completions Success Data source: South Seas Film and Television School The faculty was able to provide insights into and evidence of analysis of the above figures. For example, an account of events was articulated to the evaluation team surrounding the performance of the first 2012 cohort, and a range of remedial actions (including relevant decisions relating to staff performance management) was taken immediately to ensure more positive outcomes for subsequent cohorts. Overall, the above statistics show a healthy majority of graduates obtain relevant contracts within the media industry. Students are given plenty of opportunities to establish networks with current practitioners throughout the programme. Students also learn business skills essential to becoming a freelance contractor or selfemployed within the industry, such as GST invoicing. Such a foundation is of high value to students and graduates, who all expressed a desire to work in the industry. South Seas is highly focused on employment outcomes. It has been successfully tracking graduate destination outcomes since its establishment as a PTE in The school demonstrates constant reflection, using graduate destination analysis, to inform and provide assurance of the value of outcomes and relevance of programmes. 3 Six specialisation streams are: art and design; camera, lighting and audio; post-production editing; production management; directing and scriptwriting; documentary and research. 9

10 Beyond employment, South Seas also contributes to the local community by producing television programmes for ethnic groups, public access broadcasting or special interest organisations. 1.3 How well do programmes and activities match the needs of learners and other stakeholders? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. South Seas was set up with the aim of supplying the relevant sectors with a constant stream of capable talent trained to industry standards. All academic staff are current practitioners, which helps maintain the school s connection with and awareness of the ongoing evolution of the media industry. The school takes pride, rightly so, in its ability to understand and meet the needs of the sector. A variety of reviews are in operation as self-assessment practices. For example, there are annual planning reviews, week-by-week reviews (reflecting on successes and improvement opportunities on programme delivery), industry surveys, employer surveys. Academic staff record their self-reflection on a regular basis and incorporate improvements to delivery upon reflection. The evaluation team observed a cohesive alignment of training practices to the organisational purpose and learning ethos. The Diploma in Film and Television Production programme is designed to closely imitate a production house environment. Students across different specialisations sometimes together with students on the Diploma in On Screen Acting for Film and Television are grouped into production crews. Learning is via hands-on practices, with strong peer support as well as mentoring from academic staff. Production output is expected on a weekly or fortnightly basis, with each production building on skills developed to date. Students improve their technical skills and soft skills (as mentioned in Findings 1.1) as they work closely together as a team on projects throughout the year. The Diploma in Digital Photography programme emphasises training for commercial photography. There is a minimum of one shooting project per week, with a thoughtful balance of team and individual assignments, as well as shooting on campus and shooting on location. Assignments (shooting projects) are initially linear-based in the first half of the programme, eventually moving into overlapping projects in the second half of the programme, reflecting the realities of the life of a commercial photographer. Subsequent assignments integrate elements from previous ones to support the acquisition and reinforcement of knowledge and skills, which is vital to reinforce learning as students face a steep learning curve and progress from level 4 to level 6 within a relatively short period of time. The selection and admission process is sound. Prospective students complete an online application for pre-selection. An interview is conducted either in person or 10

11 over internet videoconferencing facilities (for out-of-town and international students) to ensure mutual suitability. Emphasis is placed on applicants motivation, talent and attitude. A short course taster is offered before committing to any of the offered programmes. Students may elect to withdraw without penalties within the first 10 days of commencement. The school collects and records the reason for students dropping out, and reviews the reasons and reflects on its practices. 1.4 How effective is the teaching? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. Teaching is effective and is demonstrated by the engagement levels of students, as confirmed in surveys conducted by South Seas, and also through feedback from students interviewed by the evaluation team at the on-site visit. Students drive the learning process and academic staff act more in a mentoring role which is appropriate for the artistic nature of the training content. The quality of staff is supported by recruitment processes (see Findings 1.6), ongoing professional development, engagement within the relevant industries, and formal annual performance appraisals. Students interviewed by the evaluation team expressed their appreciation of the valuable personal experiences often shared by academic staff. The evaluation team noted exemplary assessment and moderation practices in operation at South Seas. Clear marking schedules and rating rubrics are communicated to students. In the Diploma in Film and Television Production, the composition of the final grade is deliberately designed to be different for each specialisation, appropriately reflecting the differences in importance of skills in each area. In the Diploma in Digital Photography, assessment is always marked by two academic staff members, reducing the subjectivity in judging fine arts. A purposeful, effective moderation plan is now in place, after some discrepancies in the implementation of moderation process were identified by South Seas. An external advisor has been engaged and a refined set of policies and procedures were developed and implemented in 2014 to improve moderation practice. This has changed the organisational culture. There is evidence that findings from moderation are now discussed and used to improve assessment materials and assessor judgements. 1.5 How well are learners guided and supported? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. 11

12 South Seas demonstrates an embedded student-centric organisational culture. The evaluation team heard a number of examples of how issues have been identified in a timely manner and an appropriate response put in place promptly. For example, the school is aware that actor training may open up the possibility of surfacing and/or exacerbating mental health issues, which the student may or may not have previously disclosed or even be aware of. South Seas initiated regular, early morning breathing and relaxation sessions to help students to handle stress, and have made counselling services available on site, although it is too early to see the effect of this because of low uptake. A training initiative is in place to encourage film and television production Māori students to produce drama or documentaries in Te Reo Māori. The initiative encourages students to build closer relationships with Māori TV and other Māori production companies, and directly contributes to the positive employment outcomes of this priority group (see Findings 1.2). Given the de facto production house environment, absences have an impact on the progress of the production team, which may lead to delays in completion of team projects, and are therefore immediately noticed and followed up by staff and students. Students have one-to-one access to academic staff on an as-requested basis. Staff regularly discuss and share student issues (within privacy parameters) at formal meetings and in informal conversations. The school has a comprehensive set of guidance mechanisms that clearly support its excellent educational outcomes. 1.6 How effective are governance and management in supporting educational achievement? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. Governance and management is highly effective. The school has a very clear purpose and direction: training talent for the film, television and media industries through practical, hands-on experience. A comprehensive, up-to-date strategy and business plan is in place, with very detailed information that covers all aspects of the school s current and future operations. The school has a flat and fit-for-purpose organisational structure, which has demonstrated efficiency and effectiveness in implementing its corporate strategy and responding to changes in its operating environment. For example, the school has representation on the Targeted Review of Qualifications and is well informed from its involvement in the development of New Zealand qualifications within the media and creative arts streams. The evaluation team observed clear communication channels, open dialogue, a positive working relationship across all 12

13 departments, and a highly positive, constructive vibe among all staff throughout the on-site visit. The board of directors meets five times a year to review the school s performance and discuss a variety of issues, such as strategic direction and student achievement. The board s membership has remained stable with no changes over the past 11 years. The three directors are credible, experienced and wellconnected current practitioners within the film and television industry. Members of the management team possess a diverse, complementary suite of skills. Staff are carefully selected based on an extensive industry background, with their performance appraised over time before being given a higher level of responsibility. The school has built up its facilities and resources over time. The campus is purposefully set up as a production house with relevant, current technology. The school has made significant capital investment in recent years, refitted its studio, adopted new digital media technology, and upgraded its internal information systems. South Seas demonstrates a thorough understanding of its educational and business performance through very effective, coherent and timely self-assessment. The evaluation team found a reflective culture well-embedded across the school. Its connectedness with both the media and tertiary education sectors, its modus operandi of being open and transparent throughout the organisation, as well as taking a student-centric approach, ensures the school promptly responds to and addresses any issues and concerns as they arise. Capability for continuous improvement was evident throughout the evaluation team s on-site visit. 13

14 Focus Areas This section reports significant findings in each focus area, not already covered in Part Focus area: Governance, management and strategy The rating in this focus area for educational performance is Excellent. The rating for capability in self-assessment for this focus area is Excellent. 2.2 Focus area: Diploma in Film and Television Production (Level 5) The rating in this focus area for educational performance is Excellent. The rating for capability in self-assessment for this focus area is Excellent. 2.3 Focus area: Diploma in Digital Photography (Level 6) The rating in this focus area for educational performance is Excellent. The rating for capability in self-assessment for this focus area is Excellent. The evaluation team commended the programme leader on the thoughtful manner demonstrated in assisting the evaluation team s understanding of this faculty s operations during the on-site visit. The evaluative conversations were highly articulated and appropriately supported with evidence in an efficient fashion, effectively contributing to the smooth conduct of the evaluation team s on-site processes. The evaluation team noted, in randomly sampled student surveys, consistent comments on the way feedback is given. It is understood that artists often have strong personalities and opinions. Perhaps it may be worthwhile to incorporate an assessment of the ability to accept critical feedback as part of the selection and admission process as such ability is crucial to an artist s development and improvement. Consideration should also be given to further ongoing professional development of academic staff in terms of adult training qualifications at a higher level, which will likely provide additional tools and skills for providing constructive feedback to students. 14

15 Recommendations NZQA recommends that South Seas Film and Television School: Incorporate an assessment of ability to accept critical feedback as part of the selection and admission process. Consider supporting the academic staff in the photography faculty to achieve higher-level adult training qualifications. 15

16 Appendix Regulatory basis for external evaluation and review External evaluation and review is conducted according to the External Evaluation and Review (EER) Rules 2013, which are made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. Self-assessment and participation in external evaluation and review are requirements for maintaining accreditation to provide an approved programme for all TEOs other than universities. The requirements are set through the NZQF Programme Approval and Accreditation Rules 2013, which are also made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. In addition, the Private Training Establishment Registration Rules 2013 require registered private training establishments to undertake self-assessment and participate in external evaluation and review, in accordance with the External Evaluation and Review Rules (EER) 2013, as a condition of maintaining registration. The Private Training Establishment Registration Rules 2013 are also made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. NZQA is responsible for ensuring non-university TEOs continue to comply with the rules after the initial granting of approval and accreditation of programmes and/or registration. The New Zealand Vice-Chancellors Committee (NZVCC) has statutory responsibility for compliance by universities. This report reflects the findings and conclusions of the external evaluation and review process, conducted according to the External Evaluation and Review (EER) Rules The report identifies strengths and areas for improvement in terms of the organisation s educational performance and capability in self-assessment. External evaluation and review reports are one contributing piece of information in determining future funding decisions where the organisation is a funded TEO subject to an investment plan agreed with the Tertiary Education Commission. External evaluation and review reports are public information and are available from the NZQA website ( The External Evaluation and Review (EER) Rules 2013 are available at while information about the conduct and methodology for external evaluation and review can be found at NZQA Ph E qaadmin@nzqa.govt.nz 16