Report of External Evaluation and Review

Transcription

1 Report of External Evaluation and Review Royal Business College Limited Highly Confident in educational performance Confident in capability in self-assessment Date of report: 2 July 2014

2 Contents Purpose of this Report... 3 Introduction TEO in context Scope of external evaluation and review Conduct of external evaluation and review... 6 Summary of Results... 7 Findings... 9 Recommendations Appendix MoE Number: 8352 NZQA Reference: C14102 Dates of EER visit: 10, 11 and 12 March

3 Purpose of this Report The purpose of this external evaluation and review report is to provide a public statement about the Tertiary Education Organisation s (TEO) educational performance and capability in self-assessment. It forms part of the accountability process required by Government to inform investors, the public, students, prospective students, communities, employers, and other interested parties. It is also intended to be used by the TEO itself for quality improvement purposes. Introduction 1. TEO in context Name of TEO: Type: Location: Royal Business College Limited Private training establishment (PTE) 394 Riccarton Road, Christchurch Delivery sites: Christchurch campus 394 Riccarton Road, Christchurch Te Puke campus The Palmer Place, 46 Jellicoe Street, Te Puke Auckland campus 60 Federal Street, Auckland Hastings campus Level 5 The Tower, 215 Railway Road, Hastings First registered: August 1996 Courses currently delivered: Business National Diploma in Business (Levels 5 and 6) Diploma in Business Management (Level 7) RBC Diploma in Entrepreneurship (Level 7) Computing National Diploma in Computing (Level 5) RBC Diploma in Computing (Level 6) RBC Diploma in Information Technology (Level 7) 3

4 Horticulture RBC Diploma in Horticulture (Level 5) English General English Academic English (RBC IELTS Preparation Course) Code of Practice signatory: Number of students: Yes for learners 14 years of age and above Around 450 learners for the 2013 academic year Number of staff: Management and administrative support: 16 Academic: 26 Scope of active accreditation: (In addition to courses currently delivered) Business Diploma in International Marketing China (Level 5) Computing Youth Guarantee Certificate in Computing and Business Administration (Level 2) Tourism RBC Certificate in Tourism and Travel (Core Skills) (Level 3) National Certificate in Adventure Tourism (Level 4) National Diploma in Tourism (Management) (Level 5) Horticulture National Certificate in Horticulture (Introductory) (Level 2) National Certificate in Horticulture (Level 4) Distinctive characteristics: All learners are international students, except for the 26 EFTS (equivalent full-time students) funded by the Tertiary Education Commission (TEC). The majority of learners are of Indian ethnicity. Recent significant changes: The Christchurch campus has been operating from temporary accommodation since the Canterbury 4

5 earthquake of February The Te Puke campus opened in The Hastings campus opened in A new principal for the college was appointed in early Previous quality assurance history: The previous external evaluation and review (EER) of Royal Business College was conducted in NZQA was Highly Confident in the educational performance, and Confident in the capability in self-assessment of the college. The college had a validation visit from NZQA in January The visit identified a small number of minor issues and these have since been rectified. 1 Under a voluntary agreement with Primary ITO (formerly Horticulture ITO), Royal Business College ceased enrolling learners into the level 2 and 4 horticulture programmes from June Other: Royal Business College receives SAC (Student Achievement Component) funding from the Tertiary Education Commission (TEC) for its level 5 and 6 computing programmes. 1 Minor issues identified included: non-filing of a statutory declaration for any conflict of interest; referring to the Ministry of Education as administrator of the Code of Practice for the Pastoral Care of International Students in marketing materials and published information; and some processes missing from the quality management system (QMS). 5

6 2. Scope of external evaluation and review The scope of this EER included the following mandatory focus areas: Governance, management and strategy International students Other focus areas selected were: National Diploma in Business (Levels 5 and 6) Combined, the two programmes constitute approximately 40 per cent of all learners enrolled at the college (based on 2013 statistics). RBC Diploma in Entrepreneurship (Level 7) This programme has the largest group of learners enrolled at the college (approximately 25 per cent of all learners based on 2013 statistics). 3. Conduct of external evaluation and review All external evaluation and reviews are conducted in accordance with NZQA s published policies and procedures. The methodology used is described fully in the web document Policy and Guidelines for the Conduct of External Evaluation and Review available at: The TEO has an opportunity to comment on the accuracy of this report, and any submissions received are fully considered by NZQA before finalising the report. Self-assessment materials were delivered to NZQA in a timely fashion to inform the scoping of this EER. Prior to the scope of the evaluation being finalised, the lead evaluator met with the owner/director and principal in person at the Christchurch campus to agree on the focus areas and logistics of the on-site visit. The evaluation team comprised two evaluators. The on-site visit lasted three days, during which the Christchurch, Auckland and Hastings campuses were visited. The evaluation team interviewed the owner/director, three members of the academic advisory board (including an external consultant engaged long-term by Royal Business College), the principal of the college, the director of studies at the Hastings campus, 11 business tutors and 36 learners across the Christchurch, Auckland and Hastings campuses, and five management and administrative support staff at the Auckland and Hastings campuses, including the associate principal of marketing, who is based at the Auckland campus. A range of materials and records was reviewed at each campus visited. Several stakeholders (such as external moderation partners) were contacted by phone. 6

7 Summary of Results Statement of confidence on educational performance NZQA is Highly Confident in the educational performance of Royal Business College Limited. Royal Business College learners achieve very high completion rates (average of 94 per cent across all programmes and campuses in 2013) and positive outcomes for graduates, with an average of 86 per cent gaining employment or pursuing further education upon graduation (see Findings 1.1). While the learners motives for enrolment vary, the college seizes the opportunity and equips them with a quality education (see Findings 1.2). The educational value in enrolling in a level 7 programme for learners with higher degrees (Bachelor s and/or Master s levels) is clearly articulated by the college and the learners (see Findings 1.2). The college identifies, understands and meets the needs of learners, employers, local community and iwi. Programmes are designed with current labour market demand in mind. Examples of support provided to the Christchurch, Hastings and Ngāi Tahu communities are cited (see Findings 1.3). The academic team is highly qualified and effective in their teaching. The college embraces adult education principles and financially supports the professional development of all lecturers (see Findings 1.4). The evaluation team examined assessment and moderation practice in detail and is assured that this identified weakness area is being managed by the college effectively, including the engagement of credible, quality external moderation partners. The evaluation team is satisfied this will ensure valid assessment of achievement (see Findings 1.4). The college demonstrates how support and guidance to learners, including individual attention and support, contributes to the success in high achievement rates (see Findings 1.5). Governance and management is highly effective. There is a clear division of responsibilities within the management team, fully utilising the expertise of individual members. A commitment to quality and academic excellence is well embedded within the college (see Findings 1.6). 7

8 Statement of confidence on capability in self-assessment NZQA is Confident in the capability in self-assessment of Royal Business College Limited. Royal Business College analyses its achievement data by year, by programme and by campus. Findings are reported regularly to governance and management. Annual targets for completion rates are set and reviewed as part of the performance management process (see Findings 1.1). The college conducts regular learner, graduate, employer and stakeholder surveys to gauge the value of outcomes and to understand their changing needs, and responds by developing or fine-tuning programmes to meet those needs (see Findings 1.2 and 1.3). Staff self-appraise every quarter and discuss their performance and reflections with the local academic manager, which contribute to ongoing improvements in teaching practice within the college (see Findings 1.4). The academic teams meet weekly and report to the head office in Christchurch to inform them of operations. Effective processes are in place to monitor and track individual learner progress and attendance, and academic support requirements (see Findings 1.4 and 1.5). Governance and management is fully informed of the college s operations and performance through many different regular reports. Quality improvement plans are in place each year. A culture of reflective learning and continuous improvement is truly embedded within the college (see Findings 1.6). There is room for improvement to review and realign systems and processes for self-assessment to enhance coherency and efficiency (see Findings 1.2 and 1.6). 8

9 Findings How well do learners achieve? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. Course completion rates are very high across all programmes offered by Royal Business College at all campuses. Completion rates were in the high 80 per cent range in This improved to an average of around 94 per cent in The evaluation team examined assessment and moderation practices in detail and is convinced that the rates reported are valid (see Findings 1.4). The college presented abundant evidence to the evaluation team on achievement rates data in a timely manner, accompanied by analysis of such success. The data presented is dissected by year, by programme and by campus. The evaluation team is highly satisfied with the systems and processes in place, and found clear linkages showing how they contribute to the excellent educational performance of the college. These include understanding learners study objectives and motivations, ongoing review of relevance in course content and design against changing industry demand (see Findings 1.3), commitment to employ and develop a highly capable and effective teaching team (see Findings 1.4), close monitoring and tracking of individual progress and support requirements (see Findings 1.5), and a governance and management structure that facilitates academic excellence in its fields (see Findings 1.6). There is also evidence that learners gain increased awareness of New Zealand culture and local knowledge, as well as enhanced communication, presentation and time management skills through their studies at Royal Business College. In turn, these skills improve graduates employability domestically. The evaluation team is satisfied that governance and management is fully informed of achievement outcomes. As part of the performance management process, targets are set and lecturers are aware of the outcomes associated with their respective programmes/courses. There may be an opportunity to publicise such information more widely within each campus or even across different campuses. A sense of healthy internal competition, if considered appropriate to the management philosophy and culture of the college, may be another driver for continuous improvement and add to the already excellent self-assessment practices in terms of this key evaluation question. 2 The findings in this report are derived using a standard process and are based on a targeted sample of the organisation s activities. 9

10 1.2 What is the value of the outcomes for key stakeholders, including learners? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Good. Learners achieve the immigration outcomes they desire through completing studies at Royal Business College, a key motivator for enrolling in the various programmes offered by the college. This is also observed by the evaluation team through the changes in programmes on offer over the years, which is closely aligned to changes in immigration policy. Having said the above, the evaluation team is very satisfied that, regardless of the motivation of many learners, Royal Business College seizes the opportunity and equips its learners with a quality education, and plays an important role in ensuring its learners graduate to become contributing members of the New Zealand society. The evaluation team observed that some learners enrolled at the college came with higher-level degree(s) conferred overseas (in some instances, degrees at Master s level), and explored this phenomenon with the college and its learners. Some learners said they wanted to understand business practices within the New Zealand environment; some saw the level 7 programme on offer at Royal Business College as a pathway to higher studies in New Zealand universities, because of the nonrecognition of their overseas degrees; some wanted to acquire new knowledge and skills for a change in career; others wanted to gain practical skills for employment on completing their university studies. This group of learners was able to clearly articulate the value of completing a New Zealand qualification at a level lower than their immediate previous academic experience. The evaluation team is satisfied that Royal Business College is delivering education of value to its learners. There is strong evidence that employers also value the outcomes delivered by Royal Business College. High levels of employment outcomes are noted in graduate destination surveys conducted by the college. Feedback from employers, as reported in stakeholder surveys conducted by the college, are highly positive in terms of graduates employability and knowledge of local practices. The evaluation team heard that some employers paid their employees for their studies at the college, which is an indicator of the value of the outcomes to these employers. The fact that a good proportion of learners enrol on the basis of word-of-mouth referrals is another indicator of the value of the outcomes as perceived by learners and graduates. The college also contributes to the development of local communities (see Findings 1.3). Royal Business College collects graduate destination outcomes data. However, the data was presented to the evaluation team in raw form without any accompanying detailed analysis (for example, analysis of the relevance of employment outcomes to a particular course of studies). This is an area the college can enhance further in terms of self-assessment practices. 10

11 1.3 How well do programmes and activities match the needs of learners and other stakeholders? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. Learners needs are identified, understood and met by the college. Programmes are taught in modules, with a number of intakes carefully planned throughout the year to ensure learners can commence with the foundation modules. Contents and materials are tailored to the New Zealand environment. Contact hours are scheduled, with local variations, to allow learners to participate in part-time employment, taking into account the labour market demand of different cities. For example, in the main centres (Christchurch and Auckland) classes are held in the mornings while in the regional campuses (Hastings and Te Puke), full-day classes are held on specific weekdays instead. Laptops are available to all learners on loan during campus hours, which facilitates learning and is appreciated by the learners interviewed by the evaluation team. A strong emphasis is given to employability an outcome desired by the majority of the learners with assistance given for seeking employment in New Zealand. The needs of the local community are identified and well met by the college. Royal Business College continues to demonstrate its commitment to and involvement in the Christchurch earthquake recovery. Plans are in place to reconstruct the college s building destroyed in the earthquakes by The college has been in discussions with Ngāi Tahu about a project for developing programmes in Christchurch desired by the local iwi. While the project is currently on hold, discussions are ongoing and the project is expected to recommence once the construction of the new campus is completed. The establishment of the Hastings campus addresses the need of the local community to attract international students to the region. The college operates a number of effective mechanisms to gauge the needs of its stakeholders. Learner surveys at the time of enrolment, stakeholder surveys and the operation of an advisory board are examples of these mechanisms. The evaluation team heard about how the college responded to learner feedback by further developing a level 7 diploma in marketing into a diploma in business management. This involved research and careful consideration by the advisory board when reviewing the programme s relevance to industry requirements. This example illustrates the exemplary performance of the college in understanding and meeting changing market demand (both of its clients and of the domestic labour market). 11

12 1.4 How effective is the teaching? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Good. Academic staff are highly qualified 24 out of 26 staff have postgraduate-level qualifications, with four holding doctorates. The majority have teaching experience of at least six years. Royal Business College demonstrates a strong commitment to professional development by financially supporting academic staff to undertake training in adult education, for those who do not have appropriate qualifications in that area. The evaluation team is satisfied that lecturers at the college understand the needs and different learning styles of learners and adjust their teaching methods accordingly. Teaching effectiveness is monitored through regular surveys of a random sample of learners. Survey results are analysed and reported to the principal. The evaluation team noted learners high levels of satisfaction with their lecturers teaching overall (over 90 per cent satisfaction). Lecturers at each campus meet weekly to discuss a variety of academic matters, including learner progress and achievement. Minutes of meetings are shared with the principal at the head office. Staff complete a selfappraisal process every quarter, which is then used as the basis of performance discussions with the local academic manager. There is evidence of peer observations within individual campuses (but not between campuses). Suggestions for improvement in teaching delivery from lecturers at different campuses are communicated directly to the principal. There is insufficient evidence to show whether such practices and suggestions are disseminated to other campuses once communicated to head office. It appears there is room for improvement in crosscampus liaison, collaboration and sharing of ideas, particularly learnings and best practices from the Auckland campus, where there is demonstrably very strong academic leadership and robust processes (such as internal moderation) in place. Prior to the on-site visit, assessment and moderation practice at the college had been flagged as a potential risk area through various internal and external sources. The evaluation team held detailed conversations on the topic with managers and lecturers as well as thoroughly examining internal and external moderation records. The evaluation team is assured by the college that the concerns raised as a result of national external moderation outcomes are isolated to the gathering of evidence for assessment purposes during work placements specifically in relation to certain employers refusing to certify/verify a learner s practical work. The college said it has since refined its process for employer selection for placements, as well as ensuring an understanding is in place about what is required of the employer. On its own initiative, the college engaged credible, quality partners in the polytechnic sector for external moderation. Positive feedback is received by the evaluation team from these partners, and the evaluation team is satisfied that a stringent process is in place to ensure standards are raised. While the 2013 national external moderation results is inconclusive in supporting actual improvements from the efforts mentioned above, the evaluation team is satisfied that moderation 12

13 practice is, more likely than not, an area for improvement being managed effectively by the college. Learners may be allowed up to three attempts within a fortnight to achieve each unit standard. Clear information on assessment requirements and guidance, including one-to-one support, is provided to learners, which supports the high level of achievement. 1.5 How well are learners guided and supported? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Good. Royal Business College provides its international learners with the usual pastoral support services expected from a tertiary institute positioned for the international education market, such as airport pick-up, initial advice and assistance in settling into New Zealand, including accommodation arrangements. Learners interviewed by the evaluation team were highly satisfied with the well-being and pastoral support provided by the college and expressed appreciation for the approachability of staff. There is also evidence that the college provides assistance to learners in finding employment. The evaluation team is satisfied that the college meets its legal obligations under the Code of Practice for the Pastoral Care of International Students, including appropriate supervision of agents practices, which were discussed at length during the on-site visit. English-level entry requirements are being adhered to. This could be further enhanced by periodically verifying the validity and effectiveness of the internally designed/administered English-level entry test. Royal Business College operates a number of mechanisms to support academic achievement. The college monitors learners attendance closely, reviews attendance records weekly and promptly follows up on attendance concerns. The evaluation team examined the associated process, and found the policy being adhered to and the system effective. Individual progress records are kept by both the college and each learner. There is evidence that academic staff regularly discuss with individual learners their progress. These are some of the processes in place that contribute to the high completion rates. The evaluation team found the Hastings campus resources to be insufficient for the business and computing programmes being delivered there. While tutors have access to electronic reference materials, there was no evidence presented that learners enjoy the same level of access. The current campus/classroom setup is perhaps less than ideal for teaching information technology although the evaluation team acknowledges that there is room for immediate expansion given the availability of space in the same building. 13

14 1.6 How effective are governance and management in supporting educational achievement? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Good. Royal Business College has a clear purpose and direction. The drive for delivering quality education to international learners is apparent at all levels of staff and across all campuses visited by the evaluation team. The college engages an education consultant to conduct its own external evaluation on an annual basis. Findings are acted on and lead to ongoing improvements year after year, through the development and execution of annual quality improvement plans. Royal Business College is governed by a well-represented academic advisory board, which meets twice a year. The board is informed by periodic academic and business reports produced by management. The evaluation team sighted meeting minutes, heard examples of ideas generated and discussed at the meetings, and found that a healthy, effective work relationship exists between the governing board and the management team. The owner/director has extensive experience in the export education industry. There is a clear division of responsibility, where the owner/director applies his expertise in marketing, networking and business management, while the academic team enjoys autonomy and is entrusted with service delivery. The evaluation team commends, in particular, the strong academic leadership at Auckland campus, where staff are noticeably motivated, highly engaged, committed to the college s vision and demonstrate a high level of understanding of their responsibilities and how they align with the college s business plans. Programme design is centralised in the Christchurch head office, and campuses serve as delivery sites. The evaluation team is satisfied that there are effective mechanisms in place to ensure standardisation of practice across all campuses. The evaluation team is satisfied that the college has truly embedded a reflective culture to promote continuous improvement. It appears, however, that some of the self-assessment practices in place could be further strengthened for coherency. For example, learners study objectives and destination outcomes are collected for different purposes (understanding learner needs and establishing value of outcomes). An opportunity has been missed in not comparing the study objective against the destination outcome of the individual learner to establish a success rate for matching learner needs, simply by using data that is already being collected. This is not only a useful self-assessment measure, but potentially powerful information for marketing as well. 14

15 Focus Areas This section reports significant findings in each focus area, not already covered in Part Focus area: Governance, management and strategy The rating in this focus area for educational performance is Excellent. The rating for capability in self-assessment for this focus area is Good. 2.2 Focus area: International students The rating in this focus area for educational performance is Excellent. The rating for capability in self-assessment for this focus area is Good. 2.3 Focus area: National Diploma in Business (Levels 5 and 6) The rating in this focus area for educational performance is Good. The rating for capability in self-assessment for this focus area is Good. The rating in educational performance for this focus area reflects the need for Royal Business College to continue improvements in its moderation practice sustained improvements are yet to be demonstrated. 2.4 Focus area: RBC Diploma in Entrepreneurship (Level 7) The rating in this focus area for educational performance is Excellent. The rating for capability in self-assessment for this focus area is Good. 15

16 Recommendations There are no explicit recommendations arising from this external evaluation and review other than those mentioned and/or implied throughout this report. 16

17 Appendix Regulatory basis for external evaluation and review External evaluation and review is conducted according to the External Evaluation and Review (EER) Rules 2013, which are made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. Self-assessment and participation in external evaluation and review are requirements for maintaining accreditation to provide an approved programme for all TEOs other than universities. The requirements are set through the NZQF Programme Approval and Accreditation Rules 2013, which are also made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. In addition, the Private Training Establishment Registration Rules 2013 require registered private training establishments to undertake self-assessment and participate in external evaluation and review, in accordance with the External Evaluation and Review Rules (EER) 2013, as a condition of maintaining registration. The Private Training Establishment Registration Rules 2013 are also made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. NZQA is responsible for ensuring non-university TEOs continue to comply with the rules after the initial granting of approval and accreditation of programmes and/or registration. The New Zealand Vice-Chancellors Committee (NZVCC) has statutory responsibility for compliance by universities. This report reflects the findings and conclusions of the external evaluation and review process, conducted according to the External Evaluation and Review (EER) Rules The report identifies strengths and areas for improvement in terms of the organisation s educational performance and capability in self-assessment. External evaluation and review reports are one contributing piece of information in determining future funding decisions where the organisation is a funded TEO subject to an investment plan agreed with the Tertiary Education Commission. External evaluation and review reports are public information and are available from the NZQA website ( The External Evaluation and Review (EER) Rules 2013 are available at while information about the conduct and methodology for external evaluation and review can be found at 17

18 NZQA Ph E qaadmin@nzqa.govt.nz 18