Report of External Evaluation and Review

Transcription

1 Report of External Evaluation and Review Oasis International Education Limited trading as Retail Business and Management College Confident in educational performance Confident in capability in self-assessment Date of report: 15 April 2015

2 Contents Purpose of this Report... 3 Introduction TEO in context Scope of external evaluation and review Conduct of external evaluation and review... 5 Summary of Results... 6 Findings... 8 Recommendations Appendix MoE Number: 7987 NZQA Reference: C16417 Date of EER visit: 3 and 4 December

3 Purpose of this Report The purpose of this external evaluation and review report is to provide a public statement about the Tertiary Education Organisation s (TEO) educational performance and capability in self-assessment. It forms part of the accountability process required by Government to inform investors, the public, students, prospective students, communities, employers, and other interested parties. It is also intended to be used by the TEO itself for quality improvement purposes. Introduction 1. TEO in context Name of TEO: Type: Location: Delivery sites: Oasis International Education Limited trading as (and referred to within this report as) Retail Business and Management College (RBMC) Private training establishment (PTE) Innes Road, Mairehau, Christchurch As above First registered: 9 January 2012 Courses currently delivered: National Diploma in Business (Level 5 and 6) Diploma in Retail Management (Level 7) Code of Practice signatory: Number of students: Yes for students aged 18 years and upwards Domestic: nil International: 100 equivalent full-time students at the time of the external evaluation and review (EER) Number of staff: Scope of active accreditation: Four full-time equivalents National Diploma in Business (Level 5 and 6) Diploma in Retail Management (Level 7) Distinctive characteristics: RBMC is a small, privately owned educational organisation specialising in retail and business management-related training, enrolling international, full-fee paying students. It is located 3

4 in a multi-use educational precinct owned and operated by the University of Canterbury. Previous quality assurance history: NZQA Risk conducted a validation visit in February No actions were required by the PTE, and soon after students from another PTE which was closing were referred to and enrolled by RBMC in collaboration with NZQA. The Ministry of Education visited the PTE in March 2013 and found the organisation to be compliant with the Code of Practice for the Pastoral Care of International Students. NZQA external moderation in 2013 occurred for three unit standards. Two standards were found to have been assessed at the national standard and one assessment required modification before reuse. 2. Scope of external evaluation and review Focus Area 1: Governance management and strategy is a mandatory focus area. All students are international students, therefore international student support was covered as an integral part of the EER. Focus Area 2: National Diploma in Business (Level 5) and National Diploma in Business (Level 6). Seventy-two of the currently enrolled students are studying towards one of these two programmes. The level 5 qualification is a 40 teaching-week, 122-credit programme which aims to provide international and domestic learners with a nationally recognised qualification in business for people who hold or aspire to hold positions in operations involving a range of business activities and to carry out specific roles that involve the management and leadership of a team or business unit within that operational role. The level 6 qualification is also a 40 teaching-week, 123-credit programme which aims to provide learners with a nationally recognised qualification in business for people who hold or aspire to hold positions involving the management and leadership of different teams or business units and carry out specific roles within the business organisation. The only other programme, the Diploma in Retail Management (Level 7), had 28 students enrolled at the time of the EER, and its content and performance were considered more generally during the EER. 4

5 3. Conduct of external evaluation and review All external evaluation and reviews are conducted in accordance with NZQA s published policies and procedures. The methodology used is described fully in the web document Policy and Guidelines for the Conduct of External Evaluation and Review available at: The TEO has an opportunity to comment on the accuracy of this report, and any submissions received are fully considered by NZQA before finalising the report. Scoping of the EER was undertaken by and phone contact. The focus areas were agreed, and a draft agenda for the two-day visit by two NZQA lead evaluators was also mutually agreed. Meetings and interviews were held with two directors, all teaching staff and the two administrative and marketing staff. Almost all of the currently enrolled students were interviewed in three separate groups. A small sample of graduates was contacted by telephone. Three external stakeholders also met with the evaluators at the request of the PTE. Documentation sighted and/or discussed with RBMC included: a self-assessment summary provided prior to the on-site phase, which included educational participation and performance tables prepared for the EER (supplemented with additional information tracking cohort performance); curriculum materials such as teaching plans and course outlines; moderation samples (both internal and external); examples of teaching observations and student evaluation feedback, as well as samples of staff and student files. Planning and administrative documents sighted included staff meeting minutes, financial and planning documents, correspondence with industry, and staff CVs and job descriptions. Other supporting material such as facilities, prospectus, website and wall displays were also noted by the evaluators. Much of this documentation was sighted to triangulate the evaluative conversations described above. 5

6 Summary of Results Statement of confidence on educational performance NZQA is Confident in the educational performance of Oasis International Education Limited trading as Retail Business and Management College. RBMC is a relatively new PTE which has competent leadership, clearly documented goals and the necessary resources in place to support its ongoing and planned educational delivery. Although it is too early to demonstrate patterns of achievement, retention of students is strong, course and qualification achievement is occurring, and completion rates are increasing as students successfully complete their studies. The qualifications on offer, content being delivered, and the teaching all link explicitly to the intended New Zealand business context, in particular retail and associated supply chain employment. This specialisation is valued by students, graduates and employers. As a result, the PTE is building a credible reputation with students and local businesses which matches the goals and strategy behind the formation of the PTE. The programme portfolio matches students aspirations, the teaching is effective, and the PTE is gathering sound evidence that these inputs link to both learner achievement and the local labour market. Students with previous work experience and tertiary qualifications gained in their home country describe the content as transferable to their own home context, although most graduates have obtained employment in New Zealand, which is a common goal. Guidance and support of students clearly meets their needs. Staff are readily accessible to students, actively seek feedback from them informally and by scheduled survey, and make changes to delivery or initiate new activities as a response to student feedback. Although initial enrolments and some aspects of international recruitment were slower than anticipated, the directors have shown sound leadership in responding to change while supporting academic integrity. This was evidenced by their academically credible (and ethical) response to transferring student credit recognition applications, and adherence to their documented attendance and assessment requirements for some students enrolled from a closing PTE. 6

7 Statement of confidence on capability in self-assessment NZQA is Confident in the capability in self-assessment of Oasis International Education Limited trading as Retail Business and Management College. The RBMC organisational structure, staffing, programme portfolio and day-to-day operation correlate well with the objectives and plans provided to NZQA in order to gain initial registration. The PTE has a relatively formalised approach to selfassessment which reflects active use of a suitably structured quality management system. This approach is serving the PTE well now, and provides an appropriate infrastructure for maintaining quality as student numbers grow. There was evidence that individual student achievement is monitored closely and is appropriately and accurately recorded and reported. However, RBMC will need to develop a more comprehensive and systematic approach to aggregating and analysing performance at, for example, qualification level and by cohort. The PTE has basic but reliable knowledge of employment outcomes, and engages in ongoing follow-up with graduates. This also lends itself to further development. Annual planning and planning of semester and associated programme delivery at the PTE is appropriate to scale, and also clearly reflects the goals and strategy behind the formation of the PTE. Student, staff, industry and education sector input is used to varying degrees to add value to these processes. Processes for student recruitment, enrolment and integration into programmes of study are sound, well documented and meet regulatory requirements. There is evidence that the staff engage in formalised and informal reflective practices to make improvements in response to the experience they are gaining as student numbers grow. Compliance with the Code of Practice is intentional and foremost at RBMC, based on staff extensive experience with international students. Again, feedback from students is responded to appropriately. Although the PTE engages some useful tools and practices around teaching effectiveness, there is a need to be more systematic and robust around, for example, internal moderation planning and linking teaching observations more explicitly to performance appraisal and professional development plans. The fundamental infrastructure for effective self-assessment is in place at RBMC, but needs to be fully understood and activated in order to yield convincing evidence of benefits and improvements. 7

8 Findings How well do learners achieve? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Adequate. RBMC s first enrolments were 10 students referred to them following the closure of another Christchurch PTE in mid Five of these students were retained and completed their qualification. The other students either left study once they received fee refunds from the Public Trust, or discontinued their study soon afterwards, returning to their country of origin. Reportedly, some students were resistant to the attendance and participation expectations they encountered at RBMC. The first direct intake of new students occurred in February At the time of the EER, retention was higher than completion as more students were being retained in study than had successfully completed their studies, as indicated in the following tables ( intakes combined). Although this is not yet a pattern of strong performance, and the time students are taking to complete their qualifications is of concern, the trend shows acceptable and gradually improving achievement. Table 1. RBMC National Diploma in Business (Level 5) enrolments, retention and completions Intake Term 1 Term 2 Term 3 Term 4 New enrolments Retained in study Completed qualification Table 2. RBMC National Diploma in Business (Level 6) enrolments, retention and completions Intake Term 1 Term 2 Term 3 Term 4 New enrolments Retained in study* Completed qualification * Also includes those students progressing from the level 5 qualification 1 The findings in this report are derived using a standard process and are based on a targeted sample of the organisation s activities. 8

9 Students are gaining relevant retail and business management skills and knowledge on their programmes, and the fact that almost all students are working part-time in relevant settings allows exposure to situations where they are able to observe or apply theoretical knowledge. The evaluators came to this conclusion after investigating the programme content and delivery details and interviewing current students, a selection of graduates employed locally as well as two industry representatives. As a small PTE currently teaching a relatively small number of students, RBMC is able to closely monitor the progress of each student. Individual student course and qualification completion is recorded accurately by teachers and within the student management system. Management is using this information to track enrolments, withdrawals and completions, but as yet is not engaging in systematic analysis of performance. Some discrepancies in data were noted by the evaluators. The data provided to the evaluators also presented some difficulties for identifying trends in performance, but gave sufficient confidence that the student enrolment numbers and their correlation with in-progress results and final results is credible. Good results in external moderation by NZQA in 2013 also supports the reliability of student achievement data. However, the PTE will need to significantly advance in overall knowledge of learner achievement as student numbers increase, as is anticipated. 1.2 What is the value of the outcomes for key stakeholders, including learners? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Good. RBMC is delivering education and training which evidently relates well to sectors of economic growth and current employment opportunities in Christchurch, and has relevance to students home countries. Although most of the students have tertiary qualifications, and in some cases significant business-related experience gained in their home country (predominantly India), the focus of the content and teaching on New Zealand business conditions adds value to the employment most of them have been able to obtain concurrent with or following their studies. The PTE is maintaining contact with all locally based graduates, and has sound information about their current employment situation. In many cases graduates have progressed from entry-level casual positions to supervisory and lower-level management roles. In most cases the positions gained by graduates are congruent with the retail and business (service sector) focus of the qualifications being offered. The evaluators, although tentative in ascribing this evident career progress directly to the qualifications, judged that the knowledge and concepts learned are 9

10 being found relevant and are being applied in the workplace. In particular, graduates are helping to meet the labour market needs of the strongly performing Christchurch economy. Graduate networks are being utilised by staff and students at RBMC to assist in the cultural orientation of new students, to help them find suitable accommodation and also guide them in obtaining suitable part-time employment. This was confirmed by students and also industry representatives. Graduates engagement with the PTE, and their experience gained living in New Zealand, is in turn adding value to new international students arriving to study. The main limitations in current self-assessment practices relate to more formal tracking of graduates important here as numerous students interviewed asserted their intention of using their qualification in retail in India, which as they said is experiencing huge change and growth in a deregulating market 2 and gathering more formalised and specific data on any material or other benefits gained as a result of studying with the PTE. 1.3 How well do programmes and activities match the needs of learners and other stakeholders? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Good. Being small, RBMC has close and relatively informal staff/student relationships in which all students are known, including their living situation, part-time workplace and, crucially, particular learning and support needs. In-class learning and some relevant social activities arranged by the PTE in conjunction with the students cultivate this atmosphere. The PTE is also intentional in its external linkages (such as inviting guest speakers and networking at meetings) with industry, social services and Indian cultural organisations in Christchurch. Staged enrolments (with four intakes each year) are catering well to student need, and are appropriately linked to timetabled course delivery. New students benefit from the experience and from the existing networks of current students and graduates, assisting with accommodation and part-time employment needs, as well as wider socialisation and orientation. Student survey feedback (during the programme and at exit) provides good evidence of met vocational and study support needs. Programme content and the resulting qualifications are generic enough to link to a diverse range of commercial interests beyond retail, as is the intention of the PTE. Examples provided to the evaluators included a range of small-to-medium service

11 sector enterprises as well as larger logistics and warehouse operations. The timetable allows part-time employment in these businesses concurrent with study, while appropriately managing the attendance requirements as expected by student visa conditions. Evidence provided indicates that these links are in fact operating well. The small and decreasing number of students withdrawing relates to the clear expectations being set by management and staff, and the willingness of current students to comply with these expectations. Exit interviews support the PTE s knowledge of met need for those graduating or leaving earlier than planned. These protocols appear to match Immigration New Zealand and NZQA programme approval requirements and expectations. The processes and expertise for developing and delivering the programmes are robust, and in particular informed by the highly relevant experience and expertise of the academic director and the teaching staff. Ongoing maintenance of matched need is strengthened by the PTE s engagement with the Targeted Review of Qualifications, the PTE s three-year programme review cycle as proscribed by the quality management system policy, and active participation in NZQA s review of unit standards. As indicated above, the programme portfolio is apparently well matched to the Christchurch economy, which in retail and hospitality is growing significantly faster than the New Zealand-wide sector. 3 Although some evidence of engagement with relevant industry identities was provided to the evaluators, the PTE could do much more to demonstrate active and effective engagement, which has an impact on programme content and delivery. 1.4 How effective is the teaching? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Adequate. RBMC has employed appropriately qualified and experienced teaching staff who are well led by an experienced academic director and who undertake relevant planned professional development. Teaching staff are engaged in employersupported postgraduate study. Solid evidence of a range of useful and wellconsidered teaching practices was identified during this evaluation. The learning environment and associated facilities, including computer suite, library, Wi-Fi and recreation areas, are very good, although the current building seems to be at full capacity. 3 Statistics NZ Christchurch retail trade Indicator: June 2014 Quarter 11

12 According to feedback from current students, the majority of whom were interviewed, teachers relate well to students and explain key concepts clearly. Often these are also directly applied to New Zealand business scenarios, legislation and examples drawn from the retail environment. Although useful handouts and resources are provided, this is not simply teaching from a textbook, but is a more dynamic and locally informed teaching approach. These were key themes noted throughout the evaluation, and again link with the PTE s stated purpose and goals. The academic processes at the PTE, which are clearly described in the quality management system, are credible. Plagiarism controls are suitable for very small numbers of students but may need strengthening and formalising as numbers increase. Assessment feedback is useful and learner-centred and reportedly has reduced rates of assignment resubmission. There is good resourcing of additional out-of-scheduled class time tutorial support, which is utilised by many students. However, the evaluators suggested that the PTE will need to ensure that the robustness of resubmission processes are maintained and no lowering of standards occur given that in effect three opportunities to achieve the standard are being offered to those students requiring re-assessment. Students rate highly the quality of the teaching and the pastoral care of the staff, as evidenced both in interviews with the evaluators and in the survey feedback that has been captured and collated since enrolments began. Feedback from these surveys and management s responses could and should be shared with students. This would add more value to an already sound process. Teaching observations, although occurring, lack any obvious linkage to performance appraisal and planning of professional development. Although these weaknesses in practice are not significant and are mostly managed effectively, the PTE has a need to systematise and more fully embed them to gain maximum benefit. Student handbooks, course notes and teaching plans and related resources are of good quality. Internal moderation, although occurring, needs strengthening more structured scheduling is required to adequately cover all assessment events. External moderation results are sound and there is notable in-house capability around assessment design, administration and moderation among staff. 12

13 1.5 How well are learners guided and supported? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Good. This evaluation found credible evidence that the PTE meets the intent and requirements of the Code of Practice (e.g. first-language speakers offering culturally appropriate support and links to community; suitable processes for homestay accommodation; emergency contact availability; referral to services as required; cultural orientation and exposure to local communities, etc). There is also a low and decreasing number of early withdrawals from programmes, and an increasing number of referrals to the PTE from current students and graduates. Students are very well supported. Orientation and student activities at RBMC are comparatively basic but appropriate to scale, although the PTE could perhaps do more around socialisation and recreation as numbers grow and resources become available. Useful nonqualification related support includes CV preparation and job search assistance and training in New Zealand recruitment protocols, which also add value to the management components of the approved programmes. Networks with industry are informed through regular contact by the PTE, and this has helped numerous graduates to procure suitable employment. Expectations around assessment submissions and marking are clearly stated and provided to students in a timely manner. Attendance registers are maintained and expectations are clearly set and understood by students interviewed. Processes for monitoring and encouraging full attendance are clear, and apparently link to achievement. Credit transfer processes (as evidenced by the students who were referred to RBMC at the time of closure of another PTE) were suitably documented and credible. A significant feature of the culture at RBMC is the student-to-student support being provided as new arrivals join the existing cohort in a well-managed and planned process. The PTE values this interaction, and monitors its effectiveness. The evaluators found solid evidence that the PTE responds well to students and is maintaining good relationships. The challenge for the PTE is to gain a full and comprehensive understanding of educational performance and the role of support and guidance activities in contributing to student success as the PTE grows and establishes its position in the educational marketplace. 13

14 1.6 How effective are governance and management in supporting educational achievement? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Good. Business planning at the PTE is useful and reflects the organisation s scale, goals and strategy. Although initial enrolments, and some aspects of international recruitment, were slower than expected, the directors have shown good leadership and have responded by making changes to both marketing and staffing. RBMC is building a credible reputation with students and local businesses which matches their vision. As a result, student numbers are increasing after a disappointing start, and a growing network of graduates are working in relevant sectors following study. As noted above, current facilities are modern and appropriate, but are at full capacity. Since opening, the PTE has invested in the provision of controlled access to Wi-Fi by students for their own devices and has a suite of modern personal computers available as required. The evaluators were provided with evidence that RBMC provided valuable services to some students whose education was disrupted by the closure of a PTE in Christchurch. Succession planning and well-advanced plans to increase teaching capacity align with growth in student numbers and business plan goals. This also supports sector strategies of increasing the international student presence in Christchurch. The PTE recently participated in an Education New Zealand focus group, and the academic director s contract work with an industry training organisation and as a monitor with NZQA also serves to connect the PTE with the wider tertiary education sector and current practice. The PTE also has functional and active relationships with a range of people and organisations in business, tertiary education and management sectors across the community, adding value to the PTE s knowledge and strategy. RBMC is a relatively new PTE which has competent leadership, clearly documented goals and the necessary resources in place to support ongoing educational delivery. Although it is too early to demonstrate patterns of achievement, retention of students is strong, course and qualification achievement is occurring, and completion rates are steadily improving as students complete their studies. As the PTE consolidates its position it will need to leverage both internal expertise and administrative tools and structures to develop an approach to selfassessment which is part of a coherent and comprehensive, organisation-wide approach. At the time of the EER, the PTE had the fundamental tools and processes in place and was using them appropriately to support student achievement. 14

15 Focus Areas This section reports significant findings in each focus area, not already covered in Part Focus area: Governance, management and strategy The rating in this focus area for educational performance is Good. The rating for capability in self-assessment for this focus area is Good. 2.2 Focus area: Business Diploma Programmes The rating in this focus area for educational performance is Good. The rating for capability in self-assessment for this focus area is Good. 15

16 Recommendations NZQA recommends that Retail Business and Management College: As stated under the statement of confidence on capability in selfassessment, RBMC will need to develop a more comprehensive and systematic approach to aggregating and analysing performance at, for example, qualification level and by cohort. Investigate options for larger or extended premises. 16

17 Appendix Regulatory basis for external evaluation and review External evaluation and review is conducted according to the External Evaluation and Review (EER) Rules 2013, which are made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. Self-assessment and participation in external evaluation and review are requirements for maintaining accreditation to provide an approved programme for all TEOs other than universities. The requirements are set through the NZQF Programme Approval and Accreditation Rules 2013, which are also made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. In addition, the Private Training Establishment Registration Rules 2013 require registered private training establishments to undertake self-assessment and participate in external evaluation and review, in accordance with the External Evaluation and Review Rules (EER) 2013, as a condition of maintaining registration. The Private Training Establishment Registration Rules 2013 are also made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. NZQA is responsible for ensuring non-university TEOs continue to comply with the rules after the initial granting of approval and accreditation of programmes and/or registration. The New Zealand Vice-Chancellors Committee (NZVCC) has statutory responsibility for compliance by universities. This report reflects the findings and conclusions of the external evaluation and review process, conducted according to the External Evaluation and Review (EER) Rules The report identifies strengths and areas for improvement in terms of the organisation s educational performance and capability in self-assessment. External evaluation and review reports are one contributing piece of information in determining future funding decisions where the organisation is a funded TEO subject to an investment plan agreed with the Tertiary Education Commission. External evaluation and review reports are public information and are available from the NZQA website ( The External Evaluation and Review (EER) Rules 2013 are available at while information about the conduct and methodology for external evaluation and review can be found at 17

18 NZQA Ph E qaadmin@nzqa.govt.nz 18