Report of External Evaluation and Review

Transcription

1 Report of External Evaluation and Review Gilligan Business School Limited Confident in educational performance Confident in capability in self-assessment Date of report: 17 December 2014

2 Contents Purpose of this Report... 3 Introduction TEO in context Scope of external evaluation and review Conduct of external evaluation and review... 5 Summary of Results... 6 Findings... 7 Recommendations Appendix MoE Number: 7495 NZQA Reference: C15993 Date of EER visit: 11 September

3 Purpose of this Report The purpose of this external evaluation and review report is to provide a public statement about the Tertiary Education Organisation s (TEO) educational performance and capability in self-assessment. It forms part of the accountability process required by Government to inform investors, the public, students, prospective students, communities, employers, and other interested parties. It is also intended to be used by the TEO itself for quality improvement purposes. Introduction 1. TEO in context Name of TEO: Type: Location: Delivery sites: Gilligan Business School Limited Private training establishment (PTE) Level 4, Corner Crowhurst and Kent Streets, Newmarket, Auckland Distance learning only currently no face-to-face delivery First registered: 17 May 2002 Courses currently delivered: Small Business Accounting Office Skills MYOB Accounting Course Accounting for Busy Managers Starting and Running Your Own Successful Business Code of Practice signatory? Number of students: Number of staff: Scope of active accreditation: Distinctive characteristics: Yes Approximately 100 to 200 learners each year Two one director and one manager Small Business Accounting is accredited by NZQA as an approved training scheme Gilligan Business School (Gilligan) is a family business, managed by the owner/founding director (the husband) and the manager (the wife). The Gilligan family runs a chartered accounting firm in 3

4 Auckland. Over the years, the owner/founding director developed notes for clients of the accountancy business to aid their understanding in accounting and this formed the foundation of the courses offered at Gilligan. The general public purchase these sets of notes online as reference materials and, as part of the purchase, are given the option to submit assignments (which are included in the materials) by post. If a client chooses to submit the first assignment, they become a learner and are entitled to continuous academic support for a year without additional costs. An end-of-course assessment is sent to the client by post. A learner receives a certificate when they complete the course: i.e. when all assignments and the final assessment are submitted and passed satisfactorily. In recent years, Gilligan recalls only one client of Māori descent. Recent significant changes: Previous quality assurance history: Other: N/A The previous EER was conducted in NZQA was Confident in both the PTE s educational performance and capability in self-assessment. Gilligan receives no external funding support from government agencies, other than through referrals from the Ministry of Social Development (one learner referred in 2013). 2. Scope of external evaluation and review The scope of this EER included the following mandatory focus areas: Governance, management and strategy The other focus area selected was: Distance learning All courses are delivered by distance, so this focus area includes all operations at Gilligan. While Gilligan is a signatory to the Code of Practice for the Pastoral Care of International Students, given that all courses are intended to be completed in 12 weeks or less, international students was not selected as a focus area. 4

5 3. Conduct of external evaluation and review All external evaluation and reviews are conducted in accordance with NZQA s published policies and procedures. The methodology used is described fully in the web document Policy and Guidelines for the Conduct of External Evaluation and Review available at: The TEO has an opportunity to comment on the accuracy of this report, and any submissions received are fully considered by NZQA before finalising the report. Self-assessment materials were delivered to NZQA in a timely fashion to inform the scoping of this EER. Before the scope of the evaluation was finalised, the lead evaluator met with the owner/founding director of Gilligan in person. They agreed on the EER focus areas and the logistics of the on-site visit. The evaluation team comprised two evaluators. The on-site visit was conducted over three hours. The evaluators held interviews with the owner/founding director and manager and sampled a range of documents. The evaluation team also contacted a number of stakeholders by phone after the on-site visit. 5

6 Summary of Results Statements of confidence on educational performance and on capability in self-assessment NZQA is Confident in the educational performance of Gilligan Business School Limited. NZQA is Confident in the capability in self-assessment of Gilligan Business School Limited. NZQA expresses the above statements of confidence for the following reasons: The Gilligan family maintains the necessary systems and processes to ensure the reference materials available for purchase online are accurate and relevant (Findings 1.3). Clients acquire up-to-date accounting and/or office management skills (Findings 1.1), improve financial literacy and, in some instances, anecdotal evidence shows some learners are equipped with the knowledge and confidence to re-enter the workforce (Findings 1.2). Clients who elect to become learners are very well supported when seeking assistance (Findings 1.5). Gilligan Business School operates a simple, commercially viable business model (Findings 1.6). Gilligan Business School has a rather unique operating model. Clients may choose to purchase the materials simply for reference, without becoming a learner. Those who become learners are well supported and generally complete. The selfassessment system in operation is minimal and reflects what is required to operate the business efficiently under the current model. Given the special characteristics and operating context of the school, NZQA considers Gilligan meets the expectations of a private training establishment within the evaluative quality assurance framework on this occasion. 6

7 Findings How well do learners achieve? The rating for performance in relation to this key evaluation question is Adequate. The rating for capability in self-assessment for this key evaluation question is Adequate. Clients who purchase the reference materials and choose to submit assignments achieve and receive valid recognition (a certificate) on completion. As a result of recent feedback from clients, Gilligan no longer actively prompts all clients to submit assignments, as some clients make the purchase simply for the purpose of obtaining the materials as reference. However, once the first assignment is submitted, Gilligan continuously track the progression of those clients when they become learners. The evaluation team examined the information systems at Gilligan. Gilligan conducts no formal analysis, but given the rather unique business model and the fact that Gilligan receives no government funding, the self-assessment system is deemed to meet the minimum expectations of NZQA. 1.2 What is the value of the outcomes for key stakeholders, including learners? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Adequate. The purchase cost of reference materials is low all courses are priced at under $500 + GST (2014 pricing). Clients acquire knowledge in accounting and/or office management skills. There is anecdotal evidence that clients/learners improve their financial literacy. Some gain the confidence and skills to rejoin the workforce after a significant break from employment. The evaluation team consider these outcomes are of some value. Gilligan collects some clients demographic information through voluntary return of questionnaires. While no detailed analysis is conducted, Gilligan was able to demonstrate that many clients were either financially supported by employers or referred to Gilligan by word-of-mouth, both indicators of the value of outcomes. 1 The findings in this report are derived using a standard process and are based on a targeted sample of the organisation s activities. 7

8 1.3 How well do programmes and activities match the needs of learners and other stakeholders? The rating for performance in relation to this key evaluation question is Adequate. The rating for capability in self-assessment for this key evaluation question is Poor. The reference materials on sale are reviewed at least annually and revised as required. For example, recent law changes in minimum wage and parental leave entitlements are reflected in the latest version of the materials. Anecdotal evidence, such as word-of-mouth referrals and return business, suggest the material s content more or less meets the demand and expectations of clients and their employers. The owner/founding director is a respected chartered accountant and understands the changing environment because of his connections within the sector. The flexibility to learn at one s own pace suits learners. Gilligan collects demographic data from clients and, when they become a learner, collects further information on their study objectives. This data forms the basis of a better understanding of learners needs, but the school is not currently using this opportunity. However, NZQA considers Gilligan meets the minimum performance expectations under this key evaluation question, given the operating context and the accuracy and relevancy of contents of the materials. Self-assessment can be strengthened by having periodic external input to review and verify the relevance of reference materials (effectively the course contents). 1.4 How effective is the teaching? The rating for performance in relation to this key evaluation question is Adequate. The rating for capability in self-assessment for this key evaluation question is Poor. The evaluation team examined samples of the reference materials and considered them clear, layman friendly, relevant, up-to-date and of high quality. The school advises the materials were derived from many decades of successful teaching practices, as Gilligan previously operated face-to-face classroom-based accounting courses. The evaluation team concurs with this assessment. However, the school should still have a review mechanism to verify the effectiveness of materials. The evaluation team also examined samples of assessments (both formative and summative). Because of the subject, all assessments have model answers and, as such, the grading system is valid. Feedback is given to learners when assessments are returned, although the qualitative feedback is not recorded by Gilligan. Gilligan conducts no formal analysis on teaching effectiveness, as learning is by distance, self-paced, and optional. 1.5 How well are learners guided and supported? The rating for performance in relation to this key evaluation question is Adequate. 8

9 The rating for capability in self-assessment for this key evaluation question is Adequate. Clients who choose to become a learner are very well supported when they seek assistance. Help is offered through an 0800 number during office hours. There is a log of communications with learners. The evaluation team sighted samples of assistance offered to learners and is highly satisfied with the responsive, caring, helpful nature of the support offered to learners. Gilligan meets the standards and practices expected of a distance learning provider. 1.6 How effective are governance and management in supporting educational achievement? The rating for performance in relation to this key evaluation question is Adequate. The rating for capability in self-assessment for this key evaluation question is Adequate. Gilligan operates a commercially viable business model, proven to be financially successful over many years. The current model is perfectly legal and ethical, requires minimal input in maintenance, and serves as a continuous income flow to the Gilligan family business. The family has aspirations to improve the financial literacy of the general public and the courses reflect such aspirations at the right level with the appropriate tone, delivery mode and style. There is a clear purpose and direction for the business. The current ownership is planning retirement within the next three years and is progressively grooming the next generation of the family to eventually inherit the business. The evaluation team is satisfied that the school has an appropriate business continuity and succession plan in place. Self-assessment at Gilligan operates at the bare minimum level and, on this occasion, is deemed to be sufficient for a school with this operating context and the relatively low risk nature of a short course. However, Gilligan must keep in mind that NZQA s expectation of private training establishments evolves over time and continuous improvement may be expected in the future. 9

10 Focus Areas This section reports significant findings in each focus area, not already covered in Part Focus area: Governance, management and strategy The rating in this focus area for educational performance is Adequate. The rating for capability in self-assessment for this focus area is Adequate. 2.2 Focus area: Distance learning The rating in this focus area for educational performance is Adequate. The rating for capability in self-assessment for this focus area is Adequate. 10

11 Recommendations NZQA recommends that Gilligan Business School: Review and enhance its self-assessment systems if it plans to expand and/or depart from its current operating model. Maintain a collated, summarised, easy-to-refer-to record of key aspects of its operations (such as the number of graduates per calendar year, the number of active learners and their progress) to better meet the expectations under the evaluative quality assurance framework, if the school continues to operate the current business model. Consider engaging some external input on a periodic basis to verify the accuracy and relevance of reference materials/course contents. Build capability in the understanding of the evaluative quality assurance framework, as well as the evolving expectations NZQA has of private training establishments. 11

12 Appendix Regulatory basis for external evaluation and review External evaluation and review is conducted according to the External Evaluation and Review (EER) Rules 2013, which are made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. Self-assessment and participation in external evaluation and review are requirements for maintaining accreditation to provide an approved programme for all TEOs other than universities. The requirements are set through the NZQF Programme Approval and Accreditation Rules 2013, which are also made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. In addition, the Private Training Establishment Registration Rules 2013 require registered private training establishments to undertake self-assessment and participate in external evaluation and review, in accordance with the External Evaluation and Review Rules (EER) 2013, as a condition of maintaining registration. The Private Training Establishment Registration Rules 2013 are also made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. NZQA is responsible for ensuring non-university TEOs continue to comply with the rules after the initial granting of approval and accreditation of programmes and/or registration. The New Zealand Vice-Chancellors Committee (NZVCC) has statutory responsibility for compliance by universities. This report reflects the findings and conclusions of the external evaluation and review process, conducted according to the External Evaluation and Review (EER) Rules The report identifies strengths and areas for improvement in terms of the organisation s educational performance and capability in self-assessment. External evaluation and review reports are one contributing piece of information in determining future funding decisions where the organisation is a funded TEO subject to an investment plan agreed with the Tertiary Education Commission. External evaluation and review reports are public information and are available from the NZQA website ( The External Evaluation and Review (EER) Rules 2013 are available at while information about the conduct and methodology for external evaluation and review can be found at NZQA Ph E qaadmin@nzqa.govt.nz 12