Report of External Evaluation and Review

Transcription

1 Report of External Evaluation and Review New Zealand College of Early Childhood Education Limited Highly Confident in educational performance Highly Confident in capability in self-assessment Date of report: 3 September 2014

2 Contents Purpose of this Report... 3 Introduction TEO in context Scope of external evaluation and review Conduct of external evaluation and review... 5 Summary of Results... 6 Findings... 8 Recommendations Appendix MoE Number: 7137 NZQA Reference: C15124 Dates of EER visit: 18 and 19 June

3 Purpose of this Report The purpose of this external evaluation and review report is to provide a public statement about the Tertiary Education Organisation s (TEO) educational performance and capability in self-assessment. It forms part of the accountability process required by Government to inform investors, the public, students, prospective students, communities, employers, and other interested parties. It is also intended to be used by the TEO itself for quality improvement purposes. Introduction 1. TEO in context Name of TEO: Type: Location: Delivery sites: New Zealand College of Early Childhood Education Limited (NZCECE) Private training establishment (PTE) 114 Sawyers Arms Road, Christchurch As above First registered: 25 October 2011 Courses currently delivered: Bachelor of Teaching Early Childhood Education (Leadership) (Level 7) Diploma in Teaching (Early Childhood Education) (Level 7) (being phased out) Diploma in Leadership (Early Childhood Education) (Level 7) Certificate in Leadership (Early Childhood Education) (Level 7) New Zealand Certificate in Nanny Education (Level 5) Code of Practice signatory: Number of students: Yes Domestic: 346 currently enrolled (11 per cent Māori; 2 per cent Pasifika; three males) International: two (Canada, Singapore) Number of staff: 22 full-time equivalents 3

4 Scope of active accreditation: Distinctive characteristics: nd=0 Students study both by distance learning (fieldbased at early childhood education centres) and by attending classes on campus. Recent significant changes: A change in ownership occurred in NZCECE was previously known as ABC Developmental Learning Centres (NZ) Limited, trading as New Zealand College of Early Childhood Education. The chief executive and the senior leadership team remained the same. However, the chief executive has now moved on and the operations leader is acting in his place. Staff and student numbers have remained consistent. NZCECE relocated to a new site in 2012 following the 2011 Christchurch earthquakes. The Bachelor of Teaching Early Childhood Education (Leadership) was introduced in Previous quality assurance history: Other: At the previous external evaluation and review (EER), NZQA was Highly Confident in the educational performance and capability in selfassessment of ABC Development Learning Centres (NZ) Limited, the former entity. NZCECE is phasing out the Diploma in Teaching (ECE) from The PTE is looking at providing short courses for Chinese student teachers. 2. Scope of external evaluation and review The focus areas for the EER were selected using the college s self-assessment summary, annual plan 2013 and 2014, and discussion with the chief executive. The focus areas were: Governance, management and strategy (mandatory) Bachelor of Teaching Early Childhood Education (Leadership) (Level 7). This was chosen because it is the highest-level qualification offered and the newest 4

5 programme with 48 students in their first year. The programme takes over from the previous Diploma in Teaching (ECE). Certificate in Leadership (Early Childhood Education) (Level 7), field-based delivery only. This programme was chosen because it has been delivered since 2010 and leads to the Diploma in Leadership (Early Childhood Education). This field-based programme has 13 students enrolled. The Diploma in Teaching (ECE) was not selected because it is being phased out and is being replaced by the Bachelor s programme. 3. Conduct of external evaluation and review All external evaluation and reviews are conducted in accordance with NZQA s published policies and procedures. The methodology used is described fully in the web document Policy and Guidelines for the Conduct of External Evaluation and Review available at: The TEO has an opportunity to comment on the accuracy of this report, and any submissions received are fully considered by NZQA before finalising the report. A team of two evaluators conducted the EER over two days on site. The team spoke with the senior leadership team, board of studies, degree teaching staff, student support advisors, first-year degree students, certificate in leadership students and their tutors, research mentor, and advisory group members. Supporting documents reviewed on site included internal moderation reports for both programmes selected, programme development, research planning, analysis of student and employer feedback, and the organisation s self-assessment processes. 5

6 Summary of Results Statement of confidence on educational performance NZQA is Highly Confident in the educational performance of New Zealand College of Early Childhood Education Limited. The reasons for this confidence rating can be summarised as follows: Overall, a high number of students are achieving qualifications each year. The organisation s analysis of outcomes shows a high retention within programmes, with 86 per cent progressing to higher levels within NZCECE and gaining qualifications. Although it is too early to identify the success of the degree programme introduced this year (2014), the previous Diploma in Teaching (ECE), which the degree replaces, shows high employment outcomes, with all 2013 graduates employed in the sector at the time of this EER. NZCECE is demonstrating responsiveness to Māori by using input from Māori stakeholders into programmes and developing appropriate support structures. Achievement for Māori is comparable to non-māori achievement and is better for some levels. A strong matching of stakeholder needs is evident in the comprehensive information gathered by the organisation. This information is used to develop relevant programmes that are meeting employers and students needs for trained early childhood centre teachers who understand the Te Whāriki curriculum and are capable of undertaking leadership roles in early childhood education centres. Students are well supported with online and classroom learning resources along with comprehensive teaching support to achieve their individual goals. There is consistent feedback from students and early childhood centre leaders/teachers ( associates ) confirming that students and graduates are applying the skills and knowledge gained to their roles at early childhood centres. Teaching is effective and staff are well qualified and experienced to perform their roles. This is reinforced by the broad base of research activity within the college. Staff members are supported in their research activities by the research committee and other staff, as well as a research mentor. There is evidence of staff incorporating research outcomes into programmes and the application of learning into their teaching practice. Overall, the organisation has a clear direction and provides the relevant resources needed to achieve its goals. 6

7 Statement of confidence on capability in self-assessment NZQA is Highly Confident in the capability in self-assessment of New Zealand College of Early Childhood Education Limited. The reasons for this confidence rating can be summarised as follows: NZCECE demonstrated that it carries out authentic analysis to help understand the value of its training and how it can maintain the benefit of the outcomes for the students and the sector in general. This analysis is based on sound systems for self-assessment using a range of inputs from external education specialists, early childhood centres (associates), students and graduates. Staff take advantage of the many opportunities to review feedback to help improve programme delivery and provide support where required. The ability of the organisation to respond well to major changes for example relocation following the Christchurch earthquakes, and changes to government policies for early childhood educator upskilling requirements demonstrates the organisation s thorough planning and assessment of the environment it operates in. Another example of effective self-assessment was the identified need three years ago for a degree programme within the Canterbury region, which required staff members to engage in research. The degree programme has been successfully implemented and the organisation has an embedded collaborative research culture. Student satisfaction is monitored around teaching, content and support to cater to their needs. An example of where the organisation takes the time to understand its students and the sector is senior leadership team face-to-face interviews with employers to correlate the questions given to graduates about the nature of the programme and its value to ensure that students are receiving relevant and up-to-date training. Overall, NZCECE uses systematic assessment and many opportunities to gather information about its courses in order to make improvements. The organisation has effective feedback loops with industry and students to let them know their voices are heard, and actions are taken to address any issues that may arise. The structure of the system provides NZQA with confidence that these activities will continue with the change of chief executive. 7

8 Findings How well do learners achieve? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. Student outcomes are consistently at the national median rate or above for course completion and qualification achievement over consecutive years. In 2013, overall course completion was 80 per cent and qualification achievement was 90 per cent. The 2012 rates were 89 per cent for course completion against the national median of 86 per cent. Eleven per cent of the student population is Māori, and the achievement for this group of student is comparable to the overall outcomes of all students. The organisation records detailed information to analyse Māori and Pasifika educational progress and achievement, and in some programmes the data shows that Pasifika and Māori course completions exceed the average. NZCECE measures its overall performance in a number of ways. Overall educational performance is measured against the educational performance indicators. Overall, NZCECE s qualification achievement results remain consistently higher than TEC educational performance indicator (EPI) targets. Table 1 shows 2013 results against previous years and 2013 TEC targets. Table 1. Qualification and course completions, TEC EPI 2011 TEC EPI 2012 TEC EPI 2013 EPI investment plan prediction 2013 actual achievement Qualification completion: levels % 86% 92% 86% 90% Qualification completion: levels % 77% 75% Overall qualification completion 82% 80% 79% 81% 90% Course completion: levels % 86% 92% 86% 89% Course completion: levels % 81% 87% - - Overall course completion 86% 82% 89% 83% 80% 1 The findings in this report are derived using a standard process and are based on a targeted sample of the organisation s activities. 2 There were no targets for the Bachelor s programme as it had not been delivered. 8

9 The lower overall course completion rate of 80 per cent for 2013 is due to only 75 per cent of students completing the Certificate in Leadership via field-based learning each year since the programme began in The success rates for this programme are lower than other programmes due to an initial cohort of ABC Developmental Learning Centre teaching staff withdrawing when the qualification was no longer a requirement of their employment and there was no other motivation to continue. The new Bachelor s programme has not had any graduates to date, as it only started in 2014, but already retention figures show there have been no withdrawals from the programme in comparison with previous years for other programmes at similar levels. This can be attributed to the initial selection process as well as the support provided to the students. Preliminary results show that the Bachelor s students are all passing their assignments. The qualification achievement for this programme can be extrapolated from the Diploma in Teaching (ECE) (which the degree programme replaces), which has high outcomes for students that complete stage 1 and go on to complete stages 2 and 3. The organisation has very good analysis of individual programme performance at different stages to show how the students progress in their course of study. The organisation compares the Diploma in Teaching (ECE) on-campus student results with field-based student results from previous years to understand how it can help improve student success (see below). The stage 1 on-campus completion results for 2013, which also include students studying towards the New Zealand Certificate in Nanny Education, were 87 per cent, showing similarly high outcomes to previous years. The field-based students have a slightly lower rate of completion when using this calculation because their course of study spans a longer period of time. However, there is evidence that the completion rate for field-based students is increasing, partly attributable to the fieldbased focused student support groups established in 2013 to cater to student needs more specifically. The organisation s own analysis of achievement against targets is detailed and highlights the shortcomings of the measures used for programme funding. For example, the educational performance indicators for the level 5 and 6 qualifications include the nanny certificate as well as the first and second years of the diploma, which have different exit points. In addition, the level 7 and 8 bands capture the field-based students enrolled in the diploma who have longer to complete along with the stage 3 on-campus students, who have less time to complete. This has the effect of reducing the organisation s overall educational performance indicators. In summary, NZCECE has developed a robust system for monitoring and analysing results of individual programmes at each level to successfully identify trends and implement ways to improve student achievement. The effectiveness of the selfassessment is also demonstrated by the decision to reduce the number of funded places in 2014 based on analysis of the market and government targets. 9

10 1.2 What is the value of the outcomes for key stakeholders, including learners? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. Graduate destination data shows that 30 out of 32 of the 2013 total enrolments in the Diploma in Teaching (which the Bachelor s degree is replacing) are employed in the sector. The value to students on each of the programmes is the ability for them to acquire skills and knowledge that they then put to use in their teaching roles. This is confirmed from the formal and informal feedback from associates reporting that the students on practicums were well prepared, brought in new ideas, and were confident to share their knowledge in a respectful way. Another benefit is that young people take on positions of responsibility in leading an education centre, which is useful because the evaluation team routinely heard from staff and advisory group members that recent graduates often become centre leaders early on in their employment. There is also evidence that the training helps current centre leaders make improvements to their leadership practice. Many students spoken to say they chose to study at NZCECE based on friend and family recommendations, as well as their preference for small class sizes. Supporting this is the similar number of enrolments in the newly introduced Bachelor s programme this year in comparison with a similar university programme (37 and 43 respectively). The introduction of a degree programme has taken staff out into the community since development began two years ago. This is because staff are encouraged to present their research at external conferences and to represent the college on other educational committees. For example, the research leader is also the secretary of the Canterbury branch of the New Zealand Educational Administration and Leadership Society (an organisation that promotes leadership in education). Students formal and informal feedback was consistently highly positive about the skills and knowledge gained on practicum in the centres. NZCECE gathers student feedback at the end of each topic and at the end of the programme about the course and tutors. A follow-up online survey is used to gather information about the value of the course six months after graduation. Students and graduates consistently highly rate the learning about Te Whāriki (early childhood education curriculum), and knowledge and understanding of the early childhood centre context and regulations (100 per cent satisfaction). Overall, NZCECE demonstrated an authentic analysis used to understand the value of the qualifications and how the organisation could improve the outcomes for the students and their employers. The chief executive and senior staff also follow up 10

11 with centre employers (associates) that take students on practicums to confirm that the students have the appropriate skills and to identify how they could improve the value to employers. 1.3 How well do programmes and activities match the needs of learners and other stakeholders? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. NZCECE demonstrates that it values external feedback and has developed a highlevel understanding of stakeholders needs as a result. Feedback on the relevance of programmes and needs of industry is gathered from other PTE leaders who are on the board of studies and through extensive engagement with advisory group members ranging from primary school principals to centre leaders and te rūnanga representatives. The organisation also reviews census data and conducts market research to understand regional population needs. This has led to a suite of programmes that recognise and are relevant to the needs of students and employers in both the Canterbury region and New Zealand as a whole. Feedback from the advisory group members spoken to during the EER visit reported that meetings were very well organised and information was well communicated by the organisation, and the chief executive always provides a follow-up to any issues raised. There is clear evidence that actions are taken to incorporate advisory group feedback into the programmes and activities. The representative of Te Rūnanga o Ngāi Tahu (a local iwi) said there is a genuine effort by NZCECE to accept input to help with incorporating tikanga into the programme. NZCECE has also recently added a Pasifika person to the staff to encourage more Pasifika early childhood educator participation. A community of collaboration within the staff internally and externally for researchbased projects means that the degree programme uses the best of the resources available to ensure the programmes contain up-to-date resources and use appropriate activities. The Bachelor of Teaching was developed for teachers working in early childhood centres to enhance their employment chances in a regulated employment market. The Bachelor s programme includes practical activities as well as more theory than the previous diploma, in response to employer and student feedback. Students complete three to six-week practicums at early childhood education centres to gain a broad range of experience. The length of the final practicum in the Diploma in Teaching (ECE) was changed from four weeks to six weeks following feedback from associates, who said the longer time provided better opportunities for learning. NZCECE has provided an upgrade one-year programme to graduates of the 11

12 Diploma in Teaching (ECE) to enable them to achieve the newly introduced degree, thereby ensuring their qualifications remain relevant. The Certificate in Leadership was developed to meet the requirement for associates to provide professional development opportunities for staff. The aim of the programme is to develop leadership skills in practice, and it therefore includes activities to develop pedagogy, working in teams, and leading change. The organisation has retained the nanny certificate as a foundation qualification for early childhood education, which can lead to the level 7 degree. The organisation upgraded its technology since relocation to a new campus in Students have access to Wi-Fi, and online access to relevant early childhood education journals, reference material and course notes. The online portal also enables the tutors to monitor student activity on forums and to provide feedback. Students can submit assignments online, which makes the programme more accessible for people who are working while on the field-based programmes. The organisation also provides plagiarism software free to students. This not only enables tutors to detect plagiarism, but also gives students help with referencing for assignments. Each programme has a curriculum strand leader selected from the teaching staff, whose role is to review each programme using student evaluations, tutor reports, moderation reports and associate feedback. The curriculum strand leader is responsible for ensuring the curriculum meets all New Zealand Teachers Council requirements. NZCECE is also involved with a local secondary school to offer students the option of completing achievement standards towards University Entrance requirements. Some students are assessed using an online literacy and numeracy tool to identify any learning needs. NZCECE is also involved in the Mandatory Review of Qualifications (MRoQ) for the early childhood education sector. This demonstrates the commitment of NZCECE to fully understanding the sector and providing programmes that meet stakeholders needs. 1.4 How effective is the teaching? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. Students are highly engaged with the learning, with the tutors using a tuakana-teina model, where the students learn from the tutors, but tutors also learn from students practical experiences in the centres. To encourage this relationship, the organisation has developed tutor-led student groups containing students from across the on-campus programmes and learning stages. This has provided students with further opportunities to learn from one another. In 2014, separate 12

13 groups were established for field-based students in recognition of their specific needs, and this has been successful in improving field-based learner achievement. Students have control over the type of projects they do and develop an appropriate learning plan to suit their individual needs. Evidence of teaching effectiveness is seen in the results as well as in the positive feedback from students. The tutors qualifications and experience enable them to cater to a range of students learning styles. Staff members develop their own professional development plans according to their individual learning goals and are encouraged to complete them with time off and financial support from the organisation. All teaching plans are shared among the tutors. Tutors teach both on-campus and field-based students. Teaching effectiveness and assessment and moderation outcomes are discussed to ensure consistency. NZCECE has identified the need for staff to develop their knowledge and skills in tikanga and te reo as a result of advisory group feedback, which has been supported by staff. Tutors are also empowered to manage their online course pages and there is information technology training available on a regular basis. Three years ago, when NZCECE decided to develop a degree programme, a research project was led by an external research specialist to help embed a culture of research within the college and to support staff with the transition. This project has been very successful, with a number of staff applying their research in practice and sharing their findings with the other tutors. The research committee ensures that there are sufficient resources available for staff conducting research and presenting at external education conferences. This support has boosted staff confidence in carrying out research, and there is notable evidence of staff members completing their professional development plans. However, there is still further work needed to increase the number of published research papers. Tutors monitor their own performance using the regular course evaluations, as well as gaining feedback at annual performance reviews and twice-yearly classroom peer observations. The tutors also review student feedback to revise the course to ensure their needs are being met. The organisation has a standardised moderation process across all programmes to ensure there is consistency in assessment decisions and that outcomes are robust. All assessment activities are moderated by the tutors prior to use and reviewed annually by the curriculum strand leaders. Internal moderation of assessment decisions occurs at tutor level on a regular basis usually after each assignment. Tutors review each other s decisions using marking guidelines to ensure the assessment decisions meet the criteria. The students then receive the moderated assignments. An external moderator writes a report on the tutors moderation for each programme, with relevant feedback, and academic leaders then develop an action plan to address any issues raised. External moderation reports are also used by the curriculum strand leaders when reviewing programmes. 13

14 Overall, NZCECE has been successful in developing a research culture within the organisation. This has been a result of the strong guidance and support from the senior leadership team, the ability for staff to empower themselves by developing relevant research plans to suit their needs, and the opportunities for sharing research and practices across the organisation. This is all contributing to the quality of teaching to ensure robust outcomes for the students. 1.5 How well are learners guided and supported? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. Students receive a high level of support from the tutors and the student services staff. The students feel welcomed into the college, and the mix of students from each course in student groups provides a network of support from other students for advice and to help them navigate their courses. NZCECE has identified a need to establish a Māori student support group to provide additional, culturally relevant support. The students are able to choose their learning pathways, which helps them to keep track of their goals and stay focused. The small class sizes mean that tutors are able to identify whether anyone is struggling with assignments, and they can provide one-to-one support if needed. Often tutors recognise where someone is not engaging or needs support from monitoring students online activity. NZCECE employs a person to manage the practicum placements for the degree. (The certificate students are already in employment and therefore do not complete practicums.) Students and staff said this was a great resource as student were matched to centres according to their learning stage, and the associates were able to provide feedback on the students, which helps with the students learning and the selection of placements. Overall, NZCECE s analysis of student evaluations and graduate surveys shows that the support provided is at the appropriate level and contributes to successful student achievement of outcomes. 14

15 1.6 How effective are governance and management in supporting educational achievement? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. NZCECE sets a clear direction and guidance for staff. The senior leadership team has a strong understanding of the political and community contexts it operates within, which enables it to project resources and staffing capability to provide opportunities for students to achieve. The research group was created to provide support and foster research to enable the organisation to offer the degree. The management support is a key factor in the successful embedding of a research culture, and the organisation continues to support its staff research endeavours with the establishment of a research committee and access to a research mentor. The organisation has employed highly skilled and qualified people, including those with Master s or PhDs, who also provide mentoring to other staff. A major information technology upgrade in 2010 led to the provision of Wi-Fi and development of an online learning platform with links to the resources required by tutors and students. Student and tutor forum usage reports show that students are using the online forum and resources. NZCECE seeks independent, external advice from members of the board of studies about programme direction, mentoring and support to manage changes within the organisation, as well as to develop new markets to maintain viability in an everchanging environment. Programme evaluations and input from the advisory group are carefully considered and incorporated into programme development. The organisation uses student feedback as a broad indicator of what is happening and to make necessary changes. Student satisfaction around teaching, programme content and support is very high. Associates and staff members reported on the excellent communication from the organisation about the programmes and the effectiveness of feedback loops with the advisory committee. Any issues are followed up and reported back to the relevant groups. Staff have multiple opportunities for discussing outcomes and reviewing stakeholder feedback to improve programmes at teaching and learning meetings, tutor meetings, in student groups, and at staff meetings. The organisation demonstrated a high level of responsiveness to the Christchurch earthquakes. The aftermath of the earthquakes has affected all achievement targets since, as personal circumstances have changed and people have moved away and dropped out of courses. However, the organisation was able to respond quickly, providing continuity to on-campus and field-based students using the online resources. This was helped by the organisation s new information technology system and the storage of records off-site which it was able to access. The 15

16 organisation has raised standards and has not seen a decrease in levels of retention and achievement across all programmes in Overall, self-assessment processes reach across and into each programme area comprehensively. The systems in place enable sharing of information and feedback, which provides the senior leadership team with coherent, relevant information to inform their decision-making, which is contributing to the success of the students. 16

17 Focus Areas This section reports significant findings in each focus area, not already covered in Part Focus area: Governance, management and strategy The rating in this focus area for educational performance is Excellent. The rating for capability in self-assessment for this focus area is Excellent. 2.2 Focus area: Bachelor of Teaching Early Childhood Education (Leadership) (Level 7) The rating in this focus area for educational performance is Excellent. The rating for capability in self-assessment for this focus area is Excellent. 2.3 Focus area: Certificate in Leadership (Early Childhood Education) (Level 7) The rating in this focus area for educational performance is Excellent. The rating for capability in self-assessment for this focus area is Excellent. 17

18 Recommendations There are no recommendations arising from the external evaluation and review. 18

19 Appendix Regulatory basis for external evaluation and review External evaluation and review is conducted according to the External Evaluation and Review (EER) Rules 2013, which are made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. Self-assessment and participation in external evaluation and review are requirements for maintaining accreditation to provide an approved programme for all TEOs other than universities. The requirements are set through the NZQF Programme Approval and Accreditation Rules 2013, which are also made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. In addition, the Private Training Establishment Registration Rules 2013 require registered private training establishments to undertake self-assessment and participate in external evaluation and review, in accordance with the External Evaluation and Review Rules (EER) 2013, as a condition of maintaining registration. The Private Training Establishment Registration Rules 2013 are also made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. NZQA is responsible for ensuring non-university TEOs continue to comply with the rules after the initial granting of approval and accreditation of programmes and/or registration. The New Zealand Vice-Chancellors Committee (NZVCC) has statutory responsibility for compliance by universities. This report reflects the findings and conclusions of the external evaluation and review process, conducted according to the External Evaluation and Review (EER) Rules The report identifies strengths and areas for improvement in terms of the organisation s educational performance and capability in self-assessment. External evaluation and review reports are one contributing piece of information in determining future funding decisions where the organisation is a funded TEO subject to an investment plan agreed with the Tertiary Education Commission. External evaluation and review reports are public information and are available from the NZQA website ( The External Evaluation and Review (EER) Rules 2013 are available at while information about the conduct and methodology for external evaluation and review can be found at 19

20 NZQA Ph E qaadmin@nzqa.govt.nz 20