Professional Education Unit

Transcription

1 NORTHERN ARIZONA UNIVERSITY Professional Education Unit ASSESSMENT SYSTEM HANDBOOK 2011/2012

2 PROFESSIONAL EDUCATION UNIT ASSESSMENT SYSTEM HANDBOOK Table of Contents The Unit Assessment System Overview... 1 Unit Outcomes... 1 Unit Proficiencies... 2 The goals, values and constructs central in the Conceptual Framework lay the foundation that informs our candidate proficiencies. These proficiencies are:... 2 Table I Unit Assessments and Data Sources... 2 Alignment Between Unit Assessments and the Proficiencies They Measure... 3 Table 2. Unit Assessment Alignment Table... 4 Unit Assessment Administration and Data Management... 4 Definitions of Personnel Involved in the Assessment System... 4 Assessment #1: Candidate Licensure Exam Scores... 5 Assessment #2: Candidate Performance on Knowledge, Skills and Dispositions Course Assessments... 6 Assessment #3: Candidate Performance on Student Teaching/Internship... 6 Assessment #4: Effect on Student Learning... 7 Assessment #5: Surveys... 8 Assessment #6: Faculty and Clinical Faculty Qualifications Forms... 8 Assessment #7: Faculty and Clinical Faculty Evaluations... 9 Assessment #8: Institutional Data... 9 Analysis and Use of Assessment Data... 9 Use of Assessments to Monitor Candidates Use of Assessments to Manage and Improve Unit Operations Use of Assessments to Manage and Improve the Unit Assessment System Table 3 Unit Assessments and Data Administration Use of Assessments to Manage and Improve Programs Assessment Plans Annual Reports Table 4. Program Annual Schedule Program Changes and New Programs Seven-Year Schedule Appendices... i Appendix 1: Alignment Between Conceptual Framework Goals and Professional Standards... ii GOAL 1: To prepare professionals with the content knowledge necessary to function effectively in their professional role... ii Teacher Candidate Proficiencies... ii Other School Professionals Candidate Proficiencies... iii GOAL 2: To prepare professionals with the skills necessary to function effectively in their professional role, including the ability to effectively use technology and promote the education of diverse students... v Teacher Candidate Proficiencies... v

3 Other School Professionals Candidate Proficiencies... viii GOAL 3: To prepare professionals with the dispositions needed to be confident, open-minded, ethical, and empathic practitioners and believe all students can learn.... x Teacher Candidate Proficiencies... x Other School Professional Candidate Proficiencies... xiv Appendix 2: Unit Governance and Resources Data Assessment Table... xvii Appendix 3: Unit Operations Assessment Plan... xxii Appendix 4: Program Assessment Plan Template... xxvi Appendix 5: Data Discussion Table... xxxii Appendix 6: Annual Report Template... xxxv Appendix 7: Early Alert Form... xli Appendix 8: Accreditation Program Change Form... xlv

4 The Unit Assessment System Overview The purpose of the Assessment System of the Professional Education Unit (the unit) at Northern Arizona University is to collect, compile, analyze and maintain information in order to manage and improve candidate, unit, and program performance. Our assessment system is founded on the mission and goals of our Conceptual Framework, Arizona state standards for school professionals, and Specialized Professional Association standards. These together provide common unit outcomes that are assessed through candidate key assessments, surveys and Institutional data. The assessment system helps to ensure that candidates are ready to transition through stages of their program, and that our program completers have the professional knowledge, skills and dispositions outlined in our Conceptual Framework as well as in applicable State and Specialized Professional Association standards. An important aspect of developing competent professionals is the management of the unit s operations. The assessment system uses data from candidates course assessments as well as from survey data from candidates, alumni, employers, and P-12 partners to better ensure that unit operations facilitate candidates learning. Additionally, data from unit and Institutional databases provide information that are used to manage and improve unit operations. Unit Outcomes Our Unit outcomes for candidates are established from constructs in our Conceptual Framework and are aligned with State and Professional Association standards. The mission of the unit is to prepare competent and committed professionals who will make positive differences for children, young adults and others in schools. In order to accomplish this, the following goals have been established and are documented in our Conceptual Framework: 1. To prepare professionals with the content knowledge necessary to function effectively in their professional role, 2. To prepare professionals with the skills necessary to function effectively in their professional role, and 3. To prepare professionals with the dispositions needed to be confident, open-minded, ethical, and empathic practitioners. The Conceptual Framework identifies a core value of preparing professionals who are committed to diversity and who are prepared to serve peoples from rural Arizona communities. The importance of preparing professionals to integrate technology in their work so students learn from technology and with technology is a central construct in the Conceptual Framework. Every program in the unit has an assessment plan that incorporates these constructs. The Arizona Department of Education has developed professional standards for initial teachers, school administrators, school psychologists and school counselors. The programs in the unit that prepare professionals in these fields assess their candidates according to the specific standards for their respective fields of preparation as they transition through their program. Our candidates also demonstrate Specialized Professional Association standards through key assessments as they transition through their respective programs. The Specialized Professional Association standards fall into the general NCATE categories of: Content Knowledge, Pedagogical Knowledge and Skills, Professional Knowledge and Skills, Student Learning and Dispositions. These 1

5 broader categories are used by the unit to aggregate candidate data from disparate program assessments. The Interstate Teacher Assessment and Support Consortium standards are aligned with professional standards of our initial teacher preparation programs. Similarly, the National Board for Professional Teaching Standards are used to guide our advanced teacher education programs. The alignments between our Conceptual Framework Goals and professional standards for initial teacher candidates, advanced teacher candidates, school administrators and other school professionals are located in Appendix 1: Alignment Between Conceptual Framework Goals and Professional Standards. Unit Proficiencies The goals, values and constructs central in the Conceptual Framework lay the foundation that informs our candidate proficiencies. These proficiencies are: 1. School Professionals will have the content knowledge necessary to function effectively in their professional role; 2. School Professionals will have the professional knowledge necessary to function effectively in their professional role; 3. School Professionals will have the pedagogical and/or professional skills necessary to develop educational plans to promote student learning; 4. School Professionals will have the pedagogical and/or professional skills necessary to effectively implement plans and function in their professional role; 5. School Professionals will have the pedagogical and/or professional skills necessary to promote learning of diverse students; 6. School Professionals will have the pedagogical and/or professional skills necessary to effectively implement technology; 7. School Professionals will have the dispositions needed to be confident, open-minded, ethical, and empathic practitioners and believe all students can learn. As a unit, we assess candidates attainment of our unit proficiencies as well as the effectiveness of our unit operations. Our unit assessments and their data sources are listed in Table I: Unit Assessments and Data Sources. These assessments are discussed later in further detail. Table I Unit Assessments and Data Sources Assessment Data Source Assessment 1: Candidate State and national exams required for our initial teacher Licensure Exam Pass Rates candidates, principals, superintendents, school psychologists, school counselors, and speech pathologists in order for them to Assessment 2: Candidates Performance on Knowledge, Skills, and Dispositions Course Assessments Prior to Student Teaching/Internship be licensed. Key assessments in the areas of content knowledge, content pedagogical knowledge and skills, professional knowledge and skills, diversity, technology, and dispositions that are aligned with state, professional, and unit standards. Candidates are evaluated as not meeting a standard, meeting a standard, or being at the target level or exceeding the standard in their 2

6 Assessment 3: Candidate Performance on Student Teaching/Internship Assessments Assessment 4: Candidate Performance on Student Learning Assessments During Student Teaching/Internship. Assessment 5: Surveys Assessment 6: Faculty and Clinical Faculty Qualifications Forms Assessment 7: Faculty and Clinical Faculty Evaluations and Course Evaluations Assessment 8: Institutional Data Applicant, Enrollment, Completer and Faculty Data performances assignments. Checklist evaluations used during student teaching/internship. These are based on state and professional standards. Candidates are evaluated as not meeting a standard, meeting a standard, or being at the target level or exceeding standards in their student teaching/internship performance. Key assessments used during student teaching/internship/capstone course that measure candidates ability to enhance all students learning. For teacher candidates, this involves pre-assessing learner s knowledge/skills, designing instruction based on that preassessment, delivering the instruction, post-assessing students learning, and reflecting on their teaching. Student learning assessments for candidates in other school professional programs focus on the candidate s ability to create positive learning environments for students. Standard surveys completed on a regular basis as well as other surveys delivered on a one-time basis. The standard surveys include the Graduating Candidate Survey, the Alumni Survey, and the Employer Survey. Forms completed by faculty and faculty who mentor or supervise candidates in clinical settings. They measure faculty members professional preparation, degree, related work experience, professional development and involvement, scholarship and service to the profession. Evaluations of faculty and clinical faculty completed by peers and administrators. Evaluations of courses completed by candidates. Official counts of candidates who apply to, are enrolled in and complete programs in the Professional Education Unit, along with their campus and ethnic background. Official counts of full-time and part-time faculty in the Professional Education Unit, their tenure/tenure track status and ethnic background. Alignment Between Unit Assessments and the Proficiencies They Measure The unit assessments are specifically aligned with our unit proficiencies, and each unit proficiency is adequately assessed by multiple measures. This is illustrated in Table 2: Assessment Alignment Table. 3

7 Table 2. Unit Assessment Alignment Table Unit Assessments Unit Proficiencies and Unit Operations School Professionals will have the content knowledge necessary to function effectively in their professional role X X X 2. School Professionals will have the professional knowledge necessary to function effectively in their professional role X X X 3. School Professionals will have the pedagogical and/or professional skills necessary to develop educational plans to promote student learning x X X X 4. School Professionals will have the pedagogical and/or professional skills necessary to effectively implement plans and function in their professional role x X X X 5. School Professionals will have the pedagogical and/or professional skills necessary to promote learning of diverse students x X X X 6. School Professionals will have the pedagogical and/or professional skills necessary to effectively implement technology x X X X 7. School Professionals will have the dispositions needed to be confident, open-minded, ethical, and empathic practitioners and believe all students can learn x X X X Unit Operations X X X X x= formative assessments, X = summative assessments Unit Assessment Administration and Data Management The unit regularly gathers, aggregates, summarizes and analyzes assessment and evaluation data. As candidates progress through their program, they are monitored by their performance data. Faculty members summarize, analyze, and use candidate assessment data that have been loaded into the Assessment Library to make program improvements. These analyses and data driven improvements are included in programs annual reports. The NCATE Coordinator compiles these by unit categories. These along with survey and Institutional data are analyzed by the Coordinating Council and Head of the Unit and shared with appropriate faculty and administrators to make improvements in the unit and in unit operations. A detailed description of each of the eight assessments and the persons involved in collecting, compiling, aggregating, and disseminating the data are described below. The analysis and use of the data from the unit assessments are discussed in the following sections. Definitions of Personnel Involved in the Assessment System Program Faculty Members Faculty members responsible for teaching in a program. A program is a course of study and evaluation that has a unique assessment plan on file with the University s Office of Academic Assessment. Appointed Personnel - Faculty or staff members in a department or program that have been given the responsibility of a particular role in monitoring candidate performance through transition points. For initial teacher programs, these persons are the advisors in the College 4

8 of Education Student Services and secondary education faculty members who advise candidates in their programs. Lead Program Faculty Members Programs faculty members who have been designated as the one to access the candidate key assessment data, initiate program faculty discussions and analysis of the data, and complete the Annual Program Report. NCATE Coordinator The Individual assigned with coordinating assessment system components across the unit. This includes verifying that program changes retain integrity with regards to collecting data required by NCATE, the Arizona Department of Education, and the unit and gathering compiling unit assessment data and distributing it to appropriate entities for analysis. Director of Student Services The individual who oversees initial teacher certification program candidate advisement and verifies that initial teacher candidates meet the requirements for student teaching and program completion. This individual is also responsible for issuing Institutional Recommendations to program completers. Support Systems Analyst The individual responsible for managing the assessment and rubrics in TaskStream. This individual is also responsible for running data reports of candidates and aggregating them by program, degree and campus. The Support Systems Analyst uses these reports to populate the Assessment Library and distribute candidate records to those appointed to enter key assessment pass information into candidates advisement checklists. Head of the Unit The individual designated to provide leadership for the unit. This individual is responsible for overall administration and operation of the Assessment System. The Coordinating Council - The body that monitors and evaluates the unit s operations, governance, and qualified faculty. The Associate Deans from every college in which professional education programs are housed serve as the Coordinating Council. These colleges are: The College of Arts and Letters, the College of Education, the College of Health and Human Services, and the College of Engineering, Forestry, and Natural Sciences. The Director of the Center for Science Teaching and Learning represents the Associate Dean of the College of Engineering, Forestry, and Natural Sciences on the Council. This Council reports to the Head of the Professional Education Unit and to the Vice Provost for Academic Affairs. They also make recommendations to Deans, Department Chairs, and other entities involved with delivery of services to faculty and candidates. Assessment #1: Candidate Licensure Exam Scores The unit evaluates candidates passing rate on state and national licensure exams. Candidates in our initial programs as well as in our school administrator, counselor and psychology programs take professional licensure exams. In addition, candidates must pass state content knowledge proficiency exams in order to become teacher certified in all but two of our initial teacher preparation programs. Programs for which a content exam is not available substitute an additional content knowledge assessment for unit assessment purposes. Candidates are encouraged to take the content knowledge portion of the licensure exam prior to student teaching. They are encouraged to take the pedagogical and professional knowledge portion during or after they student teach. 5

9 The testing agency through which the licensure exams for the initial programs is conducted posts scores by program on a password-protected website, to which the NCATE Coordinator has access. The NCATE Coordinator downloads the scores after each test period (six per year) and sends them to the unit s testing resource center to be cleaned (non-nau scores removed), compiled, and recorded. At the end of the academic year, the testing agency sends an aggregated report of passing rates for each program and the unit overall. The Educational Leadership department chair receives the professional knowledge licensure scores for the school administrator program s candidates. The Educational Leadership administrative assistant cleans the data and compiles and records those scores. The School Counseling and School Psychology department chair receives licensure scores for candidates in those programs. Similar to the Educational Leadership programs, the administrative assistant cleans, compiles and records those data. Aggregated program data are reported by campus and included in the school administrator, counselor and psychologist annual reports. Assessment #2: Candidate Performance on Knowledge, Skills and Dispositions Course Assessments Candidate s performances on key assessments (signature assignments) are recorded in an electronic portfolio system. At the beginning of the spring semester, the unit s Support Systems Analyst compiles candidate performance data by assessment and places the reports in the unit s Assessment Library. This library comprises a folder for every department, in which a folder for every program within that department is housed. Each program folder contains sub-folders that coincide with program and unit proficiencies. Candidate performance on course assessments data are placed in folders The Assessment Library folders are as follows: 1. Licensure Exam Scores 2. Content Knowledge 3. Professional/Pedagogical Knowledge/Skills: Planning 4. Student Teaching/Internship/Capstone Course Evaluations 5. Effect on Student Learning 6. Professional Knowledge 7. Additional Pedagogical (teachers) or Professional (other school professionals) Knowledge/Skills 8. Diversity 9. Technology 10. Dispositions Lead program faculty members access their program s data, analyze and aggregate them, and include the aggregated sums in their annual report by Assessment Library category (see Appendix 6 for the report template). These reports are submitted by June 1. The NCATE Coordinator aggregates the data according to the Assessment Library categories from the annual reports by initial programs, advanced teacher programs, and other school professionals programs. Within each of these groups, the data are reported by degree and campus location. The NCATE Coordinator disseminates this information to the Coordinating Council as partial evidence of their evaluation of candidates knowledge, skills and dispositions of the unit, and diversity. Assessment #3: Candidate Performance on Student Teaching/Internship All candidates in a program that includes a student teaching or internship component are evaluated during this experience. Their performance data are analyzed by program faculty as well as aggregated and assessed by the unit. 6

10 The student teaching evaluation instrument includes elements that measure state teacher proficiency standards as well as disposition as defined by the unit. This evaluation is electronic and is completed by University Supervisors at the end of the student teaching experience. The NCATE Coordinator downloads the data from these evaluations and compiles and records them and uploads them into appropriate program Assessment Library folders to be included in programs annual reports. The Educational Psychology department chair gathers the internship evaluations for the school psychology and school counseling programs. Similarly, the Educational Leadership department chair gathers the internship evaluations for the school administrator programs. These are aggregated and reported in their annual reports. The NCATE Coordinator complies the student teaching/internship data from the annual reports and aggregates them by initial teacher education programs, advanced teacher education programs and other school professionals programs by degree and campus. The NCATE Coordinator then disseminates these data to the Coordinating Council to be used as partial evidence to assess whether the unit is meeting standards in the areas of candidates knowledge, skills and dispositions, fieldwork experience, and ability to enhance the learning of diverse students. Assessment #4: Effect on Student Learning All programs assess their candidates effect on student learning. Programs having a student teaching or internship component conduct a summative assessment of effect on student learning during this experience. Programs that do not have a student teaching or internship experience conduct a summative assessment during a capstone course. Initial candidates demonstrate they have a positive effect on student learning through a Candidate Work Sample. In this project they must evaluate the degree to which students learn as a result of their instruction. The project consists of six parts: 1. Pre-Assess what their students can know and do; 2. Planning instruction and learning experiences based on their pre-assessment results; 3. Deliver the instruction and learning experiences they planned; 4. Post-assess their students; 5. Compare assessment results and draw conclusions on their teaching effectiveness; 6. Reflect on how they would improve the results. Candidates in advanced teacher programs have similar assessments built in to active research projects. Candidates in other school professionals programs complete assessments that cause them to reflect on how their services promoted student learning of the particular population they serve. Candidate assessments on the effect on student learning are housed in the unit s electronic portfolio system. As with other course assessment work, the unit s Support Systems Analyst compiles the data on these assessments by program and loads the reports in the Assessment Library. Lead program faculty members access the data and analyze and aggregate report them by campus location, which are included in the program s annual report. The NCATE Coordinator compiles these data from the annual reports and provides an overall score as well as scores by initial teacher preparation, advanced teacher education and other school professionals categories. These data are also reported by campus site. The NCATE Coordinator disseminates the information to the Coordinating Council as partial evidence on the unit s performance on developing candidates knowledge, skills and dispositions and on fieldwork experiences. 7

11 Assessment #5: Surveys There are three standard surveys that are currently administered on a regular basis: The Graduating Candidate Survey, the Alumni Survey and the Employer Survey. A faculty survey will be developed as well and issued every three years. Other surveys, such as a Mentor Teacher Survey, are administered on a one-time basis to provide data pertinent to questions at hand. Graduating Candidate Survey: Toward the end of candidate s student teaching, internship, or capstone course, they voluntarily complete an electronic exit survey. This survey, while particular to elements in their respective program, measures candidates perceptions on the quality of: their instructors; course content; instructional delivery; preparation in the areas of content and professional knowledge; preparation to teach diverse students including those with exceptionalities; preparation to use technology; advising; facilities including technology; and components of fieldwork experiences. The NCATE Coordinator compiles the data for the teacher education programs, aggregates them and disseminates data particular to the Coordinating Council for partial evidence of meeting standards in developing candidates knowledge, skills, and dispositions, fieldwork experiences, diversity, faculty qualifications, unit resources, and unit operations, including facilities, technology, scheduling, quality of instruction, quality of advising and library holdings. The department chairs of the Educational Psychology and Educational Leadership departments collect the data from the exit reports for their programs. They aggregate these data and send them to the NCATE Coordinator, who further aggregates and disseminates them to the Coordinating Council for unit level evaluation. Alumni survey: Every three years invitations are sent to alumni of the past five years to complete an electronic survey about the effectiveness of their program. The questions contained in the alumni surveys relative to preparation in the areas of knowledge, skills and dispositions and ability to enhance the learning of diverse students are similar to those in the graduating candidate survey, with a focus on evaluating programs effectiveness after graduates have been in the field. The NCATE Coordinator collaborates with the NAU Alumni association to contact alumni via . Arrangements are bring made with the NAU Alumni Office to use social media means, such as Facebook and Linked-In, to solicit responses for our next distribution. Employer survey: As with alumni surveys, invitations to complete an employer survey are sent to principals and superintendents in Arizona every three years. The questions in this survey focus on the degree to which graduates demonstrate the knowledge, skills and dispositions to be successful in their roles and the degree of their ability to enhance the learning of diverse students. The NCATE Coordinator collaborates with the Arizona Department of Education to contact principals and superintendents throughout the state. The NCATE Coordinator compiles the responses of the Alumni and Employer surveys, aggregates them by teacher, administrator and other school professionals categories and disseminates the data to the Coordinating Council as partial evidence of the unit meeting standards in developing candidates knowledge, skills and dispositions as well as in preparing candidates to enhance the learning of diverse students. Assessment #6: Faculty and Clinical Faculty Qualifications Forms All faculty, university supervisors, and cooperating teachers complete electronic surveys regarding their professional qualifications. The university supervisors and cooperating teachers complete 8

12 this at the beginning of the first semester they serve in that capacity and update it each semester afterward. The Professional Education Unit faculty members complete or update a professional information form every fall semester. The content of these surveys includes degrees, areas of degree and institutions granting their degrees; years of experience in P-12 schools and higher education; professional development; service to schools; and scholarly activity. They also include demographic questions. The NCATE Coordinator compiles and aggregates the data from these surveys and disseminates them to the Coordinating Council to assess the qualifications of fieldwork faculty and the diversity of our faculty and clinical faculty. Assessment #7: Faculty and Clinical Faculty Evaluations Candidates evaluate course instruction and university supervisor and cooperating teacher effectiveness. Additionally, cooperating teachers are evaluated the university supervisor with whom they worked. The faculty evaluation data are collected by the faculty members respective department chair. The Assistant Director of Fieldwork Experience collects and compiles the university supervisor and cooperating teacher evaluations. At the beginning of the fall semester, the NCATE Coordinator receives aggregated clinical faculty evaluations from the previous year from the Assistant Director of Fieldwork Experience. These are aggregated as a unit and disseminated to the Coordinating Council members to evaluate the reviews of clinical faculty. The Coordinating Council members analyze the course instruction evaluation data of faculty within their respective colleges. Assessment #8: Institutional Data The Office of Planning And Institutional Research collects data on faculty and students in the university. These data include counts enrolled in programs and courses, counts of program completers, gender and diversity data, and campus location of candidates and faculty. These data are entered into University databases when candidates enroll in programs and courses and when faculty members are hired. Staff from the Office of Planning And Institutional Research run reports on these data at the end of every fall semester, and send them to the NCATE Coordinator. A designated Office of Planning And Institutional Research staff member along with the NCATE Coordinator complete the NAU Title II and the IPEDS reports using these data. Additionally, the Graduate College, the College of Education s Office of Student Services, and department staff for advanced programs collect data on applicant qualifications and numbers admitted to programs. These entities also manage candidates as they apply to their student teaching/internship/capstone course experiences and complete their program. Other institutional data are used by the Coordinating Council to assess unit operations. These data include faculty workloads and University budget reports. Analysis and Use of Assessment Data Unit assessments are conducted at various times throughout the academic year. The data from the assessments are used to inform the unit regarding how well it is meeting the criteria of the six NCATE standards. The data are also used to inform the unit regarding the effectiveness of its unit operations. The details regarding when the assessments are administered, who is involved with the data collection, aggregation and dissemination, and how the data are used are reported in Table 3: Unit Assessments and Data Administration. 9

13 Use of Assessments to Monitor Candidates Key assessments in required courses and fieldwork, clinical and internship experiences are used to evidence candidate s attainment of constructs in our Conceptual Framework and State and Specialized Professional Association Standards. These as well as other data points are used to monitor candidates as they transition through their programs. Candidates are monitored through four transition points: 1. Admission to their program; 2. Admission to student teaching/internship/capstone course 3. Completion of their student teaching/internship/capstone course 4. Completion of their program At each transition point, assessment data are used to provide evidence that candidates are prepared for the next stage. Applicant data (Assessment 8 as described above) are used to determine whether applicants are ready to be admitted to a particular program. The criteria for this assessment include: a minimum overall GPA, and a minimum GPA and grades in contentrelevant courses. Additionally, advanced programs require a Bachelor s degree. Data from Assessment #2 described above (Candidate performance on knowledge, skills and dispositions course assessments) are used to monitor candidates as they enter their student teaching/internship/capstone course experience. Key assessments are housed in required courses. Programs employ one of two options to monitor candidates who enter their student teaching/internship/capstone course experience. Option #1: Candidates must successfully complete the course s key assessment in order to pass the course. With this option, course grades can be used to confirm that candidates are ready to enter their student teaching/internship/capstone course, because passing the courses would require that they have met all the criteria for the key assessments. Option #2: Use of a checklist system. A checklist that includes all the key assessments that must be completed prior to student teaching/internship/capstone course is entered into our University advisement system for every program choosing this option. These program checklists are assigned to candidates. Candidate data on key course assessments are run in TaskStream. At the end of every semester our Support Systems Analyst runs reports by program of candidates status with regards to passing the key assessments. These reports are disseminated to designated staff in each department who check off each assessment successfully completed in the checklist system. When candidates apply to student teaching/internship/capstone course, the person responsible for ensuring they have met all the requirements refers to the candidates checklists in addition to their transcripts. In this manner, programs ensure all the required key assessments have been successfully completed. In order to complete the student teaching/internship/capstone course experience, candidates must demonstrate they meet constructs of our Conceptual Framework and State and Specialized Professional Association standards through key assessments measuring their effect on student learning as well as through checklist assessments that have been mapped to standards (Assessments 3 and 4 as described above). Although there may be other assignments in the course, candidates cannot pass the course unless they perform successfully on these two assessments. Program completion criteria are successful completion of the student teaching/internship/ capstone course, and meeting the GPA and course grade requirements of the program. It is at this point an Institutional Recommendation is given for certifications that require one. 10

14 Use of Assessments to Manage and Improve Unit Operations The management of unit operations is accomplished through regularly analyzing the data gathered from our eight unit assessments and using these analyses to confirm strengths and identify areas for improvement of our operations. The Coordinating Council oversees the management of unit operations. The Coordinating Council also assesses the unit with regards to Unit Governance and Resources and Faculty Qualifications, Performance and Development. This is illustrated in Appendix 2: Unit Governance and Resources Data Assessment Table. The Coordinating Council analyses assessment data from operations-related questions on candidate, alumni and employer surveys (Assessment 5) as well as course evaluations and faculty qualifications (Assessments 6 and 7) to manage and improve operations of the unit. The Coordinating Council holds a special meeting during the summer to analyze data and make recommendations for unit improvements based on their findings. Use of Assessments to Manage and Improve the Unit Assessment System The unit manages and improves its Assessment System through the Coordinating Council and college faculty meetings. The Coordinating Council monitors the assessment system through its evaluation of candidates knowledge, skills and dispositions, the assessment system, fieldwork experiences, diversity, faculty qualifications, unit governance and resources, and unit operations annually during the summer. To do this, the Coordinating Council analyzes unit, institutional and survey data. This is outlined in Appendix 2. At the beginning of every fall semester the Coordinating Council conducts a full PEU faculty meeting. At this meeting updates are given and faculty members are charged to address standards for which the Coordinating Council found weaknesses as a unit. Faculty college groups throughout the year undertake follow-up work. If a full faculty meeting is needed at the beginning of the spring semester to update and/or reorganize work efforts, one is held. Another full faculty meeting is held at the end of the academic year to update faculty on progress and implications for the following year. 11

15 Table 3 Unit Assessments and Data Administration Who Collects Data Assessment Assessment 1: Candidate Licensure Exam Scores When Beginning of calendar year for prior year Data Source National Testing Agencies NCATE Coordinator Who Compile s Data NCATE Coordinator Who Analyses Data Coordinati ng Council Data Analysis Dissemination Coordinating Council to faculty How Data are Used Direct assessment whether content knowledge meets standard Outcomes Addressed 1, 4 Direct Assessment Assessment 2: Candidates performance on course assessments. Assessment 3: Candidate performance on student teaching/intern ship assessments. Assessment 4: Candidate performance on student learning assessments. June for prior calendar year End of each semester End of each semester Candidate Data on Signature Assignments Student Teaching or Internship Evaluations Assessments during Student Teaching or Internship Lead Program Faculty Members Director of Fieldwork, Lead Program Faculty Members Director of Fieldwork, Lead Program Faculty Members Support Systems Analyst, NCATE Coordinat or NCATE Coordinator, Lead Program Faculty or Dept. Chairs NCATE Coordinator, Lead Program Faculty or Dept. Chairs Coordinati ng Council Coordinati ng Council Coordinati ng Council Coordinating Council to faculty Coordinating Council to faculty Coordinating Council to faculty Direct assessment whether candidates content knowledge, pedagogical knowledge/skills, professional knowledge/skills, diversity, and technology and dispositions meet standard. Direct assessment whether candidates content knowledge, pedagogical knowledge/skills, professional knowledge/skills, diversity, and technology and dispositions meet standard; Determine efficacy of student teaching/internship experiences. Direct assessment whether candidates ability to affect all students learning meets standard. Determine efficacy of student teaching/internship experiences. 1, 2, 3, 4, 5 Direct Assessment 1, 2, 3, 4, 5 Direct Assessment 2, 3, 4, 5 Direct Assessment 12

16 Assessment Assessment 5: Surveys A. Graduating Candidate Survey B. Alumni Survey C. Employer Survey Assessment 6: Faculty and Clinical faculty qualifications forms. Assessment 7: Faculty and Clinical faculty evaluations. Assessment #8: Institutional Data When A. End of each semester B. & C. Every 3 years Annually (each semester new personnel are hired) Annually and Each Semester Annually and Each Semester Data Source Electronic Surveys through Survey Monkey Electronic Faculty and clinical Faculty Information forms Student Course Evlauations; Exit Survey University enrollment and completer databases, University faculty database Who Collects Data Director of Fieldwork, Dept. Chairs, NCATE Coordinator NCATE Coordinator, Director of Field-work Coordinati ng Council, Director of Field-work The Office of Planning and Institutional Research Who Compile s Data Dept. Chairs, NCATE Coordinator NCATE Coordinator Coordinat ing Council, NCATE Coordinator NCATE Coordinator Who Analyses Data Coordinati ng Council Coordinati ng Council Coordinati ng Council Coordinati ng Council Data Analysis Dissemination Coordinating Council to faculty Coordinating Council to administration Coordinating Council to administration Coordinating Council to administration How Data are Used Indirect assessment on programs impact on the development of candidates content, pedagogical and professional knowledge and skills, the quality of instruction, field experiences, preparation to work with diverse candidates, unit operations, and coursework. Direct assessment whether faculty and clinical faculty meet qualifications standards. Indirect assessment of whether faculty and clinical members performance meet standards; For Title II, IPEDS, and NCATE reports. For assessing diversity of faculty and candidates. For assessing trends in program enrollment and completers and candidate to faculty ratios. For assessing ratios of part-time to full-time faculty and tenure track faculty and faculty workloads. For assessing the budget. Outcomes Addressed 1, 2, 3, 4, 5 Indirect Assessment Unit Operations Unit Operations Unit Operations 13

17 Use of Assessments to Manage and Improve Programs Data from all the assessments are used to manage and improve programs. All programs are responsible for developing an assessment plan to manage their program s candidate data (Assessments 1 4). Assessment Plans When a new program is initiated, it must develop an assessment plan. The assessments in their assessment plan must be aligned with professional and state standards as well as with our Conceptual Framework. This plan must be updated when a program makes curricular changes. This plan is submitted to the NCATE Coordinator who checks it and forwards it to the Office of Academic Assessment where it is housed. The Assessment Plan Template is found in Appendix 4. Annual Reports Program faculty members use the data collected on their candidates to evaluate their program and make changes accordingly. These changes are documented in the program annual report. Each program is responsible for completing and submitting their annual report. The annual report process involves candidate data being collected in our electronic portfolio system, compiled and entered into our Assessment Library by the Support Systems Analyst, accessed from the Assessment Library by lead program faculty members, and analyzed and discussed by full program faculty members. The annual reports are based on these analyses and discussions. The timeline for this is illustrated in Table 3. The Data Discussion Table should be used to document data-driven discussions and decisions, and should be submitted with the Annual Report. The Data Discussion table template is found in Appendix 5. The Annual Report Templates for Initial and Advanced programs are found in Appendix 6. These are submitted to the NCATE Coordinator who will forward them to the Office of Academic Assessment. The annual schedule of when program faculty members and other personnel should engage in the various activities of monitoring assessments and program candidate data is found in Table 4: Program Annual Schedule. Table 4. Program Annual Schedule Month Activities Persons Responsible Summer and Fall Semesters May December Collect Candidate Data All program faculty members evaluate candidates May April - June In LOUIE, Check off Signature Assignments Candidates Completed Spring Semester Check Spring Candidate Data for Transition Points 2, 3 and 4 Annual Program Reports Due to OAA and the Assistant Vice Provost Appointed personnel of programs using the checklist system to verify completion of signature assignments Appointed personnel responsible for verifying requirements have been met for candidates to enter student teaching/internship and/or completion of the program. Lead program faculty members 14

18 August September October November June 1. In LOUIE, Check off Signature Assignments Candidates Completed Summer Semester Check Summer Candidate Data for Transition Points #2, 3, and 4 *Program Faculties Review Program s Annual Report for the Prior Year; Implement Plans for Change Spring Candidate Data are Loaded in the Assessment Library Faculty Check Accuracy Summer Candidate Data are Loaded in the Assessment Library Faculty Check Accuracy Invitations to Complete the Exit Survey are ed to Initial Candidates Appointed personnel of programs using the checklist system that verify completion of signature assignments Appointed personnel responsible for verifying requirements have been met for candidates to enter student teaching/internship and/or completion of the program. *Program faculty members Lead program faculty members Lead program faculty members NCATE Coordinator; Director of Student Services 15

19 Month Activities Persons Responsible December Spring Semester January May January March 1 April April - June In LOUIE, Check off Signature Assignments Candidates Completed Fall Semester Check Fall Candidate Data for Transition Points #2, 3, and 4 Collect Candidate Data Fall Candidate Data are Loaded in the Assessment Library Faculty Check Accuracy Program Faculty Members Meet to Review Prior Year s Data, Discuss Implications and Changes to be Made Begin Writing Annual Program Reports **Title II Information due to the Assistant Vice Provost Invitations to Complete the Exit Survey are ed to Initial Candidates Title II Report Submitted Submit Annual Program Reports Due to OAA and the Assistant Vice Provost June 1. Appointed personnel of programs using the checklist system that verify completion of signature assignments Appointed personnel responsible for verifying requirements have been met for candidates to enter student teaching/internship and/or completion of the program. All program faculty members evaluate candidates Lead program faculty members Program faculty members Lead program faculty members Lead program faculty members of programs that have implemented changes; Science, Mathematics, and Special Education lead faculty members who must report Institutional goals in these areas NCATE Coordinator; Director of Student Services NCATE Coordinator Lead program faculty members 16

20 Month Activities Persons Responsible *Program Faculties Review their Program s Annual Report *Program faculty members May In LOUIE, Check off Signature Assignments Candidates Completed Spring Semester Check Spring Candidate Data for Transition Points 2, 3 and 4 Appointed personnel of programs using the checklist system that verify completion of signature assignments Appointed personnel responsible for verifying requirements have been met for candidates to enter student teaching/internship and/or completion of the program. *Program faculty should meet at least one time during the academic year to review their program s annual report. **Title II information applies to all programs approved by the Arizona Department of Education. This information includes the Program Change Table for programs that have changed the courses in their curriculum or that have approved changes that will be implemented the following fall semester. This also includes a report on goals met and setting of new goals for science, math, and special education programs. Program Changes and New Programs Any changes in courses, transition points, or signature assignments in programs must be approved by the NCATE Coordinator in order to ensure the change does not inadvertently eliminate or reduce the amount of candidate data required of the unit for NCATE accreditation. The NCATE Coordinator s responsibility is to ensure that measures to obtain program assessment data required by the state, specialized professional associations, and the unit are included in the program s changed assessment plan. The NCATE Coordinator does not evaluate the content of the program outside these parameters. Similarly, the NCATE Coordinator reviews the proposals for new programs to ensure program assessment plans include that which is required by the state, specialized professional associations, and the unit. When program faculty know they will be requesting approval for a program change or new program, they should submit an Early Alert Form to the NCATE Coordinator. This form outlines the necessary procedures to submit the program for approval. This form can be found in Appendix 7. Program faculty requesting the NCATE Coordinator s approval of program changes with respect to accreditation implications must complete and submit the Accreditation Program Change form. This can be found in Appendix 8. Program faculty proposing a new program must complete the Program Assessment Plan. This form can be found in Appendix 4. Upon successful review, the NCATE Coordinator writes a memo stating that there are no negative accreditation implications with the program change and sends it to the lead program faculty member who submits it with the curriculum change materials to the respective university committee (the Undergraduate Curriculum Committee or the University Graduate Committee). 17

21 Seven-Year Schedule The annual schedule is repeated until data need to be collected for NCATE and SPA reporting. The following timeline illustrates when data collection begins for NCATE and SPA reports. For programs not having a SPA to which to report, a University Seven-Year Self Study is written and submitted to the Associate Vice Provost for Academic Affairs. 18

22 NCATE Timeline 19