Assessment #1: AEPA: Administrator Certification, Superintendent

Transcription

1 Assessment #1: AEPA: Administrator Certification, Superintendent 1. Description of Assessment: The Arizona Educator Proficiency Assessments (AEPA): Administrator Certification, Superintendent is the licensure examination for the state. This examination is composed of five subareas: Leadership of the Educational Organization; Social, Cultural, and Legal Context; Effective Communication; Management of the Educational Organization; and Curriculum and Instruction. These subareas are directly aligned with ISLLC Standards. The examination is typically taken by candidates at or near the end of the program. 2. Alignment with Specified Standards: The AEPA: Administrator Certification, Superintendent licensure examination five subareas are aligned to the ELCC standards as follows: Leadership of the Educational Organization: ELCC Standards 1-6 Social, Cultural, and Legal Context: ELCC Standards 5 & 6 Effective Communication: ELCC Standards 1 & 4 Management of the Educational Organization: ELCC Standard 3 Curriculum and Instruction: ELCC Standard 2 3. Analysis of Data Findings: Overall, the pass rate for candidates taking the AEPA: Administrator Certification, Superintendent has been 92% or greater for the past three years. 4. Interpretation of How Data Provides Evidence for Meeting Standards: As noted in item two, there is close alignment between the AEPA five subareas and the ELCC standards. Candidates pass rate of this examination has consistently exceeded the 80% requirement, indicating that candidates have met the standards. 5. Assessment Documentation (a) Description of Assessment Tool(s)/Assignment(s): The Arizona Educator Proficiency Assessments (AEPA): Administrator Certification, Principal licensure exam tests for the following objectives: Leadership of the Educational Organization o 0001 Understand how the mission of the educational organization is developed and promoted. o 0002 Understand how to sustain a climate of innovation and change in the educational organization. o 0003 Understand strategies for promoting professional development in the educational organization. Social, Cultural, and Legal Context o 0004 Understand how to promote equity in educational programs. o 0005 Understand and ensure compliance with Arizona and federal statutes and regulations. o 0006 Understand how to formulate policies and work effectively with educational policymakers. Effective Communication o 0007 Understand the use of effective communication strategies to build support for improving the learning environment. 1

2 o 0008 Understand how to encourage and respond to feedback from members of the educational and local community. o 0009 Understand how to encourage and support teamwork to promote the mission of the educational organization. Management of the Educational Organization o 0010 Apply principles of management and organizational behavior in problem solving, decision making, and organizational development. o 0011 Understand public school finance and financial management. o 0012 Understand school physical plant management and the management of school services. o 0013 Apply principles of effective human resource management. Curriculum and Instruction o 0014 Understand principles and characteristics of curriculum planning, design, implementation, and evaluation. o 0015 Understand principles and characteristics of instruction designed to promote student achievement. o 0016 Understand principles, purposes, and methods of student assessment. (National Evaluation Systems, Inc., 1998) (b) Scoring Guide(s): Subarea Range of Objectives Approximate Test Proportions I. Leadership of the Educational % Organization II. Social, Cultural, and Legal Context % III. Effective Communication % IV. Management of the Educational % Organization V. Curriculum and Instruction % (c) Aggregate Candidate Data: During the past three years, 40 candidates have taken the AEPA Superintendent Test. On average, 97% of these candidates have passed this examination. Test Results for AEPA Superintendent Tests (through July 2007 test) Year Number of Test Takers Pass Rate (Superintendent - AEPA Test #80) % % % 2

3 Assessment #2: Content Knowledge: Assessment 1. Description of Assessment: Three assessment activities in EDL 600, Leadership Skills, namely, Professional Development Assessment and Plan, Case Presentations/Analyses, and Case Discussion/Reactions measure candidate proficiency on knowledge of issues such as vision, management, and change. 2. Alignment with Specified Standards: From Fall 2006 to Summer 2007, the Educational Leadership faculty reviewed in detail all ELCC District Level Standards and revised required courses for the superintendent program to directly align with the standards that were assigned and considered a good fit for the course. The three assessment activities (Professional Assessment and Plan, Case Presentations/Analyses, and Case Discussion/Reactions) are aligned to and measure student proficiency on ELCC Standards 1.4, 2.4, 3.2, 5.2, and 5.3. The rubric for the Professional Development Assessment and Plan was developed and implemented in Fall The scoring guide below shows how the various aspects of these three assessment activities are aligned with each of these standards. 3. Analysis of Data Findings: A summary table aggregating data collected from the three assessment activities in the Fall 2007 term is documented as part of question 5 (below). The data is disaggregated by the following locations: Phoenix, Tucson, and Rural. Of the 5 standards addressed by this assessment, 86% or more of candidates are meeting or exceeding (a score of 3 or 4) all standards. Based on the data collected, there may be evidence that there is not an adequate difference between the descriptions for levels 3 and 4. These descriptions will be reviewed again after aggregate data is available for the Spring 2008 and Summer 2008 terms. 4. Interpretation of How Data Provides Evidence for Meeting Standards: Overall, the data indicates that 86% of candidates or more at all sites are meeting or exceeding all five standards contained in this assessment. The following list summarizes the percentage of candidates meeting or exceeding each standard addressed by Assessment #2: Content Knowledge. ELCC 1.4 Steward a District Vision of Learning, 93% ELCC 2.4 Design Comprehensive Professional Growth Plans, 98% ELCC 3.2 Manage the Operations, 91% ELCC 5.2 Acts Fairly, 97% ELCC 5.3 Acts Ethically, 96% 5. Assessment Documentation: a) Description of Assessment Tool(s)/Assignment(s): EDL 600 Professional Development Assessment & Plan: Drawing on information derived from the use of various self-assessment instruments, including one based on the ISLLC Standards, and reflection on their 1

4 experience, beliefs, and dispositions, candidates develop a three-year Professional Development Plan. Case Presentations/Analyses: Candidates select a case from an assigned text (Snowden & Gorton) and analyze the case, listing problems and recommended actions. These analyses are presented to their peers. Case Discussion/Reactions: Candidates post reactions to cases presented by other candidates in the course. Knowledge of leadership concepts are needed to adequately address problems and issues found in the case as well as to recommend actions to keep similar problems from surfacing in the future. a) Scoring Guide(s): 2

5 EDL 600 Professional Development Assessment & Plan: CRITERIA Case Study Analysis Stewardship of the Vision (ELCC 1.4) Leadership Assessment and Professional Growth Plan (PDP) (ELCC 2.4) Managing Operations (ELCC 3.2) Acts Fairly (ELCC 5.2) 4 Pts 3 Pts Cutoff Score* Case analyses address need for reaching consensus of the vision; include effective communication strategies; recommended actions are based on all relevant data Objectives are clear and based on reflection of personal assessment information. Strategies for meeting objectives are manageable and use available resources. Plan displays a high degree of rigor. Identifies all conflict in case needs assessments, lists priorities accurately, and recommended actions are based on research data from reading and interviews. Ethical standards statement in PDP and problems found in case analyses display an high concern for diversity and the needs of others Case analyses address need for reaching consensus of the vision; include at least one communication strategy; recommended actions are based on a majority of data Objectives are clear and based on reflection of personal assessment information. Strategies for meeting objectives are manageable and use available resources. Plan displays adequate rigor. Identifies a majority of conflict in case needs assessments, lists priorities accurately, and actions are based on some research data from reading and interviews. Ethical standards statement in PDP and problems found in case analyses display an adequate concern for diversity and the needs of others Pts 1 Pt Case analyses address need for vision but fail to include strategy or realistic plan for shared vision Objectives and criteria for evaluation are very general in nature with some alignment with personal assessment information. few or no available resources used, no manageable timeline Identifies less than majority of conflict in case needs assessments, prioritizing not accurate, and actions are based on little research data from reading and interviews. Ethical standards statement in PDP and problems found in case analyses display little concern for diversity or the needs of others Case analyses fail to address vision or strategies for reaching a shared vision Objectives are vague, no available resources used, no manageable timeline or criteria for evaluation and not based on reflection of personal assessment information Identifies few conflicts in case needs assessments, prioritizing not accurate, and actions are not based on research data from reading and interviews. Ethical standards statement in PDP and problems found in case analyses display no concern for diversity or the needs of others Acts Ethically Ethical standards Ethical standards statement Ethical standards statement Ethical standards 3

6 CRITERIA Case Study Analysis (ELCC 5.3) 4 Pts 3 Pts Cutoff Score* statement in PDP and all decisions/ recommended actions in case analyses display an high regard to ethics in PDP and most decisions/recommended actions in case analyses display an adequate regard to ethics 2 Pts 1 Pt in PDP and some decisions/ recommended actions in case analyses display a low regard to ethics statement in PDP and decisions/recommended actions in case analyses display no regard to ethics. 4

7 c) Aggregate Candidate Data: The following tables include data collected from EDL 600 courses taught during the Fall 2007 term. Initial implementation of the signature assignments and rubric occurred during the Fall 2007 term. Overall data was collected for 152 candidates out of 245 enrolled (67% data collection rate). Steps were taken to collect missing data. Implementation of the department s Course Mentor Program and other department and distance learning follow-up procedures is continuing so as to better ensure that data is collected from all instructors in future terms. EDL 600, Phoenix (Fall 2007) ELCC Assessment #2 Criteria N 1 % 2 % 3 % 4 % % Meeting Standard ELCC % 6 5% 16 14% 83 72% 86% ELCC % 0 0% 58 50% 57 50% 100% ELCC % 6 5% 24 21% 75 65% 86% ELCC % 0 0% 37 32% 70 61% 93% ELCC % 2 2% 27 23% 76 66% 90% Overall Average 91% EDL 600, Tucson (Fall 2007) ELCC Assessment #2 Criteria N 1 % 2 % 3 % 4 % % Meeting Standard ELCC % 3 7% 14 32% 27 61% 93% ELCC % 3 7% 12 27% 29 66% 93% ELCC % 6 14% 15 34% 23 52% 86% ELCC % 1 2% 21 48% 22 50% 98% ELCC % 1 2% 20 45% 23 52% 98% Overall Average 94% EDL 600: Rural (Fall 2007) ELCC Assessment #2 Criteria N 1 % 2 % 3 % 4 % % Meeting Standard ELCC % 0 0% 0 0% 4 100% 100% ELCC % 0 0% 0 0% 4 100% 100% ELCC % 0 0% 1 25% 3 75% 100% ELCC % 0 0% 0 0% 4 100% 100% ELCC % 0 0% 0 0% 4 100% 100% Overall Average 100% 5

8 Assessment #3: Professional Knowledge, Skills & Dispositions: Supervisory Plan 1. Description of Assessment: EDL 725, Supervision Project, is a simulation which involves candidates supervising an adult educator. 2. Alignment with Specified Standards: From Fall 2006 to Summer 2007, the Educational Leadership faculty reviewed in detail all ELCC District Level Standards and revised required courses for the superintendent program to directly align with the standards that were assigned and considered a good fit for the course. The rubric for the EDL 725, Supervision Project, was developed and implemented in Fall The EDL 725, Supervision Project, is aligned to the following ELCC Standards: 2.1, 2.2, 2.3, 2.4, 4.2, 5.1, 5.2, and 6.1. The scoring guide below shows how the various aspects of each assignment are aligned to the standards. 3. Analysis of Data Findings: A summary table aggregating data collected from EDL 725 for the Fall 2007 term is documented as part of question 5 (below). The data is disaggregated by the following locations: Phoenix, Tucson, and Rural. Of the 8 standards addressed by this assessment, 81% of candidates or more are meeting or exceeding (a score of 3 or 4) all standards. The data indicates that some instructors are using a restricted range when scoring this assignment. For example, instructors at some sites tended to only rate candidates at the 3 or 4 level for all criteria, indicating that training to improve interrater reliability is needed. In addition to reviewing the data, the Course Mentor gathered feedback from EDL 725 instructors at the conclusion of the semester regarding the signature assignment and rubric. The following recommendations were made: ELCC 4.1: Student and parent feedback is not typically included in teacher evaluation. Recommended that the criteria related to this standard be deleted from the assignment and rubric. The standard is also assessed in EDL 729 and EDL 762. ELCC 5.3: After implementation of the Supervision Project, instructors found 5.3 to be a poor fit for the assignment and recommended that the standard be deleted as a criterion for the assignment. Content knowledge related to 5.3 is assessed in EDL 600 and EDL 762 and as a skill in EDL 796. ELCC 6.1: After implementation of the Supervision Project, instructors found little evidence in projects related to standard 6.1. Through a discussion of lead department faculty, it was determined that 6.1 would remain as a criteria for the assignment, but the assignment instructions would be changed to ask candidates to address and include citations related to Title Interpretation of How Data Provides Evidence for Meeting Standards: Overall the data indicates that 86% of candidates or more at all sites are meeting or exceeding all 10 standards contained in this assessment. The following list summarizes the percentage of candidates meeting or exceeding each standard addressed by Assessment #3: Professional Knowledge, Skills & Dispositions: Supervisory Plan. ELCC 2.1 Promote a Positive District Culture, 94% 1

9 ELCC 2.2 Provide Effective Instructional Program, 93% ELCC 2.3 Apply Best Practice to Student Learning, 89% ELCC 2.4 Design Comprehensive Professional Growth Plans, 91% ELCC 4.1 Collaborate with Families and Other Community Members, 90% ELCC 4.2 Respond to Community Interests and Needs, 95% ELCC 5.1 Acts with Integrity, 94% ELCC 5.2 Acts Fairly, 96% ELCC 5.3 Acts Ethically, 88% ELCC 6.1 Understand the Larger Educational Context, 91% 5. Assessment Documentation: a) Description of Assessment Tool(s)/Assignment(s): EDL 725 Supervision Project: Each candidate is required to complete a field experience, in which they simulate the supervision of another adult educator. The project must be in an area of supervision discussed in the course, requiring the application of theory to practice. is not to simply reenact district protocol; the candidate must develop and implement research-based best practices for supervision of instructors. A mixture of short and long observations should be used. Formative feedback and summative evaluation should be documented. Candidates must participate in post-conference role play scenarios in class prior to beginning supervision project. b) Scoring Guide: (see below) 2

10 EDL 725, Supervision Project CRITERIA 4 Pts 3 Pts Cutoff Score** 2 Pts 1 Pt Exemplary Proficient 3 2 Basic Unacceptable Holds Preobservation conference (ELCC 2.4) demonstrated the ability to plan and initiate a well-designed, context appropriate professional development process by holding a collaborative pre-observation conference. The reason, purpose and focus of the upcoming observation were determined. The time of the observation and the post-conference were determined in collaboration with the teacher. Candidate s conference was based on reflective practice and research (included detailed description of the discussion and additional information, such as teacher background, class demonstrated the ability to plan and initiate a welldesigned, context appropriate professional development process by holding a preobservation conference. The reason, purpose and focus of the observation were determined. The time of the observation and the post-conference were determined by student. Candidate s conference was based on reflective practice and research (included brief description of the discussion and additional information, such as teacher background, class demonstrated the ability hold a pre-observation conference. The reason, purpose and focus of the observation were included but unclear. The time of the observation or the postconference was not included. Candidate s conference was based on their own reflective practice and research on student learning (included little description of discussion). did not demonstrate the ability to hold a pre-observation conference. The reason, purpose and focus of the observation were not included. The time of the observation and the postconference was not included. If held, conference was not based on any premise of pedagogy (no description of discussion was included). 3

11 make-up, etc.). make-up, etc.). AVERAGE FOR CELL: Designs an Observation Tool Understanding the Larger Context (6.1) Based on the preobservation conference, the candidate was able to design an observation tool that addressed pre-conference focus and was tailored to the needs and interest of the teacher. The observation tool included an information key, including the beginning and ending time of the observation and procedure/ explanation for use of the tool. Candidate demonstrates policies, laws, and regulations enacted by local, state, or federal authorities that affect schools, by researching, citing, and describing applicable policies and laws considered when designing their evaluation process. Based on the preobservation conference, the candidate was able to design an observation tool that addressed pre-conference focus and was tailored to the needs and interest of the teacher. The observation tool included an information key and procedure for the use of the tool. Candidate demonstrates policies, laws, and regulations enacted by local, state, or federal authorities that affect schools, by researching and citing applicable policies and laws considered when designing their evaluation process. Candidate was able to design an observation tool that addressed predetermined observational needs, not tailored to the needs and interest of the teacher. The observation tool included an information key and procedure for the use of the tool. Candidate demonstrates a developing policies, laws, and regulations enacted by local, state, or federal authorities that affect schools, by vaguely describing applicable policies and laws that should be considered when designing their evaluation process. Observation tool was not designed to meet the predetermined observation focus and purpose. The observation tool did not include an explanation for how it was used and key for interpretation. Candidate does not demonstrate understanding of policies, laws, and regulations enacted by local, state, or federal authorities that affect schools; fails to research, citing, or describing applicable policies and laws considered when designing their evaluation process. 4

12 Promotes Positive School Culture (2.1) Promotes Effective Instructional Program (2.2) Completes Observation in a professional manner (ELCC 2.4) utilizes observation tool and post conference feedback to reinforce gender, language, race, socio-economic equity. identifies multiple principles of effective instruction, curriculum, and technology to fully accommodate learners diverse needs. used the observation tool to complete at least 45 minutes, or full lesson, classroom observation. took detailed notes w/ analysis, and excellent descriptions of classroom events as they related to the purpose of the observation. utilizes observation tool and post conference feedback to reinforce student equity in at least one area. identifies at least two of the following: principles of effective instruction, curriculum, or technology to accommodate learners diverse needs. used the observation tool to complete a minimum of a 30 minute classroom observation. demonstrated the ability to take notes and describe classroom events as they related to the purpose of the observation. AVERAGE FOR CELL: utilizes observation tool or post conference feedback to reinforce student equity. identifies at least one of the following: principles of effective instruction, curriculum, or technology to accommodate learners diverse needs. used the observation tool to complete a minimum of a 20 minute classroom observation. s notes and description of classroom events were not detailed and descriptive but did relate to the purpose of the observation. does not reinforce student equity in any manner. does not identify any major principles to accommodate learners diverse needs. did not spend sufficient time in the classroom to complete a meaningful observation. s notes were not descriptive and did not relate sufficiently to the purpose of the observation. Advocates for success for all students (4.2) Completes Observation in a Uses appropriate assessment strategies to serve students special and exceptional needs, and capitalize on diversity. The descriptions and details were presented Attempts assessment strategies to serve students special and exceptional needs, and capitalize on diversity. The descriptions and details were presented Attempts assessment strategies to serve students special and exceptional needs, or capitalize on diversity. The descriptions and details were presented in Candidate does not attempt to use assessment strategies to serve students special and exceptional needs, or capitalize on diversity. A lack of concern for teacher s rights and dignity 5

13 professional manner (ELCC 5.1) Written analysis and summary Written analysis and summary (ELCC 2.3) in a manner that showed excellent respect for the teacher s classroom, dignity, and concern for maintaining the teacher s confidentiality. Candidate was able to identify multiple issues related to each: classroom climate, diversity, management, and instructional practices. Candidate was able to reflect on, review, analyze and identify and categorize the strengths and refinement areas of instructional practices, classroom management or other issues based on observation data with great detail and precision. s summary was comprehensive and provided multiple recommendations based on research or best practices. Emphasizes student achievement as the result of instructional practice. in a manner that showed some respect for the teacher s classroom, dignity, and concern for confidentiality. Candidate was able to identify one issue related to each: classroom climate, diversity, management, and instructional practices. AVERAGE FOR CELL: Candidate was able to reflect on, review, analyze and identify and categorize the strengths and refinement areas of instructional practices, classroom management or other issues based on observation data with effective detail. s summary was well written but made only one recommendation based on research or best practices. Refers to student achievement as the result of instructional practice. a manner that showed a little respect for the teacher s classroom and dignity, but did not show any concern for confidentiality. Candidate was able to identify one issue related to classroom climate, diversity, management, or instructional practices. Candidate was able to reflect on, review, analyze and identify the strengths and refinement areas, but was unable to narrow and categorize them into issues of instructional practices, classroom management or other issues based on observation data. s summary was wellwritten and provided constructive feedback with no clear tie to research or best practice, or inadequate emphasis on student achievement as the result of instructional practice. and/or absence of confidentiality was displayed in the descriptions. Candidate was unable to identify one issue related to classroom climate, diversity, management, or instructional practices. did not collect sufficient data to make an analysis and draw conclusions. The summary was not well written and provided no feedback tied to research or best practices. Does not connect student achievement to instructional practice. 6

14 Written analysis and summary (ELCC 5.2) Post-conference profession conduct (ELCC 5.3) Post-conference: develops professional growth plan (ELCC 2.4) The summary was comprehensive in scope: using in-depth task analysis for developing feedback (+/ chart), addressing areas of concern, showing excellent impartiality, sensitivity to student diversity, and constructive feedback which eliminated issues of personal bias. Candidate was able to hold the post conference and provided comprehensive feedback and explanation to the teacher. demonstrated the ability to collaborate with the teacher by allowing him/her to respond to the feedback and participate in the development of a professional growth plan. The professional growth plan reflected a commitment to life-long learning and included: goals and objectives for professional growth as identified in the observation; activities and strategies to be The summary was adequate in scope: using task analysis for developing feedback (+/ chart), written in a manner that eliminated concerns of personal bias, impartiality, and was sensitive to the needs and feelings of the teacher. Candidate was able to hold the post conference and provided feedback and explanation to the teacher. demonstrated the ability to collaborate with teacher by allowing him/her to respond to the feedback or participate in the development of a professional growth plan. The professional growth plan reflected a commitment to learning and included at least 3 of the following: goals and objectives for professional growth as identified in the observation; activities The summary provided some feedback but lacked adequate analysis of lesson to provide constructive feedback, or had overtones of personal bias or insensitivity. Candidate was able to hold the post conference and provide feedback to the teacher. provided feedback and/or created professional growth plan, without any indication of collaboration. The professional growth plan included 1-2 of the following: goals and objectives for professional growth; activities/ strategies to be undertaken; a timeline for start and completion of the activities; resources The summary contained no evidence of lesson analysis, or strong personal biases and/or overt display of insensitivity to the feelings of others. Candidate did not hold the post conference and provided no feedback to the teacher. There was no professional growth plan developed. The professional growth plan did not identify any of the following: : goals/objectives for professional growth; activities and strategies; a timeline for the activities; resources needed. 7

15 undertaken to address the objectives; a timeline for start and completion of the activities; and resources needed for implementation. and strategies to be undertaken to address the objectives; a timeline for start and completion of the activities; resources needed for implementation. AVERAGE FOR CELL: needed for implementation. 8

16 c) Aggregate Candidate Data: The following tables include data collected from EDL 725 courses taught during the Fall 2007 term. Initial implementation of the signature assignment and rubric occurred during the Fall 2007 term. Overall the data was collected for 84 candidates out of 113 enrolled (74% data collection rate). Steps were taken to collect missing data. Implementation of the department s Course Mentor Program and other department and distance learning follow-up procedures is continuing as to better ensure that data is collected from all instructors in future term. EDL 725, Phoenix (Fall 2007) ELCC Assessment #3 Criteria N 1 % 2 % 3 % 4 % % Meeting Standard ELCC % 3 7% 13 28% 30 65% 93% ELCC % 5 11% 13 28% 28 61% 89% ELCC % 6 13% 6 13% 34 74% 87% ELCC 2.4* % 21 14% 27 18% % 86% ELCC % 7 15% 1 2% 38 83% 85% ELCC % 5 11% 2 4% 39 85% 89% ELCC % 6 13% 5 11% 35 76% 87% ELCC % 3 13% 2 9% 18 78% 87% ELCC % 8 17% 4 9% 34 74% 83% ELCC % 6 13% 7 15% 33 72% 87% Overall Average 87% *Row aggregates four separate rubric criteria related to ELCC 2.4. EDL 725, Tucson (Fall 2007) ELCC Assessment #3 Criteria N 1 % 2 % 3 % 4 % % Meeting Standard ELCC % 2 6% 13 42% 15 48% 90% ELCC % 2 6% 9 29% 19 61% 90% ELCC % 3 10% 8 26% 17 55% 81% ELCC 2.4* % 7 6% 22 18% 86 69% 87% ELCC % 5 16% 0 0% 26 84% 84% ELCC % 1 3% 11 35% 19 61% 97% ELCC % 2 6% 6 19% 23 74% 94% ELCC % 0 0% 11 35% 20 65% 100% ELCC % 6 19% 10 32% 15 48% 81% ELCC % 3 10% 11 35% 16 52% 87% Overall Average 89% *Row aggregates four separate rubric criteria related to ELCC

17 EDL 725, Rural (Fall 2007) ELCC Assessment #3 Criteria N 1 % 2 % 3 % 4 % % Meeting Standard ELCC % 0 0% 3 43% 4 57% 100% ELCC % 0 0% 2 29% 5 71% 100% ELCC % 0 0% 3 43% 4 57% 100% ELCC 2.4* % 0 0% 13 46% 15 54% 100% ELCC % 0 0% 6 86% 1 14% 100% ELCC % 0 0% 2 29% 5 71% 100% ELCC % 0 0% 0 0% 7 100% 100% ELCC % 0 0% 1 14% 6 86% 100% ELCC % 0 0% 3 43% 4 57% 100% ELCC % 0 0% 4 57% 3 43% 100% Overall Average 100% *Row aggregates four separate rubric criteria related to ELCC

18 Assessment #4: Professional Knowledge, Skills & Dispositions: Assessment of Internship 1. Description of Assessment: NAU s Educational Leadership Superintendnent Internship (EDL 796) is the final step in the Graduate Certificate Program preparation of candidates as future district administrators. Candidates intern in cooperating districts under the direction of an experienced, certificated superintendent and a university supervisor. It is an opportunity for candidates to practice and apply administrative theory in various areas of responsibility. 2. Alignment with Specified Standards: From Fall 2006 to Summer 2007, the Educational Leadership faculty reviewed in detail all ELCC District Level Standards and revised required courses for the superintendent program to directly align with the standards that were assigned and considered a good fit for the course. EDL 796 is designed to be both a comprehensive and summative field experience and is aligned to all ELCC standards. The Internship Assessment rating form was developed and implemented in Fall Additionally, listed below is a summary of how the internship and embedded field experiences meet ELCC standards 7.1, 7.2, 7.4, 7.5, and 7.6. ELCC Internship Standard 7.1. Substantial: a) Candidates demonstrate the ability to accept genuine responsibility for leading, facilitating, and making decisions typical of those made by educational leaders. The experience(s) should provide interns with substantial responsibilities that increase over time in amount and complexity and involve direct interaction and involvement with staff, students, parents, and community leaders Substantial: b) Each candidate should have a minimum of six months (or equivalent, see note below) of fulltime internship experience Sustained: a) Candidates participate in planned intern activities during the entire course of the program, including an extended period of time near the conclusion of the program to allow for candidate application of knowledge and skills on a full-time basis Standards-based: a) Candidates apply skills and knowledge articulated in the [ELCC] standards as well as state and local standards for educational leaders. Evidence of Meeting Standard The embedded field experiences are designed to provide candidates with initial opportunities to interact with school and district leaders and staff members and community leaders. The internship is the summative field experience providing candidates with authentic leadership roles and responsibility for complex projects. The embedded field experiences extend typically over a year and half. The internship requires 135 hours (i.e., 9 to 12 per week), and four to five months or more. The Educational Leadership Department believes the application of learning in real life settings is essential to leadership preparation; hence, field experiences are embedded in many of the courses leading to certification. A brief description of each embedded field experience and the internship is described in Section 1. As noted above, the embedded field experiences extend typically over a year and half. The internship requires 135 hours (i.e., 9 to 12 per week), and four to five months or more. Each course syllabus, embedded field experience, and the internship are aligned with ELCC and as appropriate Arizona Department of Education standards. These standards are explicitly identified, and candidates are asked to reflect upon them and use them as a framework for a variety of projects, including their internship Standards-based: b) Experiences are As noted in Section 1, the embedded field experience in 1

19 designed to accommodate candidates individual needs Real Settings: a) Candidates experiences occur in multiple district administrator settings that allow for the demonstration of a wide range of relevant knowledge and skills Real Settings: b) Candidates experiences include work with appropriate community organizations parent groups, and school boards. 7.5 Planned and Guided Cooperatively: a) Candidates experiences are planned cooperatively by the individual, the site supervisor, and institution personnel to provide inclusion of appropriate opportunities to apply skills, knowledge, and research contained in the standards. These three individuals work together to meet candidate and program needs. 7.5 Planned and Guided Cooperatively: b) Mentors are provided training to guide the candidate during the intern experience. 7.6 Credit: a) Candidates earn graduate credit for their intern experience. EDL 600 is an individualized assignment that candidates design based upon their professional development plan. For the internship, candidates are encouraged to seek a setting that is different and that complements their previous professional experience. Candidates participate in a variety of field experiences. Some of the settings for these experiences include elementary, high school, and unified school districts and school board meetings. One of the embedded field experiences in EDL 762 (i.e., Best Practices of Curriculum Leadership Process) as well as the internship involves candidates in interactions and work with community organizations and local businesses. Candidates take a leadership role in planning and implementing their Internship. Candidates are encouraged to choose a setting that will provide them with an experience that is different from and will complement their professional background. Candidates work directly with an experienced, certificated school principal or superintendent and a university supervisor to plan the activities that will be undertaken and meet regularly throughout the Internship experience to monitor progress and discuss accomplishments as well as any issues. The Internship Director holds training sessions and interacts regularly with the certified administrators who serve as the on-site mentors and the university supervisors. EDL 796 is the Internship course. The internship is a 3 credit hour course, and is designed to be completed in one semester (fall or spring only). In addition, the embedded field experiences that are described in Section 1 are all incorporated in required graduate level courses. 3. Analysis of Data Findings: A summary table aggregating data collected regarding Interns competencies related to ELCC standards for Fall 2007 is documented as part of question 5 (below). The data is disaggregated by the following locations: Phoenix, Tucson, and Rural. The data table show 100% of candidates or more met or exceeded (a score of 3 or 4) all standards, except standard 4.1. One candidate scored a 2 for item C.1. As noted earlier, this was the first time the Internship Assessment instrument was implemented. The 3 candidates that we did not receive data for prior to the report deadline did pass the Internship based on transcript data. The Internship Director is in the process of following up with the university supervisors who have not submitted their forms yet. The scores candidates earned on the Internship Assessment form are consistent with the pass rate for the AEPA: Administrator Certification Superintendent Examination. 4. Interpretation of How Data Provides Evidence for Meeting Standards: As mentioned in the analysis, overall the data indicates that 100% of candidates at all sites met or 2

20 exceeded all standards contained in this assessment with the exception of standard 4.1. The following list summarizes the percentage of candidates meeting or exceeding each standard addressed by Assessment #4: Professional Knowledge, Skills & Dispositions: Assessment of Internships. ELCC 1.1 Develop a District Vision of Learning, 100% ELCC 1.2 Articulate a District Vision of Learning, 100% ELCC 1.3 Implement a District Vision of Learning, 100% ELCC 1.4 Steward a District Vision of Learning, 100% ELCC 1.5 Promote Community Involvement in District Vision, 100% ELCC 2.1 Promote a Positive District Culture, 100% ELCC 2.2 Provide Effective Instructional Program, 100% ELCC 2.3 Apply Best Practice to Student Learning, 100% ELCC 2.4 Design Comprehensive Professional Growth Plans, 100% ELCC 3.1 Manage the Organization, 100% ELCC 3.2 Manage the Operations, 100% ELCC 3.3 Manage the Resources, 100% ELCC 4.1 Collaborate with Families and Other Community Members, 83% ELCC 4.2 Respond to Community Interests and Needs, 100% ELCC 4.3 Mobilize Community Resources, 100% ELCC 5.1 Acts with Integrity, 100% ELCC 5.2 Acts Fairly, 100% ELCC 5.3 Acts Ethically, 100% ELCC 6.1 Understand the Larger Educational Context, 100% ELCC 6.2 Respond to the Larger Educational Context, 100% ELCC 6.3 Influence the Larger Educational Context, 100% 5. Assessment Documentation a) Description of Assessment Tool(s)/Assignment(s): EDL 796 Internship Assessment: The Internship Assessment serves as a summative evaluation instrument. Candidates meet with their university supervisor throughout the course of their internship to show documentation of their work through their portfolio and log as well as to discuss issues the candidate is encountering. The Internship Assessment is used to evaluate the candidate s work related to all ELCC standards. b) Scoring Guide(s): EDL 796 Internship Assessment: 3

21 Northern Arizona University College of Education / Educational Leadership Department EDL 796 Internship Assessment Candidate: Student ID#: Certification Level: Superintendent Date Rubric 4pts. Target/Exemplary: Exceeds expectation for standards; consistently superior performance; needs little, if any, support/guidance. 3pts. Acceptable: Meets expectations for standards: is confident and consistently meets the standard. 2pts. Below Expectations: Meets some expectation for standards; needs support to perform at acceptable level. 1pt. Unacceptable: Does not meet standards / Needs further development. Evidence Code: P. = Portfolio J. = Journal/Log R. = Report or Assignment Number for Standard Professional Standards for District Administration: Evidence Code Points (a) Leadership 1. Took a leadership role in planning and implementing intern activities Can articulate the district vision and mission. (ELCC 1.2) Reviews, evaluates, and revises instructional programs on the basis of sound information, data analysis, and can apply principles of effective instruction. (ELCC 2.2) 4. Facilitates activities that promote a positive district culture. (ELCC 2.1) Promotes activities that honor academic excellence. (ELCC 2.2) Interacts with teachers in preparing and implementing professional development plans that are related to improved instruction and learning. (ELCC 2.4) 7. Helps staff align their curriculum with the state s curriculum standards Understands principles of mentoring and provides new teachers with best practices to improve student learning. (ELCC 2.3) 9. Encourages experimentation and rigorous evaluation of new pedagogical approaches. (ELCC 2.4) 10. Plans effectively for the implementation of policy decisions, taking into account unanticipated consequences and costs. (ELCC 3.3) Overall Rating for Standard (a) (b) Administration 1. Demonstrates effective oral and written communication skills. (ELCC 1.2) Uses effective methods of personnel selection, supervision, and evaluation (ELCC 3.2) 3. Demonstrates a respect for the rights of others with regard to confidentiality, dignity, and engages in honest interactions. (ELCC 5.1) 4. Practices relevant fiscal management policies and procedures. (ELCC 3.1) Demonstrates the ability to combine impartiality, sensitivity to student diversity, and ethical considerations in the interactions with others. (ELCC 5.2) 6. Makes and explains decisions based on ethical and legal principles (ELCC 5.3) Uses problem solving skills and operational planning in use of resource allocation can apply and assess technologies for district management, business procedures, and scheduling. (ELCC 3.3) Overall Rating for Standard (b) Professional Standards for District Administration: 4

22 (c) Equity 1. Assures presence and quality of educational programs that address the needs, interests, and abilities of faculty and students. (ELCC 2.1) 2. Provides programs or activities that help all faculty and students acquire a positive civic identity and sees them as an integral member of our civic communities. (ELCC 2.1) 3. Fosters understanding that effort is a key factor in achievement. (ELCC 2.3) Helps all faculty and staff see themselves as unique individuals responsible for their own actions. (ELCC 2.3) 5. Assures high academic expectations for all students. (ELCC 2.3) Holds teachers, staff, and self to high standards of performance and behavior (ELCC 2.4) Overall Rating for Standard (c) (d) Community Relationships 1. Involves families and other community members in developing the vision, goals, improvement plans, and programs for the district. (ELCC 4.1) 2. Promotes partnerships among staff, families, the business and other community groups and uses community resources to enhance instruction. (ELCC 4.1-3) Overall Rating for Standard (d) (e) Professional Responsibilities 1. Meets his or her legal and moral responsibilities. (ELCC 5.1-3) Maintains interest in current developments in the professional discipline (ELCC 6.1) 3. Knows educational principles and research that promote sound practice (ELCC 2.3) 4. Studies educational research of relevance to professional responsibilities (ELCC 6.1) 5. Is familiar with the range of instructional programs and policies that can promote academic learning for all students. (ELCC 6.2-3) 6. Is familiar with the range of faculty, family, and student support services (ELCC 4.3) 7. Is familiar with the range of professional training programs and providers for prospective teachers. (ELCC 2.4) 8. Knows multiple strategies to assess teacher s performance. (ELCC 2.2) Understands approaches to organizational change, school-based management, and district restructuring. (ELCC 1.1-5) 10. Understands federal, state, and municipal laws and regulations affecting schools, staff, and students, including laws on disability, civil rights and responsibilities, issues of liability, and the requirements of due process. (ELCC 6.1) 11. Demonstrates current issues in American education. (ELCC ) Overall Rating for Standard (e) Meeting Dates: 1 st 2 nd 3 rd District Supervisor: Date: NAU University Supervisor: Date: 5

23 c) Aggregate Candidate Data: The following tables include data collected from Internship candidates (EDL 796) during Fall Initial implementation of the instrument occurred during this term. Prior to the report deadline, data was collected for 5 candidates out of 8 enrolled (63% data collection rate, to date). The Internship Program Coordinator is following up with university supervisors to ensure data is collected for all candidates. EDL 796, Internship Phoenix 67% Data Collection Rate (One additional candidate passed the Internship, but data was not received prior to the report deadline.) % Meeting N 1 % 2 % 3 % 4 % Standard Leadership L % 0 0% 1 50% 1 50% 100% L2 (ELCC 1.2) 2 0 0% 0 0% 0 0% 2 100% 100% L3 (ELCC 2.2) 2 0 0% 0 0% 1 50% 1 50% 100% L4 (ELCC 2.1) 2 0 0% 0 0% 1 50% 1 50% 100% L5 (ELCC 2.2) 2 0 0% 0 0% 0 0% 2 100% 100% L6 (ELCC 2.4) 2 0 0% 0 0% 2 100% 0 0% 100% L % 0 0% 2 100% 0 0% 100% L8 (ELCC 2.3) 2 0 0% 0 0% 2 100% 0 0% 100% L % 0 0% 2 100% 0 0% 100% L10 (ELCC 3.3) 2 0 0% 0 0% 0 0% 2 100% 100% Overall 2 0 0% 0 0% 1 50% 1 50% 100% Administration A1 (ELCC 1.2) 2 0 0% 0 0% 0 0% 2 100% 100% A2 (ELCC 3.2) 2 0 0% 0 0% 1 50% 1 50% 100% A3 (ELCC 5.1) 2 0 0% 0 0% 0 0% 2 100% 100% A4 (ELCC 3.1) 2 0 0% 0 0% 1 50% 1 50% 100% A5 (ELCC 5.2) 2 0 0% 0 0% 0 0% 2 100% 100% A6 (ELCC 5.3) 2 0 0% 0 0% 0 0% 2 100% 100% A7 (ELCC 3.3) 2 0 0% 0 0% 2 100% 0 0% 100% Overall 2 0 0% 0 0% 0 0% 2 100% 100% 6

24 Equity E1 (ELCC 2.1) 2 0 0% 0 0% 2 100% 0 0% 100% E2 (ELCC 2.1) 2 0 0% 0 0% 1 50% 1 50% 100% E3 (ELCC 2.3) 2 0 0% 0 0% 0 0% 2 100% 100% E4 (ELCC 2.3) 2 0 0% 0 0% 0 0% 2 100% 100% E5 (ELCC 2.3) 2 0 0% 0 0% 0 0% 2 100% 100% E6 (ELCC 2.4) 2 0 0% 0 0% 0 0% 2 100% 100% Overall 2 0 0% 0 0% 0 0% 2 100% 100% Community C1 (ELCC 4.1) 2 0 0% 1 50% 0 0% 1 50% 50% C2 (ELCC ) 2 0 0% 0 0% 1 50% 1 50% 100% Overall Score 2 0 0% 0 0% 1 50% 1 50% 100% Professional Responsibilities P1 (ELCC ) 2 0 0% 0 0% 0 0% 2 100% 100% P2 (ELCC 6.1) 2 0 0% 0 0% 1 50% 1 50% 100% P3 (ELCC 2.3) 2 0 0% 0 0% 0 0% 2 100% 100% P4 (ELCC 6.1) 2 0 0% 0 0% 1 50% 1 50% 100% P5 (ELCC ) 2 0 0% 0 0% 0 0% 2 100% 100% P6 (ELCC 4.3) 2 0 0% 0 0% 0 0% 2 100% 100% P7 (ELCC 2.4) 2 0 0% 0 0% 2 100% 0 0% 100% P8 (ELCC 2.2) 2 0 0% 0 0% 2 100% 0 0% 100% P9 (ELCC ) 2 0 0% 0 0% 2 100% 0 0% 100% P10 (ELCC 6.1) 2 0 0% 0 0% 1 50% 1 50% 100% P11 (ELCC 6.1) 2 0 0% 0 0% 2 100% 0 0% 100% Overall Score 2 0 0% 0 0% 1 50% 1 50% 100% Overall Score % 0 0% 6 35% 11 65% 100% 7

25 EDL 796, Internship Tucson % Meeting Standard N 1 % 2 % 3 % 4 % Leadership L % 0 0% 0 0% 1 100% 100% L2 (ELCC 1.2) 1 0 0% 0 0% 0 0% 1 100% 100% L3 (ELCC 2.2) 1 0 0% 0 0% 0 0% 1 100% 100% L4 (ELCC 2.1) 1 0 0% 0 0% 0 0% 1 100% 100% L5 (ELCC 2.2) 1 0 0% 0 0% 0 0% 1 100% 100% L6 (ELCC 2.4) 1 0 0% 0 0% 0 0% 1 100% 100% L % 0 0% 0 0% 1 100% 100% L8 (ELCC 2.3) 1 0 0% 0 0% 0 0% 1 100% 100% L % 0 0% 0 0% 1 100% 100% L10 (ELCC 3.3) 1 0 0% 0 0% 0 0% 1 100% 100% Overall 1 0 0% 0 0% 0 0% 1 100% 100% Administration A1 (ELCC 1.2) 1 0 0% 0 0% 0 0% 1 100% 100% A2 (ELCC 3.2) 1 0 0% 0 0% 0 0% 1 100% 100% A3 (ELCC 5.1) 1 0 0% 0 0% 0 0% 1 100% 100% A4 (ELCC 3.1) 1 0 0% 0 0% 0 0% 1 100% 100% A5 (ELCC 5.2) 1 0 0% 0 0% 0 0% 1 100% 100% A6 (ELCC 5.3) 1 0 0% 0 0% 0 0% 1 100% 100% A7 (ELCC 3.3) 1 0 0% 0 0% 0 0% 1 100% 100% Overall 1 0 0% 0 0% 0 0% 1 100% 100% Equity E1 (ELCC 2.1) 1 0 0% 0 0% 0 0% 1 100% 100% E2 (ELCC 2.1) 1 0 0% 0 0% 0 0% 1 100% 100% E3 (ELCC 2.3) 1 0 0% 0 0% 0 0% 1 100% 100% E4 (ELCC 2.3) 1 0 0% 0 0% 0 0% 1 100% 100% E5 (ELCC 2.3) 1 0 0% 0 0% 0 0% 1 100% 100% 8

26 E6 (ELCC 2.4) 1 0 0% 0 0% 0 0% 1 100% 100% Overall 1 0 0% 0 0% 0 0% 1 100% 100% Community C1 (ELCC 4.1) 1 0 0% 0 0% 0 0% 1 100% 100% C2 (ELCC ) 1 0 0% 0 0% 0 0% 1 100% 100% Overall Score 1 0 0% 0 0% 0 0% 1 100% 100% Professional Responsibilities P1 (ELCC ) 1 0 0% 0 0% 0 0% 1 100% 100% P2 (ELCC 6.1) 1 0 0% 0 0% 0 0% 1 100% 100% P3 (ELCC 2.3) 1 0 0% 0 0% 0 0% 1 100% 100% P4 (ELCC 6.1) 1 0 0% 0 0% 0 0% 1 100% 100% P5 (ELCC ) 1 0 0% 0 0% 0 0% 1 100% 100% P6 (ELCC 4.3) 1 0 0% 0 0% 0 0% 1 100% 100% P7 (ELCC 2.4) 1 0 0% 0 0% 0 0% 1 100% 100% P8 (ELCC 2.2) 1 0 0% 0 0% 0 0% 1 100% 100% P9 (ELCC ) 1 0 0% 0 0% 0 0% 1 100% 100% P10 (ELCC 6.1) 1 0 0% 0 0% 0 0% 1 100% 100% P11 (ELCC 6.1) 1 0 0% 0 0% 0 0% 1 100% 100% Overall Score 1 0 0% 0 0% 0 0% 1 100% 100% EDL 796, Internship Rural 50% Data Collection Rate (Two additional candidates passed the Internship, but data was not received prior to the report deadline.) % Meeting N 1 % 2 % 3 % 4 % Standard Leadership L % 0 0% 0 0% 2 100% 100% L2 (ELCC 1.2) 2 0 0% 0 0% 1 50% 1 50% 100% L3 (ELCC 2.2) 2 0 0% 0 0% 0 0% 2 100% 100% L4 (ELCC 2.1) 2 0 0% 0 0% 0 0% 2 100% 100% 9