Assessing Academic Preparation And Performance In Education Administration. Donald G. Coleman California State University, Fresno

Transcription

1 Assessing Academic Preparation And Performance In Education Administration Donald G. Coleman California State University, Fresno Michelle Symes Los Banos High School R.C. Adams California State University, Fresno We are grateful for the loyal support of our subscribers and members that make this website possible. National FORUM Journals is one of the few professional journals that provide free access to published articles. Your contribution of any amount will ensure continued free access to the published articles of our family of journals. Donations may be sent to: National FORUM Journals, 4000 Lock Lane Suite 9/KL, Lake Charles, LA 70605

2 Abstract This article addresses the relevance of university course work in Education Administration to skills required on the job. Presented herein are the results of a study correlating scores achieved by 18 candidates on a faculty prepared knowledge-based test covering content from 16 university courses to results achieved by these candidates across the 13 administrative skills measure by the National Association of Elementary School Principals (NAESP) assessment center. The relevance of university course work in Education Administration to skills required on the job has been argued by academicians and practitioners for years. 18 candidates on a faculty prepared knowledge-based test covering content from 16 university courses to results achieved by these candidates across the 13 administrative skills measured by the National Association of Elementary School Principals (NAESP) assessment center. Recent changes in the California university requirements for preparing education administrators divides the process into distinct halves. The first half requires prospective administrators to complete requirements for a 24-unit Preliminary Administrative Services Credential. When done, graduates are then awarded a certificate of completion by the Commission on Teacher Credentialing (CTC). This certificate signifies to employers that program requirements at a university have been completed and the individual is eligible to receive the formal credential, once employed as an administrator. Once employed, therefore, candidates must apply to CTC for the initial credential; when it is issued, a five-year, second half clock begins ticking, requiring the individual to complete an additional 24-unit university-based program, this time for the Professional Administrative Services Credential. Before allowing candidates to begin the second program of studies, however, university personnel are required to conduct a formal assessment of the candidates strengths and weaknesses and to use the assessment data to develop an Individualized Induction Plan (IIP). The IIP parallels the familiar Individualized Education Plan used in Special Education. The individual s IIP provides evaluative criteria for use in developing a professional-growth agenda while enrolled in the advanced credential program. This article details the assessment model implemented at California State University, Fresno (CSUF) and reports findings from a series of correlation studies conducted on data collected from the first 18 candidates admitted to this revised program. A statistical analysis was completed correlating results from a knowledge-base test covering 16 academic courses with results achieved on the Professional Development Inventory (PDI), an assessment process offered by the NAESP. The study was conducted to evaluate the mechanics and results of this newly implemented process and to examine the relationship between required courses and success in the assessment center. Background The changes in the California Education Administration program standards require newly appointed administrators to enroll in a university approved program leading to the Professional Administrative Services Credential. Each university offering the credential is required to conduct an assessment and to establish an IIP for candidates for the 24-unit program. The advanced credential, conceptually, therefore, provides a personalized professional development agenda for newly appointed administrators to follow in developing essential knowledge and skills identified as deficient. Each candidate for the credential, along with a faculty member and a school district representative, are required to collaborate on developing the IIP. A personal mentor is also selected and becomes a valued partner later in the professional development process.

3 The Education Administration faculty member at CSUF, along with an advisory committee of local superintendents, adopted a comprehensive evaluation system for meeting the standards, requiring for each program candidate: (a) scores achieved on the PDI; (b) scores achieved on a comprehensive university knowledge-base test; and (c) results from the most recent district supervisors performance review. The results from these three assessments were proposed to be used in developing a personalized professional growth plan. Methodology Eighteen students have now provided the three sets of information. Credential program faculty members are currently examining the data collected, along with the supervisor evaluations, as a means of assisting candidates in developing the IIP. The following describes the assessment instruments, research procedures, and preliminary findings. Instrumentation. The PDI is a one-day assessment center established by NAESP, whereby individuals are assessed on 13 essential skills after completing a number of administrative simulations. The skills assessed include Planning, Organizing, Problem Solving, Creativity, Decision Making, Systems Analysis, Vision, Communications, Instructional Leadership, Group Leadership and Team Building, Climate Development, Moral Responsibility, and Instructional Analysis and Supervision. Each of the skills is assessed at three different points in 12 sets of simulations. The 12 simulations require candidates to complete a lesson analysis; debrief the lesson with the teacher; handle encounters with students, teachers, and others; resolve a serious conflict case; and process several in-basket items. A team of trained assessors later evaluate video tapes, prepared by those being assessed, while engaged in interactive situations, as well as other materials developed by the candidates. When complete, the assessor materials are sent to NAESP for scoring. A 20-page summary is returned in approximately one week to 10 days. The detailed summary of results includes a profile of individual scores compared to those achieved by experienced principals. The Knowledge-base Assessment Test was created by the members teaching 16 different courses in the Education Administration program. The courses include eight Preliminary Administrative Credential courses: (a) Educational Statistics, (b) Research Methods, (c) Advanced Educational Psychology, (d) Management, (e) Leadership, (f) Instructional Supervision and Improvements, (g) Site-based Leadership, and (h) Curriculum Development and Evaluation. Eight Professional Credential courses are also included: (a) School Law and Policy, (b) School-Community Relations, (c) Education Finance, (d) Personnel, (e) Economics of Education, (f) Facilities, (g) Systems Analysis and Design, and (h) Special Education. Each faculty member prepared a list of 15 multiple choice items believed to represent a core of knowledge for each course and important field-based practitioners. A discriminant analysis of the items within each discipline identified a representative sample of 10 questions from each. The final form of the instrument, therefore, consists of 160 items from the university knowledge base. To simplify collecting the immediate supervisor s evaluation, the candidates were advised to use the evaluative instrument required by their respective districts. The differing formats, however, did not provide a convenient means for statistically comparing or utilizing the results from the supervisors in this study. Population The candidates for the study were 18 recent applicants for admission to the redesigned Professional Administrative credential program. Two candidates were central office administrators, two were principals, seven were assistant principals, and the balance served in various other administrative positions. The candidates were invited to participate in a prototype assessment program voluntarily to provide the faculty the opportunity to observe assessments employed before formally requiring the process to all future

4 admittees. The results provided opportunity for the faculty to examine results and to study relationships among the two data sets. Data Collection Procedures All 18 candidates attended the one-day PDI, 12 at one time and the balance at a second time. The assessment center required approximately 10 hours to complete. The knowledge test was administered at a separate time and required approximately three hours to complete. While the knowledge base test measured theoretical, legal, and political concepts found in courses and the assessment center measured application of administrative skills, some significant correlations between the two were expected because several courses address skill application as well as theoretical conceptualization. Data Analysis Evaluation of the data included calculating standard descriptive statistics and Pearson s Product-Moment Correlation coefficients assessing the relationships among the eight scores achieved on each of the Preliminary Credential and the Professional Credential courses and the 13 skill areas or eight knowledgebase areas contained on the PDI. The purposes of the study included: (a) examining the variance among candidate responses on the assessment instruments, and (b) studying the relationships among scores achieved by respondents. The study produced the following findings: Findings 1. Means for the knowledge-base test for the Preliminary Credential Courses tended toward the middle of the scoring range, averaging from 50% of the items correct, for Managing Educational Organizations, to 64% for Site Leadership. Means were lower for the Professional Credential courses than for the Preliminary Credential courses, as expected, in that all had completed the former but few had completed any of the latter. Professional Credential scores ranged from 28%, for School Law, to 60%, for both Public Relations and Economics. Items were not weighted for difficulty and no factor was used to account for examinees who had not completed particular courses. But, standard deviations, like means, had a limited range, so the variance within and between data sets from tests for the two credentials appears to be fairly consistent. PDI means clustered about the natural mean, 2.0, on the 3-point scale and, again, the variance appeared rather consistent. However, the means may be illusive in that the PDI norms were established with experienced practicing administrators who produced negatively skewed data distributions and means approximating When these candidates received their scores from NAESP, they could see both their raw scores and percentile ranks comparing them to the norming group. The typical mean for this group 2.0 approximates the 30th percentile. 2. These candidates achieved a number of significant correlation coefficients between knowledge-based test scores on the Preliminary credential courses and PDI skill scores (See Table 1). Among the PDI skills, only Planning and Moral Responsibility did not correlate with at least one knowledge-based test. Decisiveness, Organization, and Instructional Analysis correlated significantly with one course each. The other skills correlated with two or more of the knowledge-based tests. Conversely, the Leadership and Site Leadership courses were not correlates of any of the skills.

5 3. PDI skills and knowledge-based test scores for the Professional Credential courses produced several significant correlation coefficient (See Table 2). Each of the PDI skills correlated significantly with at least one course knowledge base, and conversely, each of the course knowledge bases correlated significantly with at least one of the PDI skills. 4. Composite scores over Preliminary Credential courses and PDI skills correlated significantly, r(17) =.53, p =.002. Therefore, course-based knowledge and performance skills are related, at both levels Preliminary and Professional. Discussion The linkage between university course work and skills required on the job has long been debated among academics and practitioners. This study documents that knowledge attained in university courses correlates significantly to on-the-job performance, as measured by the PDI. The PDI received high marks from attendees on the similarity between the simulated activities and situations occurring on the job. The results of the study provided excellent feedback to students on their preparation for positions in Education Administration. Scores on the knowledge-based test provided candidates to the program relative performance of each individual to the group, and the group scores provided a data bank for comparing candidates for admission in the future. Knowing that success on the knowledge-based test is related to success on the PDI and success on the PDI relates to on-the-performance equips each student with individualized information to consider when preparing the IIP. Similarly, feedback from the PDI provided a profile of strengths and weaknesses to each individual on administrative skills compared to experienced practitioners on essential skills. Given these two data sets coupled with the immediate supervisors most recent evaluation, the candidates for the credential have excellent information to tailor a program to meet professional development needs. Additionally, the assessment process explained herein provides the faculty excellent information about the program. Not only can the faculty see the relationship between student scores on course content and skill application, but the data provide insights into retention of information maintained from classes and evidence regarding student application of skills supposedly learned in various courses. In places where skills fail to correlate significantly with course, or inversely, where courses do not correlate significantly with the skill areas, faculty members have supporting data for program redesign and improvement. By serving as PDI assessors, faculty members have had opportunity to see the theory of use practiced by students in simulated situations after having completed courses related to the Preliminary Credential. After viewing data from the first few candidates, faculty members have already modified content in selected courses where student deficiencies became apparent. The results from the group of 18 indicated that the more students know regarding the course content, the better they achieve on the PDI. Presumably, therefore, the more graduates know, the better they achieve on the job. The study could easily be replicated or tailored to needs by faculties at other universities. able 1 earson Product-Moment Correlation Coefficients Between Assessment Center Skills and Preliminary Credential Courses Statistics Research Management Leadership Supervision Site Leadership Curriculum Psychology limate.40* * ommunication * *.35 reativity.20.46* * ecisiveness *

6 roup Leadership.50*.45* * stru Leadership.40*.64* * stru Analysis * oral Respon rganizing *.06 lanning roblem Solving.27.45* * * ys Analysis * * ision.33.49* * able 2 earson Product-Moment Correlation Coefficients Between Assessment Center Skills and Professional Credential Courses Law Public Relations Finance Personnel Economics Buildings Special Ed. Sys. Analysis limate * *.38.64* ommunication *.36 reativity.23.43* *.40* ecisiveness * * roup Leadership * *.36.64* stru Leadership *.49* *.64* stru Analysis * oral Respon * * rganizing.46* * lanning.41* * roblem Solving * ys Analysis * * ision * *.56*