Annual Report on the Degree Program Assessment of Student Learning

Transcription

1 Annual Report on the Degree Program Assessment of Student Learning Name: Title: Ramona N. Mellott Professor and Chair Phone: (928) Academic Unit: Degree Program Covered in this Report: Educational Psychology Ph.D. Educational Psychology Counseling Psychology Annual Report Q.1. Based on the degree program assessment plan, describe the student learning assessment activities conducted in the past year (i.1., data collection, analysis, reporting, departmental discussion, etc). Please include who was involved in the assessment activities and when they occurred. Semester evaluations are conducted during on all first year doctoral students. All core Counseling Psychology faculty based in Flagstaff participated in the evaluation process. Data from these evaluations are incorporated into the report. The Outcome Assessment Plan was updated in August All Counseling Psychology Programs faculty participated in this process. Data was collected on doctoral graduates through an request for information in May Student satisfaction survey was conducted in Data was incorporated into self-study. Data was collected for this learning outcomes assessment by department chair who is also an active member of the Counseling Psychology core faculty. Counseling programs faculty reviewed the assessment data and made recommendations to curriculum. This was done in October Other curriculum changes that occurred for the past three years are a result of two internal APA self-studies. Faculty reviewed recommendations as part of these self-studies and made changes to program and curriculum. Recommendations and plan to continue with APA self-study occurred last Spring at EPS Faculty Meeting as a Whole, and the Counseling Programs Committee Meeting. The Fall 2005 retreat was also spent in reviewing program assessment and learning outcomes. Q.2. Describe key findings from the assessment process concerning student learning in the degree program. The report that follows provides evidence for each indicator of student learning related to learning outcomes for the program. Only graduates were considered in this assessment cycle. All students appear to show evidence of achieving all learning outcomes described in the plan. Students appear to have strong research skills and are able to use them effectively in their dissertation work. Internship supervisors are extremely positive about students clinical skills and their ability to function in their work settings. Our students are also effective as instructors and in teaching. Students are being trained as scientist-practitioners and the assessment reveals strong skills in the areas of research and practice. Q.3. Describe changes in curriculum, instruction, or use of resources that resulted from the above assessment process. Much of the changes in goals, curriculum and instruction have resulted from the internal self-study process. We now have clear practicum evaluation forms that help us assess all of the outcomes relevant to clinical practice. Since 2004, students are expected to participate in a research project in their first year that culminates in a presentation or publication. The research training and focus has become more formalized. Students will now be evaluated each semester so they can receive regular feedback on their performance in the program. Q.4. Identify assessment problems or difficulties encountered during the assessment process, and describe any proposed changes to the degree program assessment of student learning plan. Accessing all relevant data to assess student learning outcomes can be a challenge 1

2 Ensuring that all relevant data that is part of the outcome assessment plan resides in the student permanent file. Considering the use of an electronic portfolio to accumulate artifacts relevant to student professional development and student learning outcomes. Creating an advising sheet for advisors and students that are tied directly to learning outcomes which in turn could help students and advisors maintain continuous awareness of all learning outcomes and facilitate the achievement of these outcomes. 2

3 COLLEGE OF EDUCATION EDUCATIONAL PSYCHOLOGY Ph.D. Educational Psychology Counseling Psychology Outcome Assessment Report October 11, 2005 Data was collected on all students graduating from the Ph.D. Educational Psychology Counseling Psychology program in Fall 2004 and Spring A total of six students graduated during this time period. Additional data utilized for this report included the student satisfaction survey conducted in 2002 and the current APA internal selfstudy. Information Collected for Each Outcome and Corresponding Indicator: Student Learning Outcome 1. Students will demonstrate knowledge in core and foundational areas of psychology. (Related to Goal 1) Indicator 1.1. Performance in required courses in foundational areas History of Psych. (PSY 675), Measurement, (EPS 664), Physio./Social (PSY 650/635), Learning (PSY 620/621 & EPS 712), Development (EPS 610/611/612); Personality (PSY 610) Evidence: Students appear to do extremely well in courses required in foundational areas. An examination of final transcripts indicated that for 42 possible grades that could be earned in classes listed in this indicator, 40 were A s and only 2 B s. Indicator 1.2. Successful completion of preliminary exam. Evidence: All six students successfully completed the seven areas of the preliminary exam. Four of the six students passed all areas in their first attempt. In terms of this outcome, one student missed the section on Learning and had to do additional work/testing to remediate. Student Learning Outcome 2: Students will engage in ethical practice, professional behavior and demonstrate the knowledge and ability to work with a cultural and diverse population. (Related to Goal 2) Indicator 2.1. Practicum & Internship Evaluations Evidence: Ratings completed by practicum and internship supervisors on items related to ethical practice, professional behavior and the ability to work with a cultural and diverse population. Out of the 12 possible ratings, 9 of them were at the highest level. Two ratings were at the second highest level. Only one student received an average rating (3/5) on ethical and professional behaviors. When reviewing the comments related to that item, the rating was lowered from the mid-term rating due to a specific incident related to the student changing her mind (verbally) about completing a post doctoral position at the same setting and accepting a faculty position elsewhere. Indicator 2.2. Semester Evaluations Evidence: Professional and ethical conduct ratings on the semester evaluations were available for 3/6 students. Ratings indicate outstanding for 2/3 and excellent for 1/3. Indicator 2.3. Successful performance on ethics sections of preliminary and comprehensive examinations. Evidence: All six students successfully answered in their first attempt the ethics question on the preliminary exam and responded effectively to the multicultural component on the comprehensive examination. Indicator 2.4. Performance on following courses: Ethics (EPS 670), Multicultural (EPS 690) Evidence: Students successfully completed the two classes and earned mostly A s on these classes. Only one grade of B was recorded for EPS 690 (Multicultural Counseling) for 1 student. Student Learning Outcome 3: Students will demonstrate the ability to engage and apply appropriate research methods and statistics. (Related to Goal 3) Indicator 3.1 Participation in a first-year research project Evidence: This new indicator was established in Fall 2003; since all six students who graduated were admitted between prior to 2002 this was not an expectation. One student participated on her own with a faculty member on a research project. Indicator 3.2. Successful performance on research section of the preliminary and comprehensive exams as determined by panel of faculty reviewers. 3

4 Evidence: All students successfully completed the research section of the preliminary and comprehensive exams as determined by feedback from the panel of faculty reviewers. Two of the six students (33%) passed the research/stats question on the prelims at the second attempt. Indicator 3.3. Successful completion of proposal and dissertation as determined by dissertation committee. Evidence: Students successfully completed their proposal and dissertations as determined by a five member dissertation committee. Indicator 3.4. Dissertation Defense Evaluations by Grad. Rep. Evidence: An faculty representative selected by the graduate college outside of the College of Education attended 2/6 dissertation defense evaluations. Both students received a 5/5 on the item assessing a student s ability to design an appropriate study, explain chosen approach, and compare results to existing literature base. Indicator 3.5. Semester Evaluations Evidence: Semester evaluations were completed by counseling psychology core faculty during the student s first year in the program. Two of the three students received a 5/5 (outstanding), while one was rated 4/5 (excellent) on their research skills. Indicator 3.6. Internship Evaluations Evidence: Internship evaluations are conducted by supervisors on site using site forms. Items related to the utilization of research skills to inform clinical practice was used to assess students research skills. 4/6 students received the highest rating and 2/6 received the next highest rating on this category. Indicator 3.7. Performance in following statistics/research design courses: EPS 624, 625, 698, and 725 Evidence: Out of 24 possible grades to be earned for the 4 statistics/research design classes for six students, the majority of the grades received were As. Five B s were earned indicating an above average grasp of the knowledge. This grade was more likely to be earned in advanced statistics classes. Student Learning Outcome 4. Students will demonstrate the ability to apply best practice in counseling psychology practice in assessment, intervention, and counseling techniques. (Related to Goals 2 & 5) Indicator 4.1. Semester Evaluation Forms Evidence: Information from the semester evaluation forms were available for 3/6 students. All three students received excellent or outstanding ratings for their work with individuals, groups and their written form of communication (e.g., case study reports, assessment reports). Indicator 4.2. Successful performance on counseling psychology comprehensive exams as determined by panel of faculty reviewers. Evidence: The counseling psychology comprehensive exam provides information on a student s ability to synthesize and integrate information from a variety of sources and demonstrate their ability conceptualize the problem, plan a treatment and recommend evidence-based interventions. All students successfully passed the comprehensive exam on their first attempt. Indicator 4.3. Practicum and Internship Evaluations Evidence: Internship evaluations completed by a field supervisor, who is licensed as a psychologist, indicates that students received very favorable ratings on their skills in assessment, diagnosis, interventions and counseling. Student Learning Outcome 5. Students will demonstrate the ability to train and supervise others, and design effective prevention programs. (Related to Goal 4) Indicator 5.1. Successful performance on comprehensive exams on supervision question Evidence: Twenty-five percent of the grade on the counseling psychology comprehensive exam is related to Supervision. All students successfully completed this section and passed it. Indicator 5.2. Practicum and Internship Evaluations Evidence: While students provide supervision and training to master s students in counseling programs, no specific forms are used to assess them as supervisors. Students do receive a grade in EPS 750 (Counseling and Supervision). All students successfully passed the class. Students also conduct outreach and psycho educational training activities during internship and practicum which may not be specifically assessed. Indicator 5.3. Teaching Evaluations/Supervisor Evaluations 4

5 Evidence: Four of the six students choose to teach different classes and student ratings indicated that they performed at the satisfactory to the above average level. The majority of the student comments were positive. Ratings ranged from Student Learning Outcome Graduates will seek appropriate credentials and apply their expertise in a variety of settings with diverse persons having varied problems and needs (Related to Goal 5). Indicator 6.1. Results of student satisfaction surveys, student follow-up information gathered during self-study, and/or employer/supervisor surveys with a focus on credentials students have received post graduation, employment settings, and supervisor s estimation of overall preparedness of interns/employees. Evidence: A student satisfaction survey conducted in 2002 of doctoral students in EPS indicated that students are employed in a variety of settings as counseling psychology professionals. About 6 of the 10 respondents indicated affiliation with a university on a full-time or part-time basis. Four graduates indicated administrative positions in universities and/or mental health agencies. Two students had obtained psychology licensure and one indicated a plan to do so. An examination of the data compiled for the APA self-study indicates that there were 22 students who graduated between Seven of them had already obtained licensure as a psychologist, another ten were in the process of seeking licensure and completing post-doctoral requirements. No data was available for two graduates, and the remaining two had no immediate plans as they were in academic positions making it difficult to meet post-doctoral requirements and maintain a full academic load. 5