B.S. Psychology 1 Annual Assessment Summary B.S. in Psychology Part One: Summary and Analysis of Assessment Results In Fall 2010, we implemented two changes to our curriculum based on earlier years results on the Educational Testing Service (ETS) Major Field Test (MFT): (1) We instituted a requirement of History and Systems of Psychology; and (2) we instituted a requirement that students take one of three psychological science courses (either Biopsychology, Cognitive Psychology, or Theories of Learning). Data from the Fall 2010 semester administration of the MFT are provided below (Tables 1 and 2). The MFT was not administered in the Spring 2011 semester because the test was in the process of being revised, and scores would not be available until mid summer, precluding them from being used as part of the course grade for a spring semester class. Table 1. GSW students performance on the ETS compared to national averages, Spring 2006 Fall 2010. Includes comparison of the means for the four major subscore areas as well as the overall means. Overall Learning, Cognition, Language, Memory, and Thinking Perception, Sensory, Physiology, Comparative, and Ethology Clinical, Abnormal, and Personality Developmental and Social National mean 156.2 56.0 56.9 55.9 56.1 students (pre Fall 2010) students (Fall 2010) 151.4 51.4 51.2 51.2 51.3 157.0 57.0 57.0 58.0 58.0 Table 2. GSW students performance on the MFT assessment indicators, Spring 2006 Fall 2010. Data are represented as the mean percent correct answers in that content area on the MFT. Memory and Thinking Sensory and Physiology Developmental Clinical and Abnormal Social Measurement and Methodology GSW students (pre Fall 2010) 43 34 43 62 58 47 students (Fall 2010) 51 38 46 63 69 52 Because the MFT has only been given once since the implementation of these curriculum changes, we only have one semester s worth of data for analysis and comparison. Furthermore, we believe that the students who took the MFT in the Fall 2010 semester are students who were enrolled in the degree
B.S. Psychology 2 program prior to the institution of these new requirements, in which case their data cannot be used to evaluate the success of the program changes. The MFT will be administered again on a regular basis starting in Fall 2011, which will allow us to better evaluate the effects of these curriculum changes. In keeping with the Action Plan developed last year, we also assessed our Internship course by sending surveys to alumni and current students who had participated in the course. Because we did not initially ask students to provide any potentially identifying information, we do not have separate data for B.A. and B.S. students; however, the survey has been amended to include a question about this, so future respondents can provide this information. Approximately 90 surveys have been distributed, but only about 10% have been returned. In general, these surveys indicate that Internship students are complimentary about their experience; for example, 8/9 (89%) of respondents indicated that they were either Very or Extremely satisfied with their Internship experience, and all of the respondents reported that they either had no problems at their internship site or that their faculty supervisor was Excellent at resolving any problems that arose. However, these preliminary results have given us some ideas for improvement. For example, 2/9 (22%) of respondents reported that they found keeping a daily journal to have only Fair value, though 5/9 (55%) stated that it had Excellent value to them. We will need to look at the procedure and requirements of this journal to try to make it as valuable as possible to our students. Additionally, 5/9 (56%) of respondents indicated that they would find on site visits from their faculty internship supervisor to have Excellent value. As faculty do not currently make on site visits, we will look more closely at instituting this component after we review more data. On the other hand, only 2/9 (22%) of respondents reported that they considered the opportunity to interact with students doing internships at other sites to have Excellent value; again, we do not currently have this type of interaction as a component of the internship course but were considering instituting it, and these results make us question whether or not this is an avenue we should pursue. Despite the value of these early results, we are not satisfied with the low response rate so far and want to collect data from more students before we begin to institute any changes to the Internship course. Below is a table that summarizes our assessment results for the Student Learning Outcomes (SLOs) we examined this year. For the MFT, students are considered to have met expectations if they scored within one standard deviation of the national mean (in either direction) on the overall score or subscores. Students who scored above one standard deviation of the national mean are considered to have exceeded expectations; students who scored below one standard deviation of the national mean are considered not to have met expectations. For course grades, students with a grade of C are considered to have met the standard; students with a grade of A or B are considered to have exceeded the standard; and students with a grade of D or F are considered not to have met the standard. Because our assessment activities related to the Internship course are aimed specifically at assessing aspects of the course that students either did or did not find valuable, we cannot report data on the number of students who met, did not meet, or exceeded expectations. In this case, the course is being assessed, not the students. In general, our data indicate that the majority of our students are either meeting or exceeding expectations on the outcomes measured this year.
B.S. Psychology 3 Table 3. Percentages of students who either met, did not meet, or exceeded expectations on the outcome measures used in the 2010 2011 academic year. Program Learning Outcomes Outcome Measures Number Assessed % Did Not Meet % Met % Exceeded SLO #1. Students will demonstrate knowledge and understanding of the major concepts, theoretical perspectives, empirical findings, and historical trends in psychology. SLO #2. Students will understand and apply basic research methods in psychology, including research design, data analysis, and interpretation. SLO #5. Students will emerge from the major with realistic ideas about (overall) (Learning, Cognition, Language, Memory, & Thinking) (Perception, Sensory, Physiology, Comparative, & Ethology) (Clinical, Abnormal, & Personality) (Developmental & Social) Seminar in Psychology History & Systems of Psychology Sociological Statistics* Experimental Psychology Survey given to alumni and 2010 2011 15 0% 93% 7% 15 7% 67% 26% 15 7% 80% 13% 15 0% 87% 13% 15 7% 80% 13% 35 3% 14% 83% 19 16% 42% 42% 43** 7% 28% 65% 43 4% 33% 63% 10 NA NA NA
B.S. Psychology 4 how to implement their psychological knowledge, skills, and values in occupational pursuits in a variety of settings. internship students *Students in the B.S. in Psychology program can take either Psychological Statistics or Sociological Statistics. Because Sociological Statistics was the only one of these courses taught during the 2010 2011 academic year, only data from that course are reported. **This total includes one student who took the class three times; she withdrew once, earned a D once, and earned a B on her third try. Her final grade is counted in the exceeded standard column because she did exceed the standard on the most recent time she took the test.
B.S. Psychology 5 Part 2. Action Plans New Action Plans 1. One action plan for the upcoming year involves completing our current survey of the Internship course in order to identify avenues for change and improvements to the course. Because of our (currently) low return rate for our internship survey, we need to collect more data from current and former students before instituting specific changes to the course. Our assessment plan also involves surveying former internship supervisors to identify changes they would like us to make to the course, with an eye toward improving the internship experience for our community partners as well as our students. This survey was sent out in the Fall 2011 semester, and several surveys have already been returned. In the Spring 2012 semester, the faculty will meet to review our data and implement any necessary changes to the Internship course. 2. A second action plan, related to the assessment data presented above, relates to the ETS MFT and the data we are able to obtain from it. Our original assessment plan relied on analysis of the six Assessment Indicators to provide data regarding student performance. However, with the standard reporting options available through ETS, these Assessment Indicator data are not available for individual students, but are only available at the cohort level. This prevents us from being able to use the Assessment Indicators to track whether students are meeting, exceeding, or not meeting expectations. We would like to explore the possibility of purchasing a Design Your Own Analysis report that would allow us to obtain and analyze these data, and we will explore possible funding sources for doing so. 3. A third plan is based on the grade distribution for the Seminar in Psychology course. While we are of course delighted that our students are doing so well in the course, we do not want the course to be insufficiently challenging. A key source of potential grade inflation in this course is that the students grade each other on oral presentations, and some of these grades may be overly charitable. It is possible that having the instructor grade these oral presentations instead may provide a more accurate measure of student performance on this dimension of the Seminar course, and instill a greater challenge for the students in the course. While it is too late to make this change for the Fall 2011 semester, this change will be implemented and examined in the Spring 2012 semester. 4. Also related to the assessment data presented above, it was noted that of the four required courses for the B.S. degree, the History and Systems of Psychology course had the highest percentage of students that did not meet expectations. Therefore, the instructor of this course is instituting a change intended to help more students meet or exceed performance expectations in this course. The change involves administering a pre test and post test (at the beginning and end of the semester, respectively) that asks students to identify key historical figures in psychology. Assessing this information at the beginning of the semester will help the instructor identify gaps in the students knowledge so that the course material can be more closely targeted to fill in those gaps. 5. A new goal for this year involves the assessment of critical thinking and skeptical inquiry skills in our students. This is one of our SLOs, to be measured partly through grades in required upper division courses and partly through an exit survey. A key aim for this academic year is to either create a survey or locate and use a standardized assessment of critical thinking/skeptical inquiry. Currently, assessment of this SLO involves indirect measures, but we would like to include a direct measure (i.e., a standardized instrument) if possible. Because this is a new goal for us, our data collection this year can be considered a baseline.