Program Outcome Assessment Report Criminal Justice Program Department of Behavioral Sciences Southeastern Oklahoma State University 2010-2011
Program Assessment Report Evaluation 2010-2011 Report Criminal Justice Table of Contents I. Program Identification 3 II. Report Author.3 III. Program Mission Statement, Goals, and Learning Objectives...3 IV. Identification and Assessment of Learning Outcomes 3 11 V. Program Modifications...11 VI. Program IETV.....11, 12 VII. Faculty Contributions..12 VIII. Appropriate Signatures...12 Appendix A Criminal Justice curriculum...13 Appendix B Strengths and Weaknesses of the Program.....14 Appendix C Constituents and Stakeholders Relevant to the Program. 15 Appendix D Field Experience Evaluation Tool...16 2
(I) PROGRAM: Criminal Justice (II) AUTHOR AND DATE SUBMITTED: Hallie Stephens; September 2011 (III) MISSION STATEMENT, GOALS, AND LEARNING OBJECTIVES: The primary mission of the Criminal Justice program is to provide students with a quality education in criminal justice. We strive to prepare them to address the challenges of crime in society as criminal justice professionals, researchers, and educators. It is also imperative that we impart sensitivity to diversity and diverse viewpoints in order that they may understand our diverse society and the impact they will have on criminal justice issues. Criminal Justice is an academic field closely related to the discipline of Criminology. Criminology is the scientific study of crime, criminal behavior, and institutional responses thereto. Criminal Justice is an institutional response. The academic field of Criminal Justice is devoted to the scientific study of crime and the scientific study of the administration of Criminal Justice components: law enforcement, courts, and corrections. The curriculum has a multi-disciplinary foundation that seeks to scientifically orient and academically prepare students for careers in Criminal Justice or graduate or professional studies by relying on instructional assistance from the disciplines of Sociology, Psychology, and Political Science, as well as the field of Occupational Safety. Criminal Justice Learning Objectives Students who complete their degree requirements in Criminal Justice should be able to: 1. Establish and demonstrate a comprehensive knowledge in the five substantive areas of criminal justice: criminal justice process, criminology, law enforcement, law adjudication, and corrections. 2. Demonstrate the ability to think critically. 3. Exhibit the ability to conceptualize ideas. 4. Exhibit basic research skills by successful completion of an original research proposal. 5. Display the ability to communicate effectively in oral and written form. 6. Demonstrate the ability to process and apply information reliably. 7. Demonstrate an understanding of social problems, human behavior, and public policy. 8. Successfully complete a supervised work experience with a criminal justice, law enforcement, or correctional organization or facility. (IV) IDENTIFICATION AND ASSESSMENT OF LEARNING OUTCOMES: Assessment of Learning Outcomes # 1-3,6,7: (1) Establish and demonstrate a comprehensive knowledge in the five substantive areas of criminal justice: criminal justice process, criminology, law enforcement, law adjudication, and corrections. (2) Demonstrate the ability to think critically. (3) Exhibit the ability to conceptualize ideas. (6) Demonstrate the ability to process and apply information reliably. (7) Demonstrate an understanding of social problems, human behavior, and public policy. 3
Learning Outcome How Assessed Contributing Courses Future Assessment 1-3, 6, & 7 Educational Criminal Additional testing Testing Procedures pre-testing and posttesting in contributing Service (ETS) Criminology Subscores Introduction to Law courses. The Criminal (The ETS Enforcement Justice faculty are Examination in Criminal Justice Criminal law designing pre- and indexes student Introduction to post assessments to knowledge in (1) Corrections assess student learning theory, (2) the outcomes 1-3, 6,7 law, (3) law These assessments will enforcement, (4) be embedded within corrections, (5) the contributing the court system, (6) critical courses. thinking, and (7) research methodology and statistics. ETS reports a total scaled score (range =120-200) and scaled subscores) Assessment Method(s): Students assessed: Fall 2010 n=2; Spring 2011, n=5 Selection Notification given by advisor during graduation check and letters mailed to students Sample make-up All graduating Criminal Justice seniors Work evaluated Exam responses from the ETS How assessment conducted The ETS Examination in Criminal Justice was administered to graduating seniors during the Fall 2010 and Spring 2011 semesters in a classroom setting designated for assessment testing only. Data The ETS Criminal Justice total score and subscores in the assessment included (1) theory, (2) the law, (3) law enforcement, (4) corrections, (5) the court system, (6) critical thinking, and (7) research methodology and statistics. Summary of Results Presented in Tables 1-6: Table 1 presents a frequency distribution of the ETS total scaled scores achieved by Southeastern Oklahoma State University (SOSU) Criminal Justice students taking the Fall 2010 ETS examination (n=2). Table 2 presents a frequency distribution of the total scaled scores achieved by SOSU Criminal Justice students taking the Spring 2011 ETS examination (n=5). Table 3 presents a frequency distribution of the ETS total standardized scores of nationwide Criminal Justice students tested 2005 2010, n= 10,951. (The data, statistics and lists of schools administering the Major Field Tests are updated every 5 years). Table 4 presents the mean percent correct scores for all SOSU Criminal Justice students for each ETS assessment indicator, Fall 2010 (n=2). 4
Table 5 presents the mean percent correct scores for all SOSU Criminal Justice students for each ETS assessment indicator, Spring 2011 (n=5). Table 6 presents the mean percent correct scores for all nationwide Criminal Justice students at one hundred and seventeen (117) domestic institutions for each assessment indicator, 2005-2010. Table 1 Frequency Distribution of ETS Total Scaled Scores for SOSU Criminal Justice Students Fall 2010 (n=2) Score Frequency 138 1 149 1 Mean = 144 Std. Dev. = 7.8 Table 2 Frequency Distribution of ETS Total Scaled Scores for SOSU Criminal Justice Students, Spring 2011 (n=5) Score Frequency 144 1 147 1 149 1 151 1 160 1 Mean = 150 Std. Dev. = 6.1 5
Table 3 Frequency Distribution of the ETS Total Standardized Scores for Nationwide Criminal Justice Students Tested 2005 2011, n= 10,951 Total Score Range (120-200) % at or below 177-200 95 173-176 90 170-172 85 168-169 80 166-167 75 164-165 70 161-163 65 159-160 60 157-158 55 155-156 50 153-154 45 151-152 40 149-150 35 147-148 30 144-146 25 141-143 20 138-140 15 134-137 10 129-133 5 120-128 1 Mean = 154 Std. Dev. = 14.8 Table 4 Mean Percent Correct Scores for SOSU Criminal Justice students for each ETS Assessment Indicator, Fall 2010 (n=2) Indicator Mean Percent Correct Theory 24 The Law 52 Law Enforcement 53 Corrections 45 The Court System 43 Critical Thinking 48 Research Methodology and Statistics 41 6
Table 5 Mean Percent Correct Scores for SOSU Criminal Justice students for each ETS Assessment Indicator, Spring 2011 (n=5) Indicator Mean Percent Correct Theory 34 The Law 50 Law Enforcement 58 Corrections 47 The Court System 59 Critical Thinking 55 Research Methodology and Statistics 38 Table 6 Mean Percent Correct Scores for Nationwide Senior Criminal Justice students from domestic institutions (n=117) for each ETS Assessment Indicators, 2005-2010 Indicator Mean Percent Correct Theory 44 The Law 53 Law Enforcement 51 Corrections 49 The Court System 56 Critical Thinking 52 Research Methodology and Statistics 44 Analysis and Interpretation: ETS Total Scores As evident from tables 1, 2, & 3 Criminal Justice student scores at Southeastern Oklahoma State University are comparable to ETS Total Scaled Scores nationwide. Regarding Fall 2010 ETS Total Scores, two students were below the national mean, Regarding Spring 20011 ETS Total Scales Scores, one student was at or above the national mean and four were below. The mean for S.O.S.U. students increased by six points in the Spring 2011 semester when compared to the Fall 2010 semester. It is reasoned that the difference may have been due to an increase in the number of students graduating in Spring 2011 as compared to Fall 2010. ETS Subscores According to tables 4, 5, & 6, there is an overall understanding of Criminal Justice material presented at Southeastern Oklahoma State University for each subscore category assessed by ETS. Regarding Fall 2010 subscore results, S.O.S.U. Criminal Justice students fell slightly below the mean achieved by Criminal Justice students at 117 domestic institutions nationwide on all but one indicator, law enforcement. Regarding Spring 2011 S.O.S.U. Criminal Justice students achieved a higher mean on two out of seven indicators as compared to the mean achieved by Criminal Justice students at 117 7
domestic institutions nationwide. However, throughout the analysis, the low number of respondents (n=7 total for the year) at S.O.S.U. does not allow for a true comparison. More data must be collected. The Criminal Justice faculty is developing new assessment instruments to help address this issue (see rubrics below). Benchmark Standards for ETS Total Results Southeastern Oklahoma State University faculty will utilize the following Total Score and Subscore benchmarks to evaluate student performance on the ETS Assessment for Criminal Justice: ETS Total Score Outcomes Unacceptable Acceptable Target 156 or below 157-175 160 ETS Subscore Outcomes Unacceptable Acceptable Target 40 or below 50-60 55 Planned Assessments in Development by Criminal Justice Faculty Outcomes Increased understanding of Criminal Justice material presented at S.O.S.U. Assessments Embedded in Criminal Justice Courses Pre-tests Post-tests Courses Used Criminal Procedures Criminology Introduction to Law Enforcement Criminal law Introduction to Corrections Scoring Rubric Model for Planned Course Embedded Assessment Assessments embedded in Criminal Justice courses will develop performance rubrics that will follow the general model of the rubric presented below: Unacceptable Acceptable Target Increase in understanding of Criminal Justice material Little indication of increased understanding of Criminal Justice material Program Modification(s) Planned: Create and implement pre-tests and post-tests as indicated. Evidence of mastery of Criminal Justice material 8
Assessment of Learning Outcomes # 4,5: Learning Outcomes: (4) Exhibit basic research skills by successful completion of an original research proposal. (5) Display the ability to communicate effectively in oral and written form. Learning Outcome How Assessed Contributing Courses Future Assessment 4 & 5 Research proposal submitted as course requirement Introduction to Social Research Criminal Justice Capstone course Assessment Method(s): Students Fall 2010 n =11; Spring 2011 n=18 Selection Advisor recommendation Sample make up Criminal Justice students who have completed the prerequisite (SOC. 1113) Work evaluated grade in course How assessment conducted Course (SOC. 3003 Introduction to Social Research) instructor graded and assessed each student based on their attendance, comprehension of material, and quality of proposal completed. Data The grades were obtained for each Criminal Justice student enrolled in the course Results: Table 7 presents a frequency distribution of the research grades earned by SOSU Criminal Justice students enrolled in the required research class for the Criminal Justice program, Fall 2010 (n=11); Spring 2011 (n=18). Table 7 Frequency Distribution of SOSU Criminal Justice Student Grade Scores Fall 2010 (n=11) Spring 2011 (n=18) Fall 2010 Score Frequency Spring 2011 Frequency A (90-100) 0 A (90-100) 3 B (80-89) 6 B (80-89) 10 C (70-79) 3 C (70-79) 4 D (60-69) 2 D (60-69) 0 F (59 & below) 0 F (59 & below) 1 9
Analysis and Interpretation: According to the data, SOSU Criminal Justice students demonstrate average research ability. There will be additional data presented in future reports once the new curriculum has been in place for a few more semesters. The Criminal Justice faculty believe that sufficient additional comparison data from the new capstone course will be available by Fall 2012 or Spring 2013. Introduction to Social Research student grades, Fall 2010 (11), Spring 2011 (n=18) Unacceptable (below 69) Acceptable (70-89) Target (90 or above) 3 23 3 Program Modification(s) Planned: Create and implement other research oriented assessment measures (inclusion of Criminal Justice Capstone course data). Assessment of Learning Outcomes #8: Learning Outcome: (8) Successfully complete a supervised work experience with a criminal justice, law enforcement, or correctional organization or facility. Learning Outcome How Assessed Contributing Courses 8 Criminal Justice Field Experience Site Evaluation Tool (Appendix D) Field experience in Criminal Justice Assessment Method(s): Students Fall 2010 n=5; Spring 201 n=4 Selection The completion of at least 95 hours of coursework Sample make-up Criminal Justice Seniors or Juniors Work evaluated Work site supervisor evaluations, course paper, course attendance, and a minimum of 120 clock hours participation at work site How assessment conducted Classroom evaluation/monitoring by department instructor; instructor graded and assessed each Criminal Justice student Data The grades were obtained for each Criminal Justice student enrolled in the course and the employer evaluation summaries Fall 2010: We placed interns in the following agencies: Texas Department of Public Safety Grayson County CSCD (probation) Oklahoma Office of Juvenile Affairs Grayson County District Clerk s Office Burrage Law Firm On a scale of 1-7 (one=poor performance, 4=average performance, and 7=excellent performance), interns averaged 6.4 indicating that all of our interns performed at an 10
extremely high level. None of the interns earned a below average score, and many earned the highest possible ranking (7) for their performance. Spring 2011: We placed interns in the following agencies: Oklahoma Department of Corrections Durant Police Department Southeastern Oklahoma State University Campus Police Bryan County District Attorney s Office (Victim/Witness Center) On a scale of 1-7 (one=poor performance, 4=average performance, and 7=excellent performance), interns averaged 6.5 indicating that all of our interns performed at an extremely high level. None of the interns earned a below average score, and many earned the highest possible ranking (7) for their performance. Analysis and Interpretation: The combined students for both Fall 2010 (n=5) and Spring 2011 (n=4) enrolled in the field experience course completed all course requirements and received above average evaluations from their site supervisors (100 percent of Criminal Justice students at target). SOSU Criminal Justice students are performing well based upon data from the Field Experience Site Evaluation Tool (See Appendix D). Criminal Justice Field Experience Student Grades, Fall 2010 (n=5) and Spring 2011 (n=4) Unacceptable (evaluation Acceptable (evaluation Target (evaluation and and participation ranks and participation ranks participation ranks student at or below a student with a grade of B) student with a grade of A) grade of C) 0 0 9 Program Modification(s): Beginning in the Fall 2007, the Criminal Justice Field Experience course (CJ 4233) became a more guided experience. Students are required to meet in a classroom setting to discuss their work experiences, ethical guidelines, and produce written assignments. The field experience work site evaluations have been restructured to address student measurable student performance (see appendix E for evaluation tool). (V) MODIFICATIONS: The Criminal Justice program has implemented a new curriculum. Please see appendix A and appendix B for a description of the changes made. The new curriculum was put in place Fall 2009 and as a result there are no data available currently. (VI) DEPARTMENTAL IETV: IETV and Web-based courses offered by the department use the institutional evaluation forms designed for this purpose. The following is a brief description of the Criminal Justice courses offered Fall 2010 and Spring 2011. N= the total number of students enrolled in all courses combined. 11
Fall 2010 course # semester offered offered online yes or no offered blended yes or no offered as face-to-face yes or no CJ 2533 FA10 - IETV no no yes CJ 3113 FA10 IETV no no yes CJ 3133 FA10 IETV no yes yes Spring 2011 course # semester offered offered online yes or no offered blended yes or no offered as face-to-face yes or no CJ 2533 SP11 IETV no no yes CJ 3113 SP11 IETV no no yes CJ 3323 SP11 IETV no no yes CJ3343 SP11 IETV no no yes CJ 4403 SP11 IETV no no yes Fall 2010 satisfied* unsatisfied* IETV (n=39) 32 7 Spring 2011 satisfied* unsatisfied* IETV (n=72) 59 13 *The questions on both survey instruments were either dichotomous or a likert scale. The questions indicating a positive, or satisfied, response to the course were combined into one and the scales were collapsed. The same procedure was used for the questions indicating a negative, or unsatisfied, response. (VII) FACULTY CONTRIBUTIONS: A copy of this report was made available to the Behavioral Sciences faculty prior to the submission deadline via e-mail attachment. All faculty members in the department were encouraged to review the report and make suggestions and corrections. The Criminal Justice faculty members in the Department of Behavioral Sciences are: Dr. Hallie Stephens, program coordinator and assistant professor, Dr. Dennis Brewster, associate professor, and Dr. Delilah Dotremon, associate professor. Hallie Stephens, Criminal Justice Program Coordinator Date Dean, School of Education and Behavioral Sciences Date Department Chair, Department of Behavioral Sciences Date 12
Appendix A SOUTHEASTERN OKLAHOMA STATE UNIVERSITY CRIMINAL JUSTICE MAJOR/MINOR (57 HOURS) Required Courses (27 hrs.) Enrolled Completed CJ 2533 Introduction to Criminal Justice 3 hrs. CJ 3113 Introduction to Policing in the U.S. 3 hrs. (Prq. CJ 2533, SOC 4333) CJ 3123 Introduction to Law & the Legal System 3 hrs. (Prq. CJ 2533, SOC 4333) CJ 3133 Introduction to U.S. Corrections 3 hrs. (Prq. CJ 2533, SOC 4333) CJ 4333 Capstone in Criminal Justice 3 hrs. (Prq. CJ 2533, SOC 4333, CJ 3113, CJ 3123, CJ 3133, SOC 3523, SOC 3003, SOC 3003 Introduction to Social Research 3 hrs. (Prq. SOC 1113, CJ 2533, SOC 4333) SOC 3123 Methods of Social Research 3 hrs. (Prq. SOC 1113, CJ 2533, SOC 4333) SOC 3003, & 9 elective hours) SOC 3523 Juvenile Delinquency 3 hrs. (Prq. SOC 1113, CJ 2533, SOC 4333) SOC 4333 Criminology (Prq. SOC 1113, CJ 2533) 3 hrs. Elective Courses (15 hrs. from:) CJ 3323 Criminal Investigations & Forensics 3 hrs. CJ 3333 Administration of Criminal Justice 3 hrs. CJ 3433 Criminal Law and Procedures 3 hrs. CJ 4253 Contemporary Issues in Policing 3 hrs. CJ 4283 Correctional Law 3 hrs. CJ 4403 Comparative Criminal Justice Systems 3 hrs. CJ 4970 (three hours) Special Studies 3hrs. POSC 3513 U.S. Constitutional Law 3 hrs. (9 hrs. from:) CJ 3343 Victimology 3 hrs. CJ 4543 Advanced Criminological Theory 3 hrs. SOC 2213 Social Problems 3 hrs. SOC 3623 Race, Gender, and Ethnic Relations 3 hrs. SFTY 3133 Drugs and Society 3 hrs. (3 hrs. from:) CJ 4260 Independent Study in Criminal Justice 3 hrs. CJ 4440 Internship in Criminal Justice 3 hrs. (3 hrs. from:) PSY 3373 Psychology of Personality 3 hrs. PSY 4313 Abnormal Psychology 3 hrs. PSY 4973 Forensic Psychology 3 hrs. 13
Appendix B Strengths and Weaknesses of the Criminal Justice Program Strengths: Program structure in now aligned with accreditation standards o Multi-disciplinary approach o Curriculum design improved to accomodate different specialties: Law enforcement Courts Corrections Student enrollment is traditionally high Improved contact with off-site branches o Continued effort to establish cohort in Grayson county o New hire designated for McCurtain County campus Student involvement in student oriented organization specific to the discipline o S.O.S.U. Criminal Justice Club Weaknesses: Lack of full-time faculty lines to support continued growth potential Retention of students graduation rate is low compared to enrollment and declared majors Slow progress on distribution given to constituents and stakeholders to identify needs 14
Appendix C Constituents and Stakeholders Relevant to the Criminal Justice Program Law enforcement agencies Correctional facilities Attorneys Graduate programs 15
Appendix D Criminal Justice Field Experience Site Evaluation Tool INTERNSHIP IN Criminal Justice DEPARTMENT OF BEHAVIORAL SCIENCES CRIMINAL JUSTICE PROGRAM SOUTHEASTERN OKLAHOMA STATE UNIVERSITY Student s Name Mid-Term Evaluation : Final Evaluation (check one). EVALUATION Please evaluate the student in each of the following areas. Circle one number for each line. Personal Characteristics Poor 1. Enthusiasm 1 2 3 4 5 6 7 0 2. Friendliness 1 2 3 4 5 6 7 0 3. Cooperativeness 1 2 3 4 5 6 7 0 4. Promptness 1 2 3 4 5 6 7 0 5. Competence 1 2 3 4 5 6 7 0 6. Empathy with People 1 2 3 4 5 6 7 0 7. Shows Good Judgment 1 2 3 4 5 6 7 0 8. Confidentiality 1 2 3 4 5 6 7 0 9. Initiative 1 2 3 4 5 6 7 0 10. Responsibility 1 2 3 4 5 6 7 0 Relationships 11. Relation to staff 1 2 3 4 5 6 7 0 12. Relation to clients 1 2 3 4 5 6 7 0 Ability to Use Experience in Future 13. Adaptability 1 2 3 4 5 6 7 0 14. Level of student s performance 1 2 3 4 5 6 7 0 15. Ability to organize and carry out tasks 1 2 3 4 5 6 7 0 16. Potential for a career in this type of work 1 2 3 4 5 6 7 0 Additional Comments Concerning Student s Performance in Internship Average Excellent Can t Answer Please list any special activities, programs, or decisions in which the student was involved during his Internship with your agency or organization. Attach additional pages if necessary 16