Jim Fillpot, Giovanni Sosa, & Lori Waite Chaffey College
Self-Inflicted Historical Challenges : Lack of Student Services leadership Student Services not addressed in district processes Square pegs into round holes What constitutes evidence? Where is it? No ownership All talk, no action
Learning Outcomes Statement Use of Results Means of Assessment Summary of Evidence Criteria
Whenever possible, reflect and articulate key Student Services learning outcome themes Take the wide and varied range of students knowledge, skills, and abilities (KSAs) into consideration Be clear and simple Meaningful
Student Health Services: Students will demonstrate increased accountability via service-seeking behaviors Students will demonstrate increased independence via awareness of services offered on-campus and by community agencies
Disability Programs and Services: Students will be able to independently select and utilize appropriate assistive technologies DPS students will demonstrate the ability to self-advocate. They will demonstrate an awareness of available services and initiate discussion with both DPS and non-dps staff regarding their functional limitations and/or needs. DPS students will demonstrate independent planning and self-management. They will participate in educational planning and actively pursue their educational, professional, and personal goals.
International Students: Students will have a strong understanding of the various policies and procedures they must adhere to while attending Chaffey College International students will report a greater sense of belonging and integration with the campus community
Opening Doors to Excellence: Students will demonstrate increases in selfesteem, educational participation, and goal orientation
One size does not fit all Identified assessment instrument or evidencegathering approach reflects identified KSAs Integration into existing program processes Simple and brief Meaningful
Student Health Services Rubric (student only) Disability Programs and Services Rubrics (both student and staff) International Students Multiple choice(pre-post) Affective (per semester assessment) Opening Doors to Excellence Affective measures (pre-post)
Gathered evidence can be understood by practitioners Simplicity is key Present raw evidence Present Effect Size
Health Services (N = 155-164) Rubric (4 points) Pre (Mean) Post (Mean) Effect Size (d) U3 Accountability 2.84 3.17.37 64% Independence 1.55 2.50.81 79%
Disability Programs and Services (N = 130-224) Rubric Rubric Score of 3 or 4 Rubric Score of 1 or 2 Mean Response Student Self Ratings Effective Use of Technology 82.5% 17.5% 3.17 Ability to Self-Advocate 84.6% 15.4% 3.36 Independent Planning and Self-Management 82.0% 18.0% 2.97 DPS Staff Ratings Effective Use of Technology 57.7% 42.3% 2.61 Ability to Self-Advocate 68.5% 31.5% 2.96 Independent Planning and Self-Management 68.3% 31.7% 2.85
International Students (N = 30) Assessment Item Number Correct Pre-Test Number Correct Post-Test Effect Size (d) Item 1 Full-Time Status 30 30.00 Item 2 Online Classes 10 18.54 Item 3 Drop Approval 22 25.24 Item 4 Counselor Location 22 22.00 Item 5 Work Hours 12 27 1.13 Item 6 Office Notification 19 11 -.54 Item 7 Registration 25 24 -.09 Item 8 First Semester Tasks 19 24.37 Average Number Correct 4.47 6.03 1.13
Opening Doors to Excellence (N = 60-63) Measure Pre (Mean) Post (Mean) Effect Size (d) U3 Self-Esteem 23.30 27.98 1.52 93% Educational Participation Perceived Value of Education 19.26 23.67.84 80% 20.55 21.35.30 62% Goal Orientation 20.11 20.18.02 51%
Survey Item 1. I believe I will receive an excellent grade in this class 2. I m certain I can understand the most difficult material presented in the readings for this course 3. I m confident I can learn the basic concepts taught in this course 4. I m confident I can understand the most complex material presented by the instructor in this course 5. I m confident I can do an excellent job on the assignments and tests in this course 6. I expect to do well in this class 7. I m certain I can master the skills being taught in this class 8. Considering the difficulty of this course, the teacher, and my skills, I think I will do well in this class
d = 1.21 Average Grade (GPA) 4 3 2 1 2.25 d =.63 2.98 d =.51 3.49 0 Low SE Medium SE High SE N = 68 N = 136 N = 90
100% 89.0% 96.7% Success Rate 80% 60% 40% 20% 73.5% 0% Low SE Medium SE High SE N = 50/68 N = 121/136 N = 87/90
Needs to be meaningful: If the outcome was not meaningful, then the evidence will not be used meaningfully Opportunities: Create venues for the work to occur (e.g. departmental meetings, retreats, FLEX, etc.). Invite Institutional Research to venues!
Health Services Themes of self-advocacy and independent selfplanning incorporated throughout program Planning for additional SLO examining the awareness of resources Disability Programs and Services Promoted more frequent use of technology to reduce dependence on staff (e.g., test-taking environment) Seeking to establish new curriculum to increase technology skills needed to complete assignments
International Students Modification of Orientation Content/Activities Opening Doors to Excellence Instructors focusing specifically on targeted measures Modifications to Assessment Instrument
Culture: Use of evidence for decision-making Use of evidence for scalability Collaboration & Dialog: Administration/Institutional Research/Programs & Services Leadership: A driver for the process, pace the work, a resource, follow-up, partnership Linkages: Link SLO process to Program Review process/timelines, create annual assessment plan, reduce workload redundancy
Use of Results Summary of Evidence Learning Outcomes Statement Criteria Means of Assessment SLO Assessment is a continual process!