What is Assessment and Why is it Necessary?

Transcription

1 Dr. Vann Newkirk

2 What is Assessment and Why is it Necessary? Assessment is defined as the systematic and ongoing method of gathering, analyzing and use of performance data, from various sources, to improve student learning and or the unit operational processes. Assessment: Offers information to students about the knowledge, skills, and other attributes they can expect to possess after successfully completing coursework and academic programs.; Establishes ways for academic units to understand the dimensions of student learning when seeking to improve student achievement and the educational process; Assessment is needed for accountability. Accountability, with its external focus, provides evidence of student achievement to accreditation groups (SACS, NCATE, ABET), state legislators, and other stakeholders in education.

3 College X ASSESSMENT MODEL College X Mission (Strategic Planning) Use of Results Adjustments in Programs and Operations Formation of Programmatic Goals and Expected Educational Learning Outcomes Evaluation (Assessment)

4 Two Types of Program Assessment Academic Degree & Core Program Assessment Administrative Assessment Academic Degree & Core Programs Academic Support Units Administrative Units Community Service Functions Research

5 Academic Degree & Core Program Assessment Undergraduate Degree Program Level Graduate Degree Program Level General Education (Core program)

6 How Do THEY KNOW!

7 I know they know because I have a PhD

8 I know because I have taught this way since the beginning of time.

9 I know because I Issue Grades Papers Oral Presentations Tests Class Participation

10 I know... Well I think I know

11 PART II ESTABLISHING METHODS OF EVALUATION

12 Must Identify Expected Levels of Competency or Planned Levels of Student performance? What is an Expected Level of Student Performance? 1. It is a pre-determined level of achievement that a student will reach at the end of a program of study. 2. It is a demonstrable skill a student will exhibit at the end of a program of study. 3. It is a particular action a student will undertake when they complete a program of study. Genesis for developing Student Learning Outcomes

13 How do we Identify Student Learning Outcomes? All Assessment efforts must support the institutional mission. No institution or department has the resources or time to continually assess all possible aspects of each academic program. Given this limitation, priorities for the assessment effort must be set to avoid measuring the meaningless as an easy way out, or choking to death on an assessment effort of gargantuan size. Hence, it is logical to begin or focus the department s assessment efforts on (1) those expectations for graduates which have been identified as of primary importance or (2) those processes that support the institutional purpose.

14 Nature of Statements of Student Learning Outcomes Should Preferably be based on Demonstrable Skills! Examples: Students Completing the BA program in History will demonstrate basic understanding of U.S. (African-American) history from Students completing the BA program in History will Exhibit basic understanding of the U.S. Civil Rights movement and its impact on Central Alabama. Students completing the History Program will demonstrate the ability to identify the causes and impact of the American Civil War.

15 How Many Student Learning Outcomes Does My Program Need? Three Four Ten Twenty

16 Answer Three to Five

17 How Do You Measure The Effectiveness of Each Student Learning Outcome Must Use Multiple Means of Assessment Qualitative Quantitative

18 Sample Means of Assessment History Program History sample assessment A&M.doc General Education Program General Education Programs Exhibit basic understanding of U.S. (African-American) history from At least 80% of students completing the ACAT History Exam, will earn a final score at or above a 500 on the African-American sub-section of the exam (This exam covers African-American history from Major topical areas covered in African- American history include: Colonial America, Revolutionary War period, Slavery, Expansion of Cotton, Civil War, Reconstruction, Jim Crow era, Pan-Africanism, Legal Assault against Jim Crow, The Post War era, the Civil Rights movement, America since the Civil Rights movement). At least 60% of the students completing the History program will agree/strongly agree, on the Senior Academic Program Assessment, that they have a basic understanding of Reconstruction and Jim Crow era African-American History.

19 Identify Effective Evaluative Tools What are evaluative tools? They are qualitative and quantitative measures that examine student or unit performance? Type of Assessment Poor Tools Better Tool Best Tool Academic Degree Program Course Grades Internally developed (Program specific) exit exam must have longitudinal analysis section Externally normed exit exam with comparative analysis Academic Degree Program Imbedded assessment measures Imbedded assessment measures with longitudinal analysis of each measure Student graduation rates/graduate school attendance rates/job placement rates General Education Program Course Grades/Imbedded assessment measures Imbedded assessment measures with longitudinal analysis of each measure Externally normed core exam with comparative analysis Administrative (Academic Support) Units locally developed assessment measures with no longitudinal analysis. Locally developed assessment measures such as program specific surveys that have a longitudinal analysis. Externally normed student/faculty or user survey with comparative analysis

20 Other Tools Tool Use Rating Provides Comparative data Provides Longitudinal data Good Best ETS Major Field Exams Measures student content knowledge at the end of a course of study (provides comparative analysis) X X X ACAT Major Field Exams Measures student content knowledge at the end of a course of study (provides comparative analysis) X X X Collegiate Assessment of Academic Proficiency Measures Student content knowledge in core content areas of reading, writing, critical thinking, mathematics, and science reasoning (provides comparative analysis). X X X Select Embedded Assessment Measures Measures augment external assessment processes. Mainly used in specialized programs in which there are no external assessment tools. X General Education Skills Survey Measures student attitudes in regards to core competencies. X X Survey of Academic Advising Measures user satisfaction with services offered by the academic advising(provides comparative analysis. X Collegiate Learning Assessment CLA Assessment Services provide a means for measuring an institution's contribution to the development of key higher order competencies, including the effects of changes to curriculum and pedagogy. To gauge summative performance authentically, the CLA presents realistic problems that require students to analyze complex materials and determine the relevance to the task and credibility. Students' written responses to the tasks are evaluated to assess their abilities to think critically, reason analytically, solve problems and communicate clearly and cogently. X X X Student Academic Program Assessment Measures student attitudes in regards to discipline preparation. X X X Association of Social Work Boards (ASWB) Social Work Licensure Exam Measures student content knowledge in one of four categories of social work licensure examinations: Bachelors, Masters, Advanced Generalist, and Clinical.. X X X Locally Developed Competency Exams Juried evaluations of a specific skill using a rubric Measures user satisfaction with services offered by the academic advising(provides comparative analysis) In a Juried Assessment, a student assembles a Collection of Evidence that demonstrates proficiency in the content area. A panel of experts determines if the evidence demonstrates the student s ability to meet the content area standards X X X X X X X X

21 Future Enhancement model Evaluation Through a Formative/Summative Process Assessment Tools (Student Learning Outcomes) A. CAAP B. Sophomore Junior Diagnostic Exam C. ACAT Major Field Exams D. ETS Major Field Exams E. SAPA Survey F. General Education Skills Survey G. Oral Communication Rubric H. PRAXIS Licensing Exam I. Social Work Licensure Exam Assessment of learning Degree Program Assessment for learning Assessment Tools (Academic Programmatic Goals) A. Student Placement Rates B. Student Graduation Rates C. Course completion rates Learning Summative assessment is cumulative in nature and is utilized to determine whether students have met the course goals or student learning outcomes at the end of a course or program. Summative Formative Formative assessment is utilized to immediately determine whether students have learned what the instructor intended. This type of assessment is intended to help instructors identify material which needs to be clarified or re-taught and should not be used to evaluate or grade students. Results of formative assessment can assist instructors to ascertain whether curriculum or learning activities need to be modified during a class session or before the next class meets Assessment After Learning Assessment during learning

22 II. Administrative/Academic Support Educational Support Units Administrative Units Community Service Functions Research Activities

23 Support Unit/Academic Programmatic Assessment Support Unit IMPROVE PLAN Use Results Set Goals ASSESS ACT Evaluate

24 How Do We Know A Unit Is Operating at Optimal Efficiency Accomplishing Its Assigned Functions?

25 I know because???

26 How do we Identify Intended Administrative Objectives? All Assessment efforts must support the institutional mission. No institution or department has the resources or time to continually assess all possible aspects of each academic program. Given this limitation, priorities for the assessment effort must be set to avoid measuring the meaningless as an easy way out, or choking to death on an assessment effort of gargantuan size. Hence, it is logical to begin or focus the department s assessment efforts on those expectations that have been identified as of primary importance.

27 What is an Administrative Objective? 1. It is a statement regarding what a department/unit intends to accomplish. 2. It is a benchmark that a department sets against which performance is judged by staff or administrators within the department or the institution.

28 Nature of Administrative objectives Should preferably be written to measure effectiveness of a particular function of the unit that is essential to enhancing performance! Examples: (Admissions Office) Improve the conversion rate of inquiries-to-applications and applications-to-deposits. (College bookstore) Increase the sale of non-book merchandise. (Residence Life) Decrease residence hall damage. (Registrar s Office) Improve service levels give to all customers. Other Important tools

29 Office of Undergraduate Admission bookstore Assessment Model sample - academic support complet.doc Office of Admissions Increase the conversion rate of inquiry-to-applicant and applicant-to-deposit for new students. Develop and implement a web-based tracking system to track all inquiries by May Send at least one message weekly to all inquiries between January July Offer priority housing to all inquiries who apply between March and May of Offer conditional scholarships to all applicants with high school GPAs above 3.00 within one week of receiving application. Send one and one automated phone message to each applicant twice each week between January and July 2014.

30 Use of Results Institutional Purpose Use of Results Intended Outcomes Assessment Activities

31 QUESTIONS