Assessing Student Learning Outcomes: Institutional Examples

Transcription

1 Institutional Examples Karen Adams, Dean, ollege of Education and Human Services, entral Michigan University Rhonda D. Blackburn, Assistant Provost, Educational Enhancement, University of Texas at Dallas Don arter, Director, Learning, Northern Arizona University Kathy Koch, Associate Dean, ollege of Education and Human Services, entral Michigan University harlene Douglas, Marketing Manager, Higher Education, Desire2Learn Incorporated Assessment identifies whether students learn what is intended and provides the insight required to improve student learning by changing the way we teach 1. At the course level, assessment drives change in teaching approaches within individual classrooms 1. At the institutional level, assessment drives changes in curriculum to equip graduates with the tools and knowledge to succeed in their professions 1. Furthermore, assessment provides a basis for accountability. For the student, learning outcome assessment assists with self-motivation and guides future actions. Student learning outcomes describe measurable knowledge, attitudes, skills and behaviors that students should demonstrate upon completion of an educational program from an institution 2. Within institutions, assessment serves to 3 : Provide improved feedback, guidance and mentoring to students to help them better plan and execute their educational program Provide improved feedback about student learning to support faculty in their work Help design and modify programs to better promote learning and student success Assist in developing common definitions and benchmarks for important student abilities that will enable institutions to act more coherently and effectively to promote student learning Enable understanding of how various groups of students experience higher education differently to adapt courses and programs to the needs and capacities of all students Provide insight into how different courses and programs affect students over time to better coordinate and sequence the student s experience and produce greater and deeper learning Three prominent higher education institutions (entral Michigan University, Northern Arizona University, and the University of Texas at Dallas) have instituted unique assessment programs on their campuses. A short narrative explaining their respective program, lessons learned, and outcomes is provided for each institution to illustrate how assessment can be accomplished with success. -1-

2 Institutional Examples (ontinued) entral Michigan University, ollege of Education and Human Services Karen Adams, Dean, ollege of Education and Human Services Kathy Koch, Associate Dean, ollege of Education and Human Services At entral Michigan University we view the assessment process as a means for programs to identify areas for improvement. Our goal is to encourage programs to determine what knowledge and skills graduates should have at the completion of the program and then to develop measureable program outcomes that address the stated goals of the program. Program outcomes will reflect a broad macro-analysis of individual course objectives without the specificity required of each course. Through an analysis of the program courses, we hope to answer the question, do inputs give us the outputs we desire from the program?. For assessment to be successful, it is important that we know what outputs we actually want. The implementation of program assessment plans at entral Michigan University has been a goal of the academic division for over a decade. We have a Director of urriculum and Assessment who oversees the process whereby all programs submit five-year assessment plans and annual reports of progress in achieving the goals of the assessment plans. The Director works with an Assessment ouncil composed of faculty from the various colleges. They work to ensure that the university does indeed move forward with universal program assessment plans that are evaluated regularly. Despite the length of time we have been working on assessment plans, we still have not achieved 100% compliance. We have achieved the most success in the development of assessment plans when accreditation is at stake since assessment has been a central focus in all our varied initial program accreditation and reaccreditation visits over the past ten years. Our goal as an institution is to have all nationally accreditable programs accredited, and we are very close to meeting this goal. However, all programs, accredited or not, must complete a full program review on a seven-year cycle for which assessment plans are an expected component. Those programs with existing national accreditation must only complete an abbreviated program review on this seven-year cycle, but an analysis of assessment plans is still a part of this abbreviated review. Results of this program review process include recommendations from the Provost ranging from allocation of additional funds to assist with program growth and improvement to denial of funds and program discontinuation. -2-

3 Institutional Examples (ontinued) This program review has been an important step in ongoing attention to assessment, since it has been our experience that unless departments and programs have a reason to engage in the assessment process that can ultimately affect future allocation of resources, or the continuation of a program, then assessment does not remain a priority for all programs. All new programs that go through the curricular process must have an assessment plan, as well as all programs going through a significant change process. The various department, college, and university level curricular bodies are well aware of this assessment requirement and take their approval process very seriously, sometimes returning programs to their home departments for further detail work concerning assessment plans. Assessment plans are a valuable addition to each department s annual report. While ensuring that departments comply with assessment requests outside of accreditation, program review, or curriculum development can be difficult, expectations of unit administrators are an important part of the success of assessment across the entire institution. entral Michigan University uses a number of strategies and resources to encourage the development and use of assessment plans. The Director of urriculum and Assessment and the Assessment ouncil provide much guidance for creating an assessment plan. Some funding is devoted to strengthening assessment efforts. Our assessment website provides multiple resources for creating assessment plans as well as sharing existing plans. We are currently piloting an online assessment management software in the hopes that this will bring all assessment data into one location for easy retrieval that will improve communication about assessment across the university. -3-

4 Institutional Examples (ontinued) Northern Arizona University Don arter, Director, elearning enter Evidence is sprouting across our university community of the growing culture of assessment and the prominence of examining what we do as a public educational institution. Key indicators are: 1. entralized assessment (1999) lead by the Office of Academic Assessment (see 2. The large number of assessment plans written by programs and departments submitted for review by the University Assessment ommittee (UA), 3. The university wide annual Assessment Fair, held in April (Sixth annual in 2008 see and 4. NAU recently joined the Voluntary System of Accountability (VSA) developed by NASULG & AASU. (see The event that showcased NAU s recent assessment activities was the assessment fair held on April 4, 2008 in the new High ountry onference enter. Projects from all areas of the university, beyond the academic programs, were presented to over 200 attendees, 41 poster presentations displayed data on retention, student learning, student satisfaction, teaching effectiveness, and other assessment activities. For a brief time in the early 90s, assessment was centralized, but then relegated to individual departments in a time of decreasing enrollment and associated budget challenges. oordinated, organized, intentional assessment at NAU began in fall 1999 focused on the new Liberal Studies program. This program, since revised in , was developed partially in response to the 1997 North entral Accreditation report. After moving back to a centralized assessment office, a permanent full-time director was hired in Under the director s leadership, the University Assessment ommittee created the current rubric for evaluating program assessment plans. See (www4.nau.edu/assessment/oaalibrary/documents/rubrifinal0904.doc ) -4-

5 Institutional Examples (ontinued) The Office of Academic Assessment reports that 85% of NAU s 147 degree programs have created assessment plans. Twenty-seven new plans were submitted and approved during AY Seventy six percent of the degree programs have submitted an assessment report since Twelve of these reports indicate program changes based on the assessment results. This closing the loop is an indication that NAU s assessment process is working. Three recent innovations hold promise for increasing the number and quality of assessment activities, and the collection of assessment data and program change informed by these data. First is a small grant program to provide incentives as well as the time and resources to design and write program assessment plans. The second is the collaboration of the offices of Academic Assessment, and Faculty Development with the elearning enter in working with programs and faculty to sustain assessment, curricular change, and program improvement. A document highlighting these activities is the Learning Lines newsletter published by this collaborative. (see Finally, programs are recognized at two possible levels for their assessment efforts; Achievement or Excellence. These awards come with a seal for the program s web site and printed materials. (see -5-

6 Institutional Examples (ontinued) University of Texas at Dallas Rhonda Blackburn, Assistant Provost, Educational Enhancement The University of Texas at Dallas made it through the accreditation process by the hard work and dedication from the faculty and staff. To make assessment on campus more understandable and streamlined, the Office of Educational Enhancement (OEE) is working with two diverse programs on campus to develop an assessment process. As part of the assessment process, each program reflects on their assessment findings and additional criteria to determine how to improve and what areas require focus. The programs start with their student learning outcomes (SLOs). All the assessment data from the instructors helps determine if the students are meeting the SLOs the program stated as being crucial to graduation. At this time the program is evaluated based on these findings and it is determined if the SLOs are appropriate and critical to the overall knowledge needed to enter into the field. Once it is agreed that the SLOs are the correct ones, a curriculum map is developed to align outcomes and courses. This is an important step to understand exactly what the student needs to learn and in what order they should be proceeding through the degree. In addition to SLOs and alignment, all constituencies that are associated with the program are interviewed. This group includes the current and former students, faculty within the program, and the current and potential employers. They are surveyed to see if there are any discrepancies between what the program covers and what is actually required. This survey helps bring up important aspects of the program that cannot be determined by curriculum data. It also helps the programs plan for future growth and employment outlook. The last piece of the assessment process analyses the specific faculty and graduate student professional development that could be implemented to increase the knowledge and skills of the faculty and students. Faculty development includes pedagogy, teaching and learning, and technology workshops and seminars; for example, writing across the curriculum, large class management, student engagement, learning management system, peer review, etc. Graduate students seminars and workshops have also been developed. These include a teaching certificate within our Teaching Academy and a Research Academy. PhD students are also helped with the ABD seminars. -6-

7 Institutional Examples (ontinued) As the programs go through this assessment process program administrators understand their students, curriculum and employers better. As the process becomes more defined, resources, documents, and tutorials will be placed on the OEE web site for program heads and faculty to utilize while they work on improving their program. onclusion The effectiveness of educational experiences can be evaluated to demonstrate student attainment of desired educational goals. Various programs and experiences can also monitor to what extent they have facilitated the learning and development of students. These are the fundamental success measures surrounding student learning outcomes assessment. The three institutions above have shown how they attempt to answer these questions and, therefore, provide a basis on which to continue to improve their educational offerings and their preparation of students for their careers. This entire process is only effective when the assessment results are used to improve subsequent learning. About Desire2Learn Desire2Learn Inc. is a world-leading provider of enterprise elearning solutions that enable institutions and other organizations to create teaching and learning environments that reflect their vision, values, goals, pedagogical approaches and branding. Desire2Learn supports over five million learners worldwide and clients include higher education, K-12, as well as associations, government, and other leading organizations. For more information, visit:. About entral Michigan University, ollege of Education and Human Services The ollege of Education and Human Services at entral Michigan University consists of five departments and over 185 faculty and staff in ounseling and Special Education, Educational Leadership, Human Environmental Studies, Recreation, Parks, and Leisure Services Administration, and Teacher Education and Professional Development. Offering bachelors through doctoral degrees, programs reflect the primary mission of the college to provide the highest quality educational experiences that will develop effective professionals, critical thinkers, and lifelong learners. Through strong academic preparation, students expand their understandings of their disciplines of study and of themselves. -7-

8 Institutional Examples (ontinued) The college s many local, regional, and international field experiences and internships are designed to equip students for an increasingly diverse and global society. The majority of programs throughout the college are not only regionally accredited by the Higher Learning ommission but also maintain national and professional accreditations. The college is recognized for its extensive involvement in student professional organizations at the local, state, and national levels. The Education and Human Services Residential ollege, now in its third year of operation, brings together students selecting majors in any of the college s programs and has been very successful in bringing together students with common interests, allowing them to share introductory courses and participate in Service Learning activities and field trips. Through a broad range of technological and human services, the ollege and its graduates, in collaboration with the constituencies they serve, will be proactive forces for improving the quality of life in the twenty-first century. For more information go to About Northern Arizona University Northern Arizona University has a student population of about 20,000 at its main campus in Flagstaff and at over 35 sites across the state. With its solid reputation as a university with all the features of a large institution but with a personal touch, NAU carefully balances teaching, scholarship and service with a faculty and staff dedicated to each student s success. The emphasis is on undergraduate education, but you ll also find a wide range of graduate programs and research that extend to such national concerns as forest health and bioterrorism. NAU has carefully integrated on-campus education with distance learning, forming seamless avenues for students to earn degrees. FTE Enrollment Total: 18,281 Graduate: 3696 Undergraduate: 14,585 Degrees: 91 Undergraduate Degrees 47 Master's Degrees 8 Doctoral Degrees Average lass Size: 27 Average High School GPA of new freshmen: 3.4 Faculty: 755 ampus: 738 acres, including 24 residential buildings -8-

9 Institutional Examples (ontinued) About the University of Texas at Dallas The University of Texas at Dallas (UTD) is a young, dynamic research institution on the cutting edge of science, technology, medicine, business and the arts. Starting as a research institute and later developing graduate and undergraduate programs UTD provides a unique learning environment. It is host to seven schools, offers an array of interdisciplinary degree programs, and features a student population as diverse as its areas of study. Since its inception in 1961 as the Graduate Research enter of the Southwest, an outgrowth of technology giant Texas Instruments, UTD fosters a strong tradition of academic excellence. UTD became part of the U. T. System in 1969, offered only graduate degrees until 1975, and admitted its first freshman class in Today, it ranks at or near the top in the number of computer science degrees awarded each year in the United States. With a current enrollment of more than 14,000 students and a world-class faculty that includes two Nobel laureates, UTD aims to provide Texas and the nation with the benefits of educational and research programs of the highest quality. By merging theory with practice in classrooms and at the university's 29 research centers, we challenge curious minds to find the answers to their questions. -9-

10 Institutional Examples (ontinued) References 1. Pet-Armacost, Julia UF University-Wide System for Assessing Student Learning Outcomes. 2. Panter, Abigail, Ed Neal, Lynn Williford. Fall, A Simple Guide to Assessing Student Learning Outcomes for ontinuing Program Improvement. The University of North arolina at hapel Hill. 3. Student Learning Outcomes Institute. May Targeting Student Learning - Student Learning Outcomes: A Focus on Results. Modesto Junior ollege. Resources 1. Frye, Richard. Assessment, Accountability, and Student Learning Outcomes. Dialogue. Office of Institutional Assessment and Testing, Western Washington University. Available at 2. James, Richard A New Era in Assessing Student Learning. Assessing Learning in Australian Universities. 3. Moore, Anne H. The New Economy, Technology, and Learning Outcomes Assessment. Educause Quarterly, Vol 30, No. 3, Available at 4. Rudmann, Jerry. March, Does Assessment of Student Learning Outcomes Make a Difference? One Department s Experience. Irvine Valley ollege. 5. Spurlin, Joni E. Technology and Learning: Defining What You Want to Assess. Educause Learning Initiative, Paper 1, July, Edited by Diana Oblinger. -10-