Step-by-Step Process for Writing Learning Outcomes. Introduction

Transcription

1 Step-by-Step Process for Writing Learning Outcomes Office of Dean of Students January 2011 Gail S. Rooney, Director The Career Center Introduction This step-by-step process is intended to guide you through defining and writing learning outcomes that are tailored to your specific programs, services, and resources. The worksheets are intended to spark conversation and gather insights, as well as to help your staff collaborate to better understand your purposes and demonstrate the effectiveness of your student development programs and services. This process assumes a learning framework for programs and services. Student Affairs professionals are viewed primarily as educators, with a goal of assisting students with development that begins during our interventions and extends into life-long learning. We are teaching a process that our students will use over and over again as they navigate choices throughout their lives. We suggest a seven-step learning outcomes cycle for defining context, brainstorming and writing learning outcomes, making connections to theory and professional standards, prioritizing focus areas, conducting evaluations, and reconsidering outcomes and interventions based on the data collected. This workbook addresses the first four steps The Career Center at the University of Illinois at Urbana-Champaign 715 S. Wright St., Champaign, IL ~ (217) ~

2 Step 1. Define Context Effective learning outcomes are tailored to the unique environment, intended learners, and program/service that they are designed to assess. Let s begin with a single program or service offered by your department or office that you would like to evaluate. Begin by clearly defining the program or service that you would like to evaluate. a) How would you describe the program or service? b) How will the program / service be delivered? When will it be delivered? Where will it be delivered? c) What need is the program / service designed to meet? Second, consider the intended learners that you are trying to reach with your program / service. You may find it helpful to consider the following: d) According to the goals or preset intentions of your program / service, do you plan to serve? e) Who most often requests this type of program / service? f) Who may benefit from this type of program / service? EXAMPLE 1 Resume Review Services What is your program or service? Daily resume review services provided on a drop-in basis. Drop-ins are staffed by trained graduate and undergraduate resume reviewers conduct these reviews as a primary focus of their jobs. These services are used by clients are conducting job and internship searches, as well as by students need to prepare a resume as a part of an academic class. Page 2

3 Who are your intended learners? Current students and alumni at the University of Illinois. Page 3

4 EXAMPLE 2 Exploring Pathways in Career Success (EPICS) What is your program or service? Exploring Pathways in Career Success (EPICS) is an interactive, online, modular program that engages students in thinking about their career decision-making early in their college experience. Within EPICS modules, students assess their needs, receive immediate feedback, and plan for next steps. They are empowered to take action on developmentally appropriate tasks, as well as seek assistance if they become stuck. Each module has printable results so that instructors, advisors, and students can use the information in classroom and advising settings. Who are your intended learners? Current undergraduate students at the University of Illinois at Urbana-Champaign Page 4

5 EXAMPLE 3 Health and Graduate School Fair What is your program or service? Health and Grad School Fair is held each fall in a large ballroom. It is a time of information networking and sharing regarding educational opportunities for college graduates. Who are your intended learners? Undergraduate students may or may not have decided to attend grad school. In addition, grad school professionals, grad students, plan B students, and admissions directors are a part of the audience. Page 5

6 YOUR TURN! TRY IT NOW. Use these stems, as well as the prompts on the previous page, to clearly describe your program / service and intended learners. It may even help to think of the 4 W s and H of journalism Who, What, Where, When and How. a) Who is your program / service designed for? b) What activities are included in your program / service? c) When does your program / service occur? d) Where does your program / serviced occur? e) How is your program / service implemented? What resources (time, financial, and staff) are needed? What is your program or service? Who are your intended learners? Page 6

7 Step 2. Brainstorm Outcomes Beginning with the end in mind is a useful way to think about the intended outcomes of your specific program / service for your specific learners. Use the questions provided here to stimulate conversation and brainstorming. a) What should (intended learners) know, be able to do, be able to demonstrate, value, or feel when they have completed (program/service)? b) What difference should (program/service) have made for (intended learners) participate? EXAMPLE 1 Resume Review Services What should undergraduate students know, be able to do, be able to demonstrate, value, or feel when they have completed a drop-in resume review? Know the characteristics of a well formatted resume (Know) Demonstrate accomplishments in their descriptions of past work experiences (Do / Demonstrate) Feel confident in their ability to create a resume that highlights their strengths and abilities (Feel) EXAMPLE 2 EPICS What should undergraduate students know, be able to do, be able to demonstrate, value, or feel when they have completed online modules in the EPICS program? Page 7

8 Understand how exploring their interests helps them narrow down potential major choices (Know) Use resources to identify skills needed in their potential majors (Do / Demonstrate) Feel encouraged that they can choose a major that will be a good fit for them (Feel) EXAMPLE 3 Health and Graduate School Fair What should undergraduate students be able to do, be able to demonstrate, value, or feel when they have attended the Graduate and Professional School Fair? Understand the requirements for professional and graduate school programs (Know) Identify and implement next steps to prepare for graduate school (Do / Demonstrate) Feel confident talking with professional and graduate school representatives and exploring options (Feel) YOUR TURN! TRY IT NOW. Try out these different stems to brainstorm options. You do not need to fill out responses for each one the variety simply provides several ways of thinking about intended outcomes. Use the stems that make the most sense to you. What should your intended learners KNOW as a result of your program / service? Page 8

9 What should your intended learners BE ABLE TO DO OR DEMONSTRATE as a result of your program / service? What should your intended learners VALUE / FEEL as a result of your program / service? What DIFFERENCE should your program / service have made for your intended learners? Page 9

10 Step 3a. Write Outcomes Statements First Round The next step is to transform your brainstormed outcomes into meaningful learning outcomes statements. Try this formula* to get you started: (Intended Learners) (Action Verb 1) (Program/ Service) (Action Verb 2) (Intended Outcome) Complete Engage in Participate in Demonstrate Describe Discuss Explain Identify List Summarize * Keeling & Associates (2007) EXAMPLE 1 Resume Review Services Intended Learners Action Verb 1 Program / Service Action Verb 2 Intended Outcome 1. Under- drop-in a clear target or graduate engage in resume review articulate purpose for their students services resume. EXAMPLE 2 EPICS Page 10

11 Intended Learners Action Verb 1 Program / Service Action Verb 2 Intended Outcome 1. resources to strengthen Under- EPICS communication graduate complete Competency use skills needed in students Builder their potential work environments. EXAMPLE 3 Graduate and Professional School Fair Intended Learners Action Verb 1 Program / Service Action Verb 2 Intended Outcome 1. at least three the educational Juniors Graduate options beyond and attend and identify their Seniors Professional undergraduate School Fair degree. Page 11

12 ADDITIONAL TIPS FOR WRITING OUTCOME STATEMENTS Smart Objectives Quality outcomes statements share the characteristics of Smart objectives they are clear, specific, and measurable, as well as attainable, given particular conditions, such as time, relevance and resources. S Specific An objective should address a specific target or accomplishment. Specific implies that an observable action, behavior or achievement is described which is also typically linked to an identifiable change in rate, number, percentage or frequency. M Measurable A metric should be established that can indicate whether an objective has been met. That is, there should be a system, method or procedure for tracking and recording the change in behavior or action upon which the objective is directed. A Achievable Though not necessarily easy to conceptualize, the objective should be feasible. That is, it should be capable of being achieved. Objectives should be limited to what can realistically be done with available resources. R Relevant An objective should be significant (relevant) to the people involved in the program (from beneficiaries to the program s sponsoring organization); and, the objective should be capable of having an impact or make a change. T Time-based An objective should be achievable within a specific time frame. Generally, this takes the form of a start and end date. Page 12

13 YOUR TURN! TRY IT NOW. Use the chart below to fill-in-the-blanks on your way to writing a high-quality outcomes statement. Remember, this formula is intended to be a conversation starter. As your group gets rolling, feel free to break away from the structure if it does not fit your needs. Intended Learners Action Verb 1 Program / Service Action Verb 2 Intended Outcome Page 13

14 Step 3b. Write Outcomes Statements Second Round Now, let s take the outcomes statements one step further. Think about the different levels of complexity offered by Bloom s Taxonomy (see figure below). Each level requires a different degree of engagement, processing, and synthesis from your intended learners. Ideally, student development programs will encourage learners to achieve higher levels of cognitive performance. In this step, challenge yourself to set learning outcomes that are progressively more challenging and engaging for your students. Remember, that not all outcomes must come about immediately. Some may occur as an immediate result of your program/service, others will require time to incubate, while still others may be achieved by only a few of your learners. This is brainstorming practice, so do not hesitate to think big! Benjamin Bloom s Taxonomy: Levels of Student Performance* * Bloom (1956) Page 14

15 EXAMPLE Resume Review Services Intended Learners Action Verb 1 Program / Service Action Verb 2 Intended Outcome appropriate Under- drop-in Knowledge graduate engage in resume review list content for the education section students services of a resume when they might Comprehension Undergraduate engage in drop-in resume review describe use an objective statement on students services their resume strong, varied Under- drop-in Application graduate engage in resume review use action verbs to describe their students services experiences Page 15

16 the fit of the content of their Under- drop-in resume with the Analysis graduate engage in resume review compare requirements of students services target employers / positions accomplishments Under- drop-in / results into Synthesis graduate engage in resume review integrate descriptions of students services their experiences a combination of Under- drop-in past experiences Evaluation graduate engage in resume review select that best fit the students services intended target for their resume Page 16

17 Sample Action Verbs using Bloom s Taxonomy Knowledge Comprehension Application Analysis Synthesis Evaluation arrange define duplicate label list memorize name order recall recognize relate repeat reproduce state classify describe discuss explain express identify indicate locate recognize report restate review select translate apply choose demonstrate dramatize employ illustrate interpret operate practice schedule sketch solve use write analyze appraise calculate categorize compare contrast criticize differentiate discriminate distinguish examine experiment question test arrange assemble collect compose construct create design formulate integrate manage organize plan prepare propose appraise argue assess attach choose compare estimate evaluate judge predict rate score select support write YOUR TURN! TRY IT NOW. Now try taking the next step with your outcomes statements. Write outcome statements that reflect a growing level of complexity. Want some hints? Use the sample action verbs for Bloom s Taxonomy table provided above. Intended Learners Action Verb 1 Program / Service Action Verb 2 Intended Outcome Knowledge Page 17

18 Intended Learners Action Verb 1 Program / Service Action Verb 2 Intended Outcome Comprehension Application Analysis Synthesis Evaluation Page 18

19 Intended Learners Action Verb 1 Program / Service Action Verb 2 Intended Outcome Page 19

20 Step 4a. Connecting Theories, Models, and / or Standards First Round So far, we have been considering our programs and services on their own as if they were an island, separated by the sea from outside influences. Yet, in reality, our individual programs and services act within the wider field of student development. As student affairs professionals, we naturally create bridges between the work that we do and the theories, mental models, and standards of our profession. These professional links are accompanied by hopes, biases, expectations, and values. In this step, let s connect our unique programs and intended learners to the student development field as a le. Begin by considering the theories, models, and professional standards that influence (or should influence ) your practice. Use the questions here to stimulate conversation and brainstorming. a) What are some of the theories, models, and / or professional standards that you gravitate to when thinking about (program/service) for (intended learners)? b) What are the key components of these theories, models, and / or professional standards? EXAMPLE Resume Review Services a) What are some of the theories, models, and / or professional standards that you gravitate to when thinking about drop-in resume reviews for undergraduate students? National Career Development Guidelines Career Management Domain Social Cognitive Career Theory b) What are the key components of these theories, models, and / or professional standards? National Career Development Guidelines Career Management Domain Page 20

21 - Identify, demonstrate, and evaluate personal job seeking skills, such as resume writing - Identify, demonstrate, and evaluate ways to be proactive in marketing yourself in the workplace - Social Cognitive Career Theory - The influence of self-efficacy and outcome expectations on personal goals - Performance trials informing self-efficacy and outcome expectations - YOUR TURN! TRY IT NOW. Use the prompts below, as well as the discussion the previous page, to identify the components of theory, models, and/or professional standards that your programs and services are grounded in. Name a theory, model, or professional standard document that you gravitate to when thinking about (program/service) for (intended learners). What are the key components of that theories, models, and / or professional standards document? Page 21

22 Page 22

23 Step 4b. Connecting Theories, Models, and / or Standards Second Round Once you have identified the resources that you turn to, you can begin to explore how they inform the learning outcomes for your unique programs, services, and intended learners. This step is difficult to accomplish in a workshop setting. Getting a good grasp on how theories, models, and professional standards influence practice often requires that we are able to dig into resources and references. The length of these standards documents, and the variety of theoretical preferences, make this difficult to accomplish for each individual program or service in a large group. So, here are some questions to stimulate conversation and brainstorming when you return to your offices. a) How do your learning outcomes currently speak to these theories*? b) Would you change or re-word any of your learning outcomes based on these theories? c) What learning outcomes might you add based on theories? d) How might your preferred theories help justify or support your choice of learning outcomes? * To make these questions easier to read, we have used only the word theories here. Please note that these same questions should also be considered in regards to models and professional standards. Page 23

24 References and Helpful Resources American College Personnel Association. & National Association of Student Personnel Administrators. (2004). Learning reconsidered: A campus-wide focus on the student experience. Washington, DC: Author. Bloom, B. S. (1956). Taxonomy of Education Objectives: Handbook I: The cognitive domain. New York: David McKay Co Inc. Bresciani, M. J., Zelna, C. L, & Anderson, J. A. (2004). Assessing student learning and development: A handbook for practitioners. Washington, DC: National Association of Student Personnel Administrators. Keeling, R. P. (Ed.). (2006). Learning reconsidered 2: A practical guide to implementing a campus-wide focus on the student experience. Washington, DC: American College Personnel Association. Keeling, R. P. & Associates. (2007, June). Putting learning reconsidered into practice: Developing and assessing student learning outcomes. Workshop presented at the National Association of Student Personnel Administrators Learning Reconsidered Institute in St. Loius, MO. Miller, T. K. (Ed.). (2003). The Professional Standards for Higher Education (3rd ed.). Washington, DC: Council for the Advancement of Standards in Higher Education. Schuh, J. H., Upcraft, M. L., & Assoc. (2001). Assessment practice in student affairs: An applications manual. San Francisco, CA: Jossey Bass. Page 24