SLIDE 1 - Logic Modeling: A Guide for Program Design and Evaluation

Transcription

1 SLIDE 1 - Logic Modeling: A Guide for Program Design and Evaluation SLIDE 2 - Session Goals - Define Logic Modeling - Discuss benefits and limitations of Logic Modeling - Examine detailed Logic Model - Practice Logic Model exercise - Every park and recreation agency, whatever the focus or field of operation, is rightfully concerned with the efficiency and effectiveness of its operation. With the importance of park and recreation agencies to the quality of life each agency has an essential role in the lives of the people it serves. Proper planning and evaluation of all programs and services is paramount in the success of a well-managed agency. Every agency should have a planning and evaluation strategy for all levels of programs and services. There should be a systemic approach to ensure continuity and consistency. Just to get started, by a show of hands, who falls into the following categories? - I have created Logic Models before - I have attended other Logic Model trainings - I am new to Logic Models - Our programs have clear outcomes that we all agree to, and don t have a need for Logic Modeling What is the Logic Model? The Logic Model process is a tool that has been used for more than 20 years by program managers and evaluators to describe the effectiveness of their programs. There are many ways to describe the process and there is a great deal of literature available to explain the process. For example, The Academy for Education and Development has something called Program Logic Modeling: Creating a Blueprint for Success. The American Evaluation Association teaches a course entitled Program Evaluation:

2 The W.K. Kellogg Foundation may have the most comprehensive guide in a document entitled, Logic Model Development Guide. I encourage you to familiarize yourself with as much information, and research as many resources as you can, because everyone s approach and description is a little different. Today, we re going to focus on the University of Wisconsin Extension, Program Development and Evaluation model. It s the model and approach I find the most user-friendly for me. SLIDE 3 - A logic model is - A depiction of a program showing what the program will do and what it is to accomplish - A series of if-then relationships that, if implemented as intended, lead to desired outcomes - The core of program planning and evaluation SLIDE 4 A logic model is a systematic and visual way to present and share your understanding of the relationships among resources you have to operate your program, the activities you plan to do, and the change results you hope to achieve. W. K. Kellogg Foundation Evaluation Handbook (1998) In its simplest form a logic model looks like this SLIDE 5 - Inputs Outputs Outcomes Logic models can be complex or simple and applied to any level of programs or problems: - A small program ( tumbling class at a recreation center) - A process (a team working together to design a program brochure) - A large, multi-component program (10K race) - An organization or business (reframing the Operations Division) SLIDE 6 A bit more complex version 2

3 Logic Modeling is a tool or framework that helps identify the program or project resources, activities, outputs, and outcomes. You can even call it a roadmap. SLIDE 7 - It should answer the following questions: - Where are you going? - How will you get there? - What will show that you ve arrived? SLIDE 7 - Yogi Berra said it best: If you don t know where you re going, how are you gonna know when you get there? What s so important about using something like a Logic Model to guide you in program development and evaluation? As a public agency, we work in a transparent and highly accountable environment. We are entrusted with the wise use of tax dollars. And now more than ever we live in an age of accountability. SLIDE 8 - In this era of accountability: - What gets measured gets done - If you don t measure results, you can t tell success from failure - If you don t see success, you can t reward it - If you can t reward success, you re probably rewarding failure - If you can t see success, you can t learn from it - If you can t recognize failure, you can t correct it - If you can demonstrate results, you can win public support Reinventing Government, Osborne and Gaebler, 1992 Logic Modeling is a framework for describing the relationships between investments, activities, and results. It provides a common approach for integrating planning, implementation, evaluation and reporting. SLIDE 9 - Some of the benefits of Logic Modeling - Helps managers focus on and become accountable for what matters OUTCOMES - Provides a common language 3

4 - Supports continuous improvement - Promotes communication - Improves documentation of process Logic Modeling is a way of thinking, not just a pretty diagram. Using the basic model, here are everyday examples of how a Logic Model works. SLIDE 10 - Headache Get pills Take pills Feel better Situation Inputs Outputs Outcomes SLIDE 11 - Family Vacation Family members Drive to state Family members learn Budget park about each Car Set up camp other; family Camping Cook, play, bonds; family Equipment talk, hike has good time Underlying a Logic Model is a series of if-then relationships that express the program s theory of change. HANDOUTS #1 and #2 SLIDE 12 - Theory of Change is defined as:... A description of how and why a set of activities be they part of a highly focused program or a comprehensive initiative are expected to lead to early, intermediate, and long term outcomes over a specified period. (Andrea Anderson, Aspen Institute Roundtable, 2000) Let s look at an example of how the if-then statements work and how activities will lead to desired outcomes. SLIDE 13 - Tutoring Program Example IF then IF then IF then IF then IF then We invest We can provide students struggling They will they will They will Time tutoring for academically can learn and get better move to 4

5 And money 3 hrs/wk be tutored improve grades next for 1 school year their skills level on to 50 children time HANDOUTS #9 and #10 SLIDE 14 - There is a logical chain of connections showing what a program is to accomplish. INPUTS OUTPUTS OUTCOMES Program Investments Activities Participation Outputs Short Term Long Term So what? What is the value? A common problem is that activities and strategies often do not lead to the desired outcomes. Check your if-then statements and outcomes you want to achieve. ensure that they make sense and lead to A Logic Model makes the connections explicit. HANDOUT #19 SLIDE 15 - Show cartoon I think you should be more specific in Step Two. SLIDE 16 - Show a fully detailed logic model The first critical step in logic model development is defining the situation. HANDOUTS #22, 23, 24, 25 SLIDE 17 - What problematic condition exists that demands a programmatic response? - Why does it exist? - For whom does it exist? - Who has a stake in the problem? - What can be changed? - What additional information is needed? 5

6 SLIDE 18 Getting Started with the Logic Model HANDOUT #36 If the situation is incorrectly understood from it will be wrong. and diagnosed, everything that flows SLIDES 19 and 20 Approaches to Creating a Logic Model HANDOUT #40 Once a Logic Model is in place, it is important to always check it to see if it s accurate. - Is it meaningful? - Does it make sense? - Is it doable? - Can it be verified? Test your model by reviewing it in reverse. After you look at your goals, do the methods for achieving them make sense? SLIDE 21 - Limitations to the Logic Model: - It represents intention, not reality - Focuses on expected outcomes - Many factors influence process and outcomes - Doesn t address: Are we doing the right thing? - Can become too time consuming and just paperwork - May become too focused on outcomes without adequate attention to inputs and outputs and the logical relationships that connect them to end results - May end up perfecting the key to the wrong lock - Thinking that the logic model has to be correct - Becomes fixed rather than dynamic SLIDE 22 - Additional benefits of Logic Modeling - It provides for a common language - It helps differentiate between what we do and results OUTCOMES - It increases understanding about a program - It guides and helps focus work - It leads to improved planning, management and evaluation - It increases intentionality and purpose - It provides coherence across complex tasks and diverse environments - It enhances teamwork - Guides prioritization and allocation of resources 6

7 - Motivates staff - Helps to identify important variables to measure and evaluate - It increases resources, opportunities, and recognition - It supports replication - It is often required! SLIDES 23 and 24 - Logic models are essential to program evaluation. - It provides the program description that guides our evaluation process - It helps match evaluation to the program - It helps us know what and when to measure (Are you interested in the process and/or the outcomes?) - Helps us focus on key and important information SLIDE 25 Visual Presentation by Florence Corpus Practice Logic Model Exercise: Parent Education Program Creating Logic Models help staff focus on programs and services generating outcomes for participants or stakeholders. With the completion of a detailed and accurate logic model, staff can be confident that their efforts will be effective and their resources well spent. The time spent developing a Logic Model should be viewed as an investment rather than an expense. Given the public s expectations for performance, park and recreation professionals can ill afford not to use Logic Models as a tool for program planning. HANDOUT #51 Revised 3/20/2013 7